I am looking for the ideal/thread safe implementation of a task queue that will start each task based on an internal condition (which i also only want to be checked at a timed interval), but also check that there aren't too many other tasks running. I also want a way to return progress of each task in the queue.
To give some background, I have a FileSystemWatcher that sends an event on a new folder being made. I then want to put this into a queue to process the folder, when it is no longer locked. But at the same time I want to guard against too many instances of the Process task from running.
Heres some pseudo-code of what I am after.
private static void EventToQueue(object sender, EventArgs e)
{
ProcessQueue.Enqueue(Task, condition);
}
private static async void NewQueueObject(object sender, QueueObject e)
{
if (e.Condition && ProcessQueue.ActiveTask < 4)
{
var progress = new Progress<int>();
progress.ProgressChanged += ( s, e ) =>
{
UpdateProgress(e.Value);
};
await LongProcess(e, progress);
}
else
{
e.Delay(30);
}
}
Related
I have a list with hundreds items,
when I try to sort the items, it always blocking my program.
It' so too annoying.
I try to use Task.Run(), It doesn't work.
private void resultListView_ColumnClick(object sender, ColumnClickEventArgs e)
{
// doesn't work
Task.Run(() => ListViewOperate((ListView)sender, ListViewOP.SORT));
// doesn't work
resultListView.BeginUpdate();
ListViewOperate((ListView)sender, ListViewOP.SORT);
resultListView.EndUpdate();
}
private delegate void ListViewOperateDelegate(System.Windows.Forms.ListView liv,
ListViewOP op);
private void ListViewOperate(System.Windows.Forms.ListView liv, ListViewOP op)
{
if (liv.InvokeRequired)
{
var addDele = new ListViewOperateDelegate(ListViewOperate);
liv.Invoke(addDele, new object[] { liv, op});
}
else
{
if (op == ListViewOP.SORT)
{
liv.BeginUpdate();
liv.Sort();
liv.EndUpdate();
}
}
}
The trick you are trying to use, doesn't work here, because you are starting the
ListViewOperator on a thread/task but the callback checks if it is started on the UI thread. If it isn't on the UI thread, invoke it on the UI thread. So what benefits did you gain? none..
You should check the Virtual Mode. With the virtual mode you are able to sort it (a copy) of the list on a task/thread and then swap the list "variable" and trigger the listview to refresh it self.
Here is an Example
I have a C# Windows Service that runs a few tasks inside.
One of the tasks is a infinite async looping and the others are triggered from a Timer and then execute the task.
private readonly QueueProcessor _queueProcessor;
protected override void OnStart(string[] args)
{
// first task
_queueTask = _queueProcessor.Run(_cancellation.Token);
// second task
affiliate_timer = new System.Timers.Timer();
affiliate_timer.AutoReset = true;
affiliate_timer.Interval = _model.Interval_Affiliate * 60000;
affiliate_timer.Elapsed += new
System.Timers.ElapsedEventHandler(affiliate_timer_Elapsed);
// third task
invoice_timer = new System.Timers.Timer();
invoice_timer.AutoReset = true;
invoice_timer.Interval = _model.Interval_Invoice * 60000;
invoice_timer.Elapsed += new
System.Timers.ElapsedEventHandler(invoice_timer_Elapsed);
}
private void invoice_timer_Elapsed(object source, System.Timers.ElapsedEventArgs e)
{
if (!_isAffiliateBusy)
{
_isAffiliateBusy= true;
var task = Task.Run(() => StartAffiliateTask());
task.Wait();
_isAffiliateBusy= false;
}
}
private void invoice_timer_Elapsed(object source, System.Timers.ElapsedEventArgs e)
{
if (!_isInvoiceBusy)
{
_isInvoiceBusy = true;
var task = Task.Run(() => StartInvoiceTask());
task.Wait();
_isInvoiceBusy = false;
}
}
private void StartAffiliateTask()
{
_affiliateModule = new Modules.Affiliate();
_affiliateModule.RunSync();
}
private void StartInvoiceTask()
{
_invoiceModule = new Modules.Invoice();
_invoiceModule.RunSync();
}
This is my QueueProcessor class that implements await/async to execute a infinite looping job:
public class QueueProcessor
{
private readonly IQueueBroker _serviceBroker;
public QueueProcessor()
{
}
public async Task Run(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
var receiveMessageResponse = await _serviceBroker.ReceiveMessageAsync("test", cancellationToken);
if (!receiveMessageResponse.Messages.Any())
{
continue;
}
foreach (var message in receiveMessageResponse.Messages)
{
// some other tasks here...
await _serviceBroker.DeleteMessageAsync(message, cancellationToken);
}
}
}
}
My Affiliate and Invoice module classes doesn't implement any await/async code inside looks like this:
public class Affiliate
{
/// <summary>
/// Start the sync process
/// </summary>
public void RunSync()
{
try
{
// some code here...
}
catch (Exception ex)
{
}
}
}
My question is:
When my queue procesor infinite loop is running, does my other tasks that are triggered by the timers still can run independently?
When I use:
var task = Task.Run(() => StartAffiliateTask());
task.Wait();
Does the Wait method stop the whole service thread until this task is finished? or that won't block my StartInvoiceTask to run independantly?
Any recommendation on the best way to have my 3 tasks running independant on each other?
Summing up multiple potential issues:
Race condition (access/write to _isBusy).
Potential deadlock (in low ThreadPool size).
Potential incosistent state of flag in case of errors or thread aborts (_isBusy can be left in 'true' state).
Further I will assume your 'task' should be running in single instance, so we will disgard timer callbacks if it is still running.
You should change your timer event handlers like so (best to just wrap it in some kind of class):
//the flag, do mention volatile modifier - it tells particular
//systems to watch for variable changes by reference,
//instead of just copying it into thread stack by value.
private volatile bool _isAffiliateBusy = false;
//sync object for flag to eliminate race condition
private object _affiliateSync = new object();
private void affiliate_timer_Elapsed(object source, System.Timers.ElapsedEventArgs e)
{
//very fast lookup at flag to filter threads which comes when task is still processing
if(_isAffiliateBusy)
return;
lock(_affiliateSync) //taking lock
{
//checking again for those threads which 'happen' to be faster than you think.
if(_isAffiliateBusy)
return;
//aquire lock for business 'task'
_isAffiliateBusy = true;
}
try
{
StartAffiliateTask();
}
finally
{
//resetting singleton business 'task' lock.
//do not forget to use finally block, to handle disposing
//even if something rise up in 'try' section - you will not be left with invalid state of flag.
_isAffiliateBusy = false;
}
}
As far as I can see AutoResetEvent and ManualEvent simply control a single signal for cross-thread communication.
Often you would want to know some sort of result in the waiting thread and I can only see the option of storing a class member which is accessed by each thread e.g.:
this.WaitHandle = new AutoResetEvent(false);
DoStuff();
WaitHandle.WaitOne();
lock(this._lock)
{
if(this.Result ...){...}
}
void DoStuff()
{
...
lock(this._lock)
{
this.Result = ...;
}
this.WaitHandle.Set();
}
This is a bit cumbersome so I wondered if there is something built in that will let me pass a value when signalling?
The best approach would be to use the async patterns consistently in all your code.
If you want to actually block execution and wait for the result to be available in a blocking way using traditional methods, you can use a BlockingCollection<T> as a channel to both communicate the result of the operation and that the operation is finished at the same time.
private void button1_Click(object sender, EventArgs e)
{
BlockingCollection<string> blockingQueue = new BlockingCollection<string>();
// Start DoStuff on parallel thread
Task.Run(() => DoStuff(blockingQueue));
// Wait for DoRun to finish AND get the result at the same time
string result = blockingQueue.Take();
MessageBox.Show(result);
}
private void DoStuff(BlockingCollection<string> result)
{
// Simulate work
Thread.Sleep(1000);
// return result
result.Add("SomeResultValue");
}
This also allows you to have two threads adding stuff to the queue and retrieving stuff from the queue at the same time in parallel, the consumer always blocking as long as there is no new data available.
private void button1_Click(object sender, EventArgs e)
{
BlockingCollection<string> blockingQueue = new BlockingCollection<string>();
// Start DoStuff on parallel thread
Task.Run(() => DoStuff(blockingQueue));
// Wait for something to be returned from DoStuff and handle it
foreach (string data in blockingQueue.GetConsumingEnumerable())
{
textBox1.AppendText(data + Environment.NewLine);
}
MessageBox.Show("Finished");
}
private void DoStuff(BlockingCollection<string> result)
{
for (int i = 1; i <= 10; i++)
{
// Simulate work
Thread.Sleep(1000);
// return result
result.Add("Result number " + i);
}
// Signal we are done
result.CompleteAdding();
}
If you use a BlockingCollection with BoundedCapacityset to 1, trying to add to the collection would actually wait until the previous value has been removed.
I am trying to create a way to queue up Tasks to run, so I have tried to implement it using a BlockingCollection. The problem I find is whenever I try to add the Task, the Task executes. Sample code as below:
private void button1_Click(object sender, EventArgs e)
{
textBox2.Clear();
for (int i = 0; i < 10; i++)
_processCollection.Add(BigTask(i));
}
static BlockingCollection<Task> _processCollection = new BlockingCollection<Task>();
Thread ConsumerThread = new Thread(LaunchConsumer);
private static async void LaunchConsumer()
{
while (true)
{
var processTask = _processCollection.Take();
await Task.Run(() => processTask);
}
}
async Task BigTask(int i)
{
await Task.Delay(5000);
textBox2.AppendText($"Text{i}\n");
}
What seems to happen in debug is all the tasks seem to run as they are added into the blocking collection. I tried switching the blocking collection to use Action, but that just leads to nothing happening. As below (only changes shown):
private void button1_Click(object sender, EventArgs e)
{
textBox2.Clear();
for (int i = 0; i < 10; i++)
{
int iC = i;
_processCollection.Add(async () => await BigTask(iC));
}
}
static BlockingCollection<Action> _processCollection = new BlockingCollection<Action>();
Thread ConsumerThread = new Thread(LaunchConsumer);
private static async void LaunchConsumer()
{
while (true)
{
var processTask = _processCollection.Take();
await Task.Run(processTask);
}
}
I feel like I have made some small error somewhere, because it feels like this should work. I have tried to find someone doing something similar but have had no luck, which makes me think maybe my concept is flawed so feel free to suggest an alternative.
_processCollection.Add(BigTask(i)); doesn't work because this calls BigTask(i) immediately, and when that is called, the work starts.
You were on the right track by wrapping this in a separate BigTask launcher, but by using Action, you don't provide your LaunchConsumer with any means to track the progress. await Task.Run(processTask) will continue pretty much immediately with the next task. You need to use Func<Task> to avoid that.
The reason you don't see any results is likely unrelated. Now that you manage to launch the task from your newly created thread, the call to textBox2.AppendText is no longer done from the UI thread. That's not supported. Only the UI thread can access UI objects. You can use textBox2.Invoke to pass an action back to the UI thread, and that action can then call AppendText.
Tested working code:
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
ConsumerThread.Start();
}
private void button1_Click(object sender, EventArgs e)
{
textBox2.Clear();
foreach (var i in Enumerable.Range(0, 10))
_processCollection.Add(() => BigTask(i));
}
static BlockingCollection<Func<Task>> _processCollection = new BlockingCollection<Func<Task>>();
Thread ConsumerThread = new Thread(LaunchConsumer);
private static async void LaunchConsumer()
{
while (true)
{
var processTask = _processCollection.Take();
await Task.Run(processTask);
}
}
async Task BigTask(int i)
{
await Task.Delay(5000);
textBox2.Invoke(new Action(() => textBox2.AppendText($"Text{i}\n")));
}
}
That said, BlockingCollection is not really the best collection type to use here. It dedicates one thread to pretty much nothing but waiting. Also, Task.Run when you're already in a background thread can admittedly sometimes be useful, but doesn't add anything here. What to do instead depends on your needs. Whether all tasks are known beforehand makes a difference. Whether you may want multiple consumers makes a difference. Other things I haven't thought of may also make a difference.
Never tried asynch calling with Windows Form. I cannot use the new aynch/await because I don't own recent Visual Studio/.NET. What I need is "execute an operation that request long time (populate an IList)" and, when it has finished, write on a TextBox the result of that list.
Searching on Internet I found this example that seems work, but TOO comply in my opinion (maybe there are somethings fast and simple):
private void button1_Click(object sender, EventArgs e)
{
MyTaskAsync();
}
private void MyTaskWorker()
{
// here I populate the list. I emulate this with a sleep of 3 seconds
Thread.Sleep(3000);
}
private delegate void MyTaskWorkerDelegate();
public void MyTaskAsync()
{
MyTaskWorkerDelegate worker = new MyTaskWorkerDelegate(MyTaskWorker);
AsyncCallback completedCallback = new AsyncCallback(MyTaskCompletedCallback);
AsyncOperation async = AsyncOperationManager.CreateOperation(null);
worker.BeginInvoke(completedCallback, async);
}
private void MyTaskCompletedCallback(IAsyncResult ar)
{
MyTaskWorkerDelegate worker = (MyTaskWorkerDelegate)((AsyncResult)ar).AsyncDelegate;
AsyncOperation async = (AsyncOperation)ar.AsyncState;
worker.EndInvoke(ar);
AsyncCompletedEventArgs completedArgs = new AsyncCompletedEventArgs(null, false, null);
async.PostOperationCompleted(delegate(object e) { OnMyTaskCompleted((AsyncCompletedEventArgs)e); }, completedArgs);
}
public event AsyncCompletedEventHandler MyTaskCompleted;
protected virtual void OnMyTaskCompleted(AsyncCompletedEventArgs e)
{
if (MyTaskCompleted != null)
MyTaskCompleted(this, e);
// here I'll populate the textbox
textBox1.Text = "... content of the Iteration on the List...";
}
really I NEED somethings like 50 lines of code for this easy operation? Or I can remove some stuff? I just need a simple asynch call->callback when finished.
No lock, no concurrency at all...
You can use the TPL with C# 4.0 like so:
private void button1_Click(object sender, EventArgs e)
{
Task.Factory.StartNew(() => DoWork())
.ContinueWith(t => UpdateUIWithResults(t.Result)
, CancellationToken.None
, TaskContinuationOptions.None
, TaskScheduler.FromCurrentSynchronizationContext());
}
This starts DoWork in a thread pool thread, allowing it to do processing out of the UI thread, then runs UpdateUIWithResults in a UI thread, passing it the results of DoWork.
You can use Task.Factory.StartNew to push work onto the thread pool. Task.ContinueWith will give you a "completed callback".
private void button1_Click(object sender, EventArgs e)
{
var ui = TaskScheduler.FromCurrentSynchronizationContext();
Task<List<T>> task = Task.Factory.StartNew(() => MyTaskWorker());
task.ContinueWith(t => OnMyTaskCompleted(t), ui);
}
private List<T> MyTaskWorker()
{
// here I populate the list. I emulate this with a sleep of 3 seconds
Thread.Sleep(3000);
return ...;
}
protected virtual void OnMyTaskCompleted(Task t)
{
// here I'll populate the textbox with t.Result
textBox1.Text = "... content of the Iteration on the List...";
}