I have the following lines in my code:
var taskA = Task.Factory.StartNew(WorkA);
var taskB = Task.Factory.StartNew(WorkB);
var allTasks = new[] { taskA, taskB };
Task.Factory.ContinueWhenAll(allTasks, tasks => FinalWork(), TaskContinuationOptions.OnlyOnRanToCompletion);
But when I run this, I get the following error:
It is invalid to exclude specific continuation kinds for continuations off of multiple tasks.
Which is caused by the option TaskContinuationOptions.OnlyOnRanToCompletion.
My question is how to check that all tasks have done their work properly (all tasks statuses are RanToCompletion) and then do FinalWork()?
In the meantime, the application performs other tasks.
Based on #Peter Ritchie and #Ben McDougall answers I found a solution. I modified my code by removing redundant variable tasks and TaskContinuationOptions.OnlyOnRanToCompletion
var taskA = Task.Factory.StartNew(WorkA);
var taskB = Task.Factory.StartNew(WorkB);
var allTasks = new[] { taskA, taskB };
Task.Factory.ContinueWhenAll(allTasks, FinalWork);
Where FinalWork is:
private static void FinalWork(Task[] tasks)
{
if (tasks.All(t => t.Status == TaskStatus.RanToCompletion))
{
// do "some work"
}
}
If all tasks have status RanToCompletion, "some work" will be done. It will be performed immediately after all tasks have completed and will not block the main task.
If I cancel at least one of the tasks, nothing will be done.
Alternatively you can do this,
var taskA = Task.Factory.StartNew(WorkA);
var taskB = Task.Factory.StartNew(WorkB);
var allTasks = new[] { taskA, taskB };
var continuedTask = Task.WhenAll(allTasks).ContinueWith((antecedent) => { /*Do Work*/ }, TaskContinuationOptions.OnlyOnRanToCompletion));
You haven't provided any code that does anything with any of the tasks that ran to completion (your tasks variable is ignored). You'd get the same result if you simply removed TaskContinuationOptions.OnlyOnRanToCompletion. i.e. If you could use ContinueWhenAll with TaskContinuationOptions.OnlyOnRanToCompletion your continuation isn't going to be called until all tasks have either completed or failed. If you don't do anything with just the completed tasks, that's the same as Task.Factory.ContinueWhenAll(allTasks, tasks => FinalWork());
If there's something more specific that you want to do, please provide the details so that someone might be able to help you.
To answer the actual question you posed:
My question is how to check that all tasks have done their work properly (all tasks statuses are RanToCompletion) and then do FinalWork()? In the meantime, the application performs other tasks.
at least that is what I read as the question check the following code:
var taskA = Task.Factory.StartNew(WorkA);
var taskB = Task.Factory.StartNew(WorkB);
var allTasks = new[] { taskA, taskB };
taskA.Wait();
taskB.Wait();
if (taskA.Status == TaskStatus.RanToCompletion && taskB.Status == TaskStatus.RanToCompletion)
Task.Factory.ContinueWhenAll(allTasks, tasks => FinalWork());
else
//do something
You actually answered the question yourself with your question if you did mean that.
Related
This question already has an answer here:
Task.WaitAll throws OperationCanceledException [closed]
(1 answer)
Closed 2 years ago.
I have a list of Tasks, the tasks do work on different databases then update a set of results on a central database.
If I can't connect to the central database the desired behaviour is to cancel all the tasks, wait for the running tasks to successfully stop (as it's a cooperative thing) then quit the program.
I've written a few lines of code to test cancelling tasks:
var cancellationTokenSource = new CancellationTokenSource();
var taskList = new List<Task>();
taskList.Add(new Task(() => { Thread.Sleep(5000); }, cancellationTokenSource.Token));
taskList.Add(new Task(() => { Thread.Sleep(5000); }, cancellationTokenSource.Token));
taskList.ForEach(task => task.Start());
cancellationTokenSource.Cancel();
Task.WaitAll(taskList.ToArray());
I know I should normally periodically check the cancellation token within the tasks executing code but for now I'm just testing my understanding of how cancelling works.
When I run it I get the error:
You can pass a cancellation token to WaitAll but thats for actually cancelling the WaitAll
Essentially I just want to wait until all tasks have either run to completion or stopped due to being cancelled so I can safely quit.
I feel like I'm probably missing something simple, if anyone could shed some light on how to accomplish this that would be really appreciated.
Many Thanks
The TaskCanceledException is an exception that is always thrown when you cancel a running Task. You simply need to handle it with a try-catch block. This goes against Microsoft's own philosophy of trying to avoid throwing exceptions when possible (Best Practices For Exceptions).
The easiest solution would look something like this:
var cancellationTokenSource = new CancellationTokenSource();
var taskList = new List<Task>
{
Task.Run(() => { Thread.Sleep(1); }),
Task.Run(() => { Thread.Sleep(5000); }, cancellationTokenSource.Token),
Task.Run(() => { Thread.Sleep(5000); }, cancellationTokenSource.Token)
};
cancellationTokenSource.Cancel();
try
{
Task.WhenAll(taskList).GetAwaiter().GetResult();
}
catch (TaskCanceledException)
{
Console.WriteLine("Tasks were cancelled");;
}
Console.WriteLine(taskList[0].Status);
Console.WriteLine(taskList[1].Status);
Console.WriteLine(taskList[2].Status);
First of all, avoid mixing blocking code (Task.WaitAll, Wait, ...) with nonblocking code; Task.WhenAll may be the the better choice. As is hinted at in the post in Amogh's answer here, you likely want to use this instead:
await Task.WhenAll(taskList.ToArray())
.ContinueWith(t => { }, TaskContinuationOptions.NotOnRanToCompletion);
For example, the following code will print Canceled, RanToCompletion and Faulted:
var cancellationTokenSource = new CancellationTokenSource();
var taskList = new List<Task>
{
new Task(() => { Thread.Sleep(1000); }, cancellationTokenSource.Token),
new Task(() => { Thread.Sleep(5000); }),
new Task(() =>
{
Thread.Sleep(3000);
throw new InvalidOperationException();
})
};
taskList.ForEach(task => task.Start());
cancellationTokenSource.Cancel();
await Task.WhenAll(taskList.ToArray())
.ContinueWith(t => { }, TaskContinuationOptions.NotOnRanToCompletion);
Console.WriteLine(taskList[0].Status);
Console.WriteLine(taskList[1].Status);
Console.WriteLine(taskList[2].Status);
That said, this only makes sense when you intend to observe the individual tasks afterwards. If you don't, embrace the exception(s) instead.
This is a possible duplicate of Task.WaitAll throws OperationCanceledException
Essentially what you need is
Task.WhenAll(taskA, taskB, taskC).Wait()
As mentioned in the answer above.
I have the following code:
myObject object1 = null;
Thread obj1Thread = new Thread(() => { object1 = _myService.GetMethod(variable1, variable2); });
obj1Thread.Start();
obj1Thread.Join();
myObject object2 = null;
Thread obj2Thread = new Thread(() => { object2 = _myService.GetMethod2(variable3, variable4); });
obj2Thread.Start();
obj2Thread.Join();
As far as I understand, this code will create 2 new threads, run the specified methods, pause the main thread until both these threads complete, and then continue execution.
Assuming what I say is correct, all fine so far.
Next I want to try this:
myObject object1 = null;
Thread obj1Thread = new Thread(async () => { object1 = await _myService.GetMethod(variable1, variable2); });
obj1Thread.Start();
obj1Thread.Join();
myObject object2 = null;
Thread obj2Thread = new Thread(async () => { object2 = await _myService.GetMethod2(variable3, variable4); });
obj2Thread.Start();
obj2Thread.Join();
Basically adding async and await to each thread.
The compiler accepts this change and it seems to run locally, but is this code ok, and is it likely to cause me any problems further down the line, for example will the threads get confused, fail to wait, mix up results etc.
I have a reasonably good understanding of async and a basic understanding of multi threading, and I cannot think of any reason why this would not work.
The code runs locally, but my worry is that under heavy load on the server issues may appear that were not present in a local version....
The compiler accepts this change and it seems to run locally, but is this code ok, and is it likely to cause me any problems further down the line, for example will the threads get confused, fail to wait, mix up results etc.
I have a reasonably good understanding of async and a basic understanding of multi threading, and I cannot think of any reason why this would not work.
Yes, this code will cause you problems. The threads are not waiting as you expect. You're passing an async void lambda to the Thread constructor, and that thread will exit as soon as it hits the await in that lambda, before it sets the object1/object2 variable. So it's entirely possible those variables remain null after the Join.
The proper solution, as FCin posted, is to use asynchronous concurrency. (I avoid the term "parallel" here to reduce confusion with the Parallel type and the Task Parallel Library). Asynchronous concurrency uses Task.WhenAll:
// Start both methods concurrently
var task1 = _myService.GetMethod(variable1, variable2);
var task2 = _myService.GetMethod2(variable3, variable4);
// (Asynchronously) wait for them both to complete.
await Task.WhenAll(task1, task2);
// Retrieve results.
myObject object1 = await task1;
myObject object2 = await task2;
Your code can be parallel and asynchronously awaited with a single line:
await Task.WhenAll(_myService.GetMethod(variable1, variable2), _myService.GetMethod2(variable3, variable4)).
That's all you need. No threads. No joining. If these methods are truely i/o, there will be no thread.
As always, must read: https://blog.stephencleary.com/2013/11/there-is-no-thread.html
If your methods return different results and need to be assigned to variables, then you can do this:
Task<int> task1 = _myService.GetMethod(variable1, variable2);
Task<string> task2 = _myService.GetMethod2(variable3, variable4);
// Both tasks started.
int a = await task1; // Wait for first task
string b = await task2; // If this already finished, it will just return the result
Your above code doesn't take advantage of threading, it blocks the first thread until it finishes then starts the second
obj1Thread.Join(); instructs your main thread to wait until obj1Thread exits before continuing. This means it will spin up obj1Thread then wait for it to complete, meaning you will:
create thread 1
Run thread 1
Exit thread 1
create thread 2
Run thread 2
Exit thread 2
You want to do:
myObject object1 = null;
Thread obj1Thread = new Thread(async () => { object1 = await _myService.GetMethod(variable1, variable2); });
obj1Thread.Start();
myObject object2 = null;
Thread obj2Thread = new Thread(async () => { object2 = await _myService.GetMethod2(variable3, variable4); });
obj2Thread.Start();
obj1Thread.Join();
obj2Thread.Join();
In the docs for TPL I found this line:
Invoke multiple continuations from the same antecedent
But this isn't explained any further. I naively assumed you could chain ContinueWiths in a pattern matching like manner until you hit the right TaskContinuationOptions.
TaskThatReturnsString()
.ContinueWith((s) => Console.Out.WriteLine(s.Result), TaskContinuationOptions.OnlyOnRanToCompletion)
.ContinueWith((f) => Console.Out.WriteLine(f.Exception.Message), TaskContinuationOptions.OnlyOnFaulted)
.ContinueWith((f) => Console.Out.WriteLine("Cancelled"), TaskContinuationOptions.OnlyOnCanceled)
.Wait();
But this doesn't work like I hoped for at least two reasons.
The continuations are properly chained so the 2nd ContinueWith gets the result form the 1st, that is implemented as new Task, basically the ContinueWith task itself. I realize that the String could be returned onwards, but won't that be a new task with other info lost?
Since the first option is not met, the Task is just cancelled. Meaning that the second set will never be met and the exceptions are lost.
So what do they mean in the docs when they say multiple continuations from the same antecedent?
Is there a proper patter for this or do we just have to wrap the calls in try catch blocks?
EDIT
So I guess this was what I was hoping I could do, note this is a simplified example.
public void ProccessAllTheThings()
{
var theThings = util.GetAllTheThings();
var tasks = new List<Task>();
foreach (var thing in theThings)
{
var task = util.Process(thing)
.ContinueWith((t) => Console.Out.WriteLine($"Finished processing {thing.ThingId} with result {t.Result}"), TaskContinuationOptions.OnlyOnRanToCompletion)
.ContinueWith((t) => Console.Out.WriteLine($"Error on processing {thing.ThingId} with error {t.Exception.Message}"), TaskContinuationOptions.OnlyOnFaulted);
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
}
Since this wasn't possible I was thinking I would have to wrap each task call in a try catch inside the loop so I wouldn't stop the process but not wait on it there. I wasn't sure what the correct way.
Sometimes a solution is just staring you in the face, this would work wouldn't it?
public void ProccessAllTheThings()
{
var theThings = util.GetAllTheThings();
var tasks = new List<Task>();
foreach (var thing in theThings)
{
var task = util.Process(thing)
.ContinueWith((t) =>
{
if (t.Status == TaskStatus.RanToCompletion)
{
Console.Out.WriteLine($"Finished processing {thing.ThingId} with result {t.Result}");
}
else
{
Console.Out.WriteLine($"Error on processing {thing.ThingId} - {t.Exception.Message}");
}
});
tasks.Add(task);
}
Task.WaitAll(tasks.ToArray());
}
What you did is to create a sequential chain of multiple tasks.
What you need to do is attach all your continuation tasks to the first one:
var firstTask = TaskThatReturnsString();
var t1 = firstTask.ContinueWith (…);
var t2 = firstTask.ContinueWith (…);
var t3 = firstTask.ContinueWith (…);
Then you need to wait for all the continuation tasks:
Task.WaitAll (t1, t2, t3);
I am writing a set of async tasks that go away an download and parse data, however I am running in to a bit of a blank with the next step where I am updating a database.
The issue is that for the sake of performance I am using a TableLock to load rather large datasets, so what I am wanting to do is have my import service wait for the first Task to return, start the import. Should another Task complete while the first import is running the process joins a queue and waits for the import service is complete for task 1.
eg.
Async
- Task1
- Task2
- Task3
Sync
- ImportService
RunAsync Tasks
Task3 returns first > ImportService.Import(Task3)
Task1 return, ImportService is still running. Wait()
ImportService.Complete() event
Task2 returns. Wait()
ImportService.Import(Task1)
ImportService.Complete() event
ImportService.Import(Task2)
ImportService.Complete() event
Hope this makes sense!
You can't really use await here, but you can wait on multiple tasks to complete:
var tasks = new List<Task)();
// start the tasks however
tasks.Add(Task.Run(Task1Function);
tasks.Add(Task.Run(Task2Function);
tasks.Add(Task.Run(Task2Function);
while (tasks.Count > 0)
{
var i = Task.WaitAny(tasks.ToArray()); // yes this is ugly but an array is required
var task = tasks[i];
tasks.RemoveAt(i);
ImportService.Import(task); // do you need to pass the task or the task.Result
}
Seems to me however that there should be a better option. You could let the tasks and the import run and add a lock on the ImportService part for instance:
// This is the task code doing whatever
....
// Task finishes and calls ImportService.Import
lock(typeof(ImportService)) // actually the lock should probably be inside the Import method
{
ImportService.Import(....);
}
There are several things bothering me with your requirements (including using a static ImportService, static classes are rarely a good idea), but without further details I can't provide better advice.
While this is likely not the most graceful solution, I would try launching the work tasks and have them place their output in a ConcurrentQueue. You could check the queue for work on a timer until all tasks are completed.
var rand = new Random();
var importedData = new List<string>();
var results = new ConcurrentQueue<string>();
var tasks = new List<Task<string>>
{
new Task<string>(() =>
{
Thread.Sleep(rand.Next(1000, 5000));
Debug.WriteLine("Task 1 Completed");
return "ABC";
}),
new Task<string>(() =>
{
Thread.Sleep(rand.Next(1000, 5000));
Debug.WriteLine("Task 2 Completed");
return "FOO";
}),
new Task<string>(() =>
{
Thread.Sleep(rand.Next(1000, 5000));
Debug.WriteLine("Task 3 Completed");
return "BAR";
})
};
tasks.ForEach(t =>
{
t.ContinueWith(r => results.Enqueue(r.Result));
t.Start();
});
var allTasksCompleted = new AutoResetEvent(false);
new Timer(state =>
{
var timer = (Timer) state;
string item;
if (!results.TryDequeue(out item))
return;
importedData.Add(item);
Debug.WriteLine("Imported " + item);
if (importedData.Count == tasks.Count)
{
timer.Dispose();
Debug.WriteLine("Completed.");
allTasksCompleted.Set();
}
}).Change(1000, 100);
allTasksCompleted.WaitOne();
I have two operations - long running OperationA and much quicker OperationB. I was running them in parallel using TAP and returning results as they both finish :
var taskA = Task.Factory.StartNew(()=>OperationA());
var taskB = Task.Factory.StartNew(()=>OperationB());
var tasks = new Task[] { taskA, taskB };
Task.WaitAll(tasks);
// processing taskA.Result, taskB.Result
No magic here. Now what I want to do is repeat OperationB when it's finished indefinitely in case OperationA is still running. So whole procedure finish point will occur when OperationA is finished and last pass of OperationB is finished. I'm looking for some sort of effective pattern for doing that will not involve polling for OperationA's Status in while loop if that's possible. Looking toward improving WaitAllOneByOne pattern proposed in this Pluralsight course or something similar.
Try this
// Get cancellation support.
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
// Start off A and set continuation to cancel B when finished.
bool taskAFinished = false;
var taskA = Task.Factory.StartNew(() => OperationA());
Task contA = taskA.ContinueWith(ant => source.Cancel());
// Set off B and in the method perform your loop. Cancellation with be thrown when
// A has completed.
var taskB = Task.Factory.StartNew(() => OperationB(token), token);
Task contB = taskB.ContinueWith(ant =>
{
switch (task.Status)
{
// Handle any exceptions to prevent UnobservedTaskException.
case TaskStatus.RanToCompletion:
// Do stuff.
break;
case TaskStatus.Canceled:
// You know TaskA is finished.
break;
case TaskStatus.Faulted:
// Something bad.
break;
}
});
then in the OperationB method you can perform your loop and include a cancellation upon TaskA's compleation...
private void OperationB(CancellationToken token)
{
foreach (var v in object)
{
...
token.ThrowIfCancellationRequested(); // This must be handeled. AggregateException.
}
}
Note, instead of complicating with a cancellation, you can just set a bool from with in the continuation of TaskA and check for this in TaskB' loop - this will avoid any faffing about with cancellations.
I hope this helps
Took your approach as basis and adapted a bit :
var source = new CancellationTokenSource();
var token = source.Token;
var taskA = Task.Factory.StartNew(
() => OperationA()
);
var taskAFinished = taskA.ContinueWith(antecedent =>
{
source.Cancel();
return antecedent.Result;
});
var taskB = Task.Factory.StartNew(
() => OperationB(token), token
);
var taskBFinished = taskB.ContinueWith(antecedent =>
{
switch (antecedent.Status)
{
case TaskStatus.RanToCompletion:
case TaskStatus.Canceled:
try
{
return ant.Result;
}
catch (AggregateException ae)
{
// Operation was canceled before start if OperationA is short
return null;
}
case TaskStatus.Faulted:
return null;
}
return null;
});
Made two continuations that returns results for respective operations so I could make wait for them both to be finished (tried to do that only with second one and it didn't work).
var tasks = new Task[] { taskAFinished, taskBFinished };
Task.WaitAll(tasks);
First one is just passing antecedent's task Result further, second takes aggregate results available at this point in OperationB (both RanToCompletion and Canceled statuses are considered correct end of process). OperationB now looks like this :
public static List<Result> OperationB(CancellationToken token)
{
var resultsList = new List<Result>();
while (true)
{
foreach (var op in operations)
{
resultsList.Add(op.GetResult();
}
if (token.IsCancellationRequested)
{
return resultsList;
}
}
}
Changes logic a bit - all loops inside OperationB now are considered as single task, but this is easier than keep them atomic and write some sort of coordination primitive that will gather results from each run. In case I don't really care which loop produced which results this seems to be a decent solution. May improve to more flexible implementation later if needed (what I was actually looking for is to chain multiple operations recursively - OperationB itself may have smaller repeating OperationC's inside with same behavior, OperationC - multiple OperationD's that are running when C is active etc.).
edit
Added exception handling in taskBFinished for case when OperationA is quick and cancellation is issued before OperationB is even started.