I have the following synchronous code:
foreach ( var step in result ) {
step.Run();
}
I tried to convert it to tasks but I failed to do so. I tried to convert it using Task.WhenAll like this (and I did append async to the method signature):
var tasks = new List<Task>();
foreach ( var step in result ) {
tasks.Add( new Task( () => step.Run() ) );
}
await Task.WhenAll( tasks );
This returns immediately and doesn't execute the Run() method. Then I tried to convert it to the following code:
var tasks = new List<Task>();
foreach ( var step in result ) {
tasks.Add( new Task( () => step.Run() ) );
}
var task = Task.WhenAll( tasks );
task.Wait();
This blocks forever. However, when I create a within the loop it works:
foreach ( var step in result ) {
var t = Task.Run( () => step.Run() );
t.Wait();
}
If I use instead await Task.Run( () => step.Run() ); it awaits only the first one and resumes the main thread.
The run method looks like this:
public async void Run() {
var result = Work();
if ( null != result && result.Count > 0 ) {
var tasks = new List<Task>();
foreach ( var step in result ) {
await Task.Run( () => step.Run() );
}
}
}
All steps implement a Work() method (which is abstract in a base class). My first step looks like this:
class NoWorkStep : WorkerStep {
protected override IList<WorkerStep> Work() {
Console.WriteLine( "HERE" );
List<WorkerStep> newList = new List<WorkerStep>();
for ( int i = 0; i < 10; i++ ) {
newList.Add( new NoWorkStep2() );
}
return newList;
}
}
And my second step looks like this:
class NoWorkStep2 : WorkerStep {
protected override IList<WorkerStep> Work() {
Console.WriteLine( "HERE-2" );
return new List<WorkerStep>();
}
}
I simple create an instance of NoWorkStep and call instance.Run().
Where do I have a problem with executing the steps with Task.WhenAll?
Edit: Calling code after I changed the Run method to async Task RunAsync:
private static async void doIt() {
var step = new NoWorkStep();
await step.RunAsync();
}
Lets map out the problems with your code:
new Task(() => step.Run())
This returns a cold Task, meaning the Task isn't actually started. In order for it to start you would need to call:
new Task(() => step.Run()).Start)
But, you shouldn't use new Task anyway, you should use Task.Run.
If I use instead await Task.Run( () => step.Run() ); it awaits only
the first one and resumes the main thread.
That is because Run is async void which cannot be awaited. async void is ment to be used only in top level event handlers, where this clearly isn't the case here.
If you want to await on until all the tasks are completed, you can do that following:
public async Task RunAsync()
{
var result = Work();
var stepTasks = result.Select(step => Task.Run(() => step.Run()));
await Task.WhenAll(steps);
}
This will guarantee all tasks have completed execution once RunAsync finishes.
You don't seem to be starting the tasks.
Try:
var tasks = new List<Task>();
foreach (var step in result)
{
var t = new Task(() => step.Run());
t.Start();
tasks.Add(t);
}
Task.WhenAll(tasks);
You can use Parallel.ForEach.
Parallel.ForEach(result, step => step.Run());
This way you don't even muck around with the lower level parts of the Parallel Framework.
Related
I have an async method in a loop
public async System.Threading.Tasks.Task WatermarXabeAsyncBase()
{
var result = await conversion.Start();
}
...
foreach (var x in y)
{
WatermarXabeAsyncBase()
}
I want to call another method ONLY when the await function is over.
What should I do and how?
Save returned Task and then you can check if it is complete or not by awaiting it.
To wait inside the loop
foreach (var x in y)
{
var task = WatermarXabeAsyncBase();
// .. do something else
await task;
// next lines of code will be executed only after task is complete
// do something after method is complete
}
To wait for all tasks to complete
var tasks = new List<Task>();
foreach (var x in y)
{
var task = WatermarXabeAsyncBase();
tasks.Add(task);
}
await Task.WhenAll(tasks);
// Do something after all methods are complete.
I have a C# console app. In this app, I have a method that I will call DoWorkAsync. For the context of this question, this method looks like this:
private async Task<string> DoWorkAsync()
{
System.Threading.Thread.Sleep(5000);
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
var length = random.Next(10, 101);
await Task.CompletedTask;
return new string(Enumerable.Repeat(chars, length)
.Select(s => s[random.Next(s.Length)]).ToArray());
}
I call DoWorkAsync from another method that determines a) how many times this will get ran and b) if each call will be ran in parallel or sequentially. That method looks like this:
private async Task<Task<string>[]> DoWork(int iterations, bool runInParallel)
{
var tasks = new List<Task<string>>();
for (var i=0; i<iterations; i++)
{
if (runInParallel)
{
var task = Task.Run(() => DoWorkAsync());
tasks.Add(task);
}
else
{
await DoWorkAsync();
}
}
return tasks.ToArray();
}
After all of the tasks are completed, I want to display the results. To do this, I have code that looks like this:
var random = new Random();
var tasks = await DoWork(random.Next(10, 101);
Task.WaitAll(tasks);
foreach (var task in tasks)
{
Console.WriteLine(task.Result);
}
This code works as expected if the code runs in parallel (i.e. runInParallel is true). However, when runInParallel is false (i.e. I want to run the Tasks sequentially) the Task array doesn't get populated. So, the caller doesn't have any results to work with. I don't know how to fix it though. I'm not sure how to add the method call as a Task that will run sequentially. I understand that the idea behind Tasks is to run in parallel. However, I have this need to toggle between parallel and sequential.
Thank you!
the Task array doesn't get populated.
So populate it:
else
{
var task = DoWorkAsync();
tasks.Add(task);
await task;
}
P.S.
Also your DoWorkAsync looks kinda wrong to me, why Thread.Sleep and not await Task.Delay (it is more correct way to simulate asynchronous execution, also you won't need await Task.CompletedTask this way). And if you expect DoWorkAsync to be CPU bound just make it like:
private Task<string> DoWorkAsync()
{
return Task.Run(() =>
{
// your cpu bound work
return "string";
});
}
After that you can do something like this (for both async/cpu bound work):
private async Task<string[]> DoWork(int iterations, bool runInParallel)
{
if(runInParallel)
{
var tasks = Enumerable.Range(0, iterations)
.Select(i => DoWorkAsync());
return await Task.WhenAll(tasks);
}
else
{
var result = new string[iterations];
for (var i = 0; i < iterations; i++)
{
result[i] = await DoWorkAsync();
}
return result;
}
}
Why is DoWorkAsync an async method?
It isn't currently doing anything asynchronous.
It seems that you are trying to utilise multiple threads to improve the performance of expensive CPU-bound work, so you would be better to make use of Parallel.For, which is designed for this purpose:
private string DoWork()
{
System.Threading.Thread.Sleep(5000);
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
var length = random.Next(10, 101);
return new string(Enumerable.Repeat(chars, length)
.Select(s => s[random.Next(s.Length)]).ToArray());
}
private string[] DoWork(int iterations, bool runInParallel)
{
var results = new string[iterations];
if (runInParallel)
{
Parallel.For(0, iterations - 1, i => results[i] = DoWork());
}
else
{
for (int i = 0; i < iterations; i++) results[i] = DoWork();
}
return results;
}
Then:
var random = new Random();
var serial = DoWork(random.Next(10, 101));
var parallel = DoWork(random.Next(10, 101), true);
I think you'd be better off doing the following:
Create a function that creates a (cold) list of tasks (or an array Task<string>[] for instance). No need to run them. Let's call this GetTasks()
var jobs = GetTasks();
Then, if you want to run them "sequentially", just do
var results = new List<string>();
foreach (var job in jobs)
{
var result = await job;
results.Add(result);
}
return results;
If you want to run them in parallel :
foreach (var job in jobs)
{
job.Start();
}
await results = Task.WhenAll(jobs);
Another note,
All this in itself should be a Task<string[]>, the Task<Task<... smells like a problem.
I have the following code.
var tasks = new[]
{
Task.Factory.StartNew(() => GetSomething1()),
Task.Factory.StartNew(() => GetSomething2()),
Task.Factory.StartNew(() => GetSomething3())
};
var things = Task.WhenAll(tasks);
How can I get the results from all three tasks separately and print them?
In order to get the results seperately you have multiple ways I would do something like this:
var task1 = GetSomething1();
var task2 = GetSomething2();
var task3 = GetSomething3();
// your method will continue when everything's completed, but you won't tie up a thread to just hang around until that time.
await Task.WhenAll(task1, task2, task3);
var result1 = await task1;
var result2 = await task2;
var result3 = await task3;
One way is to make each Task responsible for storing its own result, and then all you have to do is await the collection of Tasks. Note that you'll have to use await to make the WhenAll() execute the tasks that you pass into it.
var results = new int[3];
var tasks = new[] {
Task.Factory.StartNew(() => results[0] = GetSomething1()),
Task.Factory.StartNew(() => results[1] = GetSomething2()),
Task.Factory.StartNew(() => results[2] = GetSomething3())
};
await Task.WhenAll(tasks);
Console.WriteLine(results[0]);
Console.WriteLine(results[1]);
Console.WriteLine(results[2]);
Working demo: https://dotnetfiddle.net/HS32QA
Note that you may need to be careful with using e.g. a List<T> instead of an array, and then calling list.Add(result), as there is no guarantee of the order in which the tasks are executed or when they will be finished.
You should use async..await pattern with When, e.g.
private async Task MyExecution() {
var tasks = new[] {
//TODO: Task.Run is a better option than Task.Factory.StartNew
Task.Factory.StartNew(() => GetSomething1()),
Task.Factory.StartNew(() => GetSomething2()),
Task.Factory.StartNew(() => GetSomething3())
};
// Await until all tasks are completed
await Task.WhenAll(tasks);
// Since all tasks are completed, we can (safely) query for their `Result`s:
var things = tasks
.Select(task => task.Result) // or task => await task
.ToArray();
// Let's print the things
for (int i = 0; i < things.Length; ++i)
Console.WriteLine($"Task #{i + 1} returned {things[i]}");
...
}
Have you considered actually using async functions? Then you have an array of tasks with the results, and avoid the highly unpredictable behaviours of Task.Factory.StartNew.
private async Task MyExecution()
{
var tasks = new[] {
GetSomething1(),
GetSomething2(),
GetSomething3()
};
// Await until all tasks are completed
await Task.WhenAll(tasks);
foreach(var t in tasks){
//t.Result is available
}
}
public static async Task<int> GetSomething1() { return 1; }
public static async Task<int> GetSomething2() { return 2; }
public static async Task<int> GetSomething3() { return 3; }
Fiddle here: https://dotnetfiddle.net/3ffs9L
Lets say that i have a couple of tasks:
void Sample(IEnumerable<int> someInts)
{
var taskList = someInts.Select(x => DownloadSomeString(x));
}
async Task<string> DownloadSomeString(int x) {...}
I want to to get the result of first successful task. So, the basic solution is to write something like:
var taskList = someInts.Select(x => DownloadSomeString(x));
string content = string.Empty;
Task<string> firstOne = null;
while (string.IsNullOrWhiteSpace(content)){
try
{
firstOne = await Task.WhenAny(taskList);
if (firstOne.Status != TaskStatus.RanToCompletion)
{
taskList = taskList.Where(x => x != firstOne);
continue;
}
content = await firstOne;
}
catch(...){taskList = taskList.Where(x => x != firstOne);}
}
But this solution seems to run N+(N-1)+..+K tasks. Where N is someInts.Count and K is position of first successful task in tasks, so as it's rerunning all task except one that is captured by WhenAny.
So, is there any way to get first task that finished successfully with running maximum of N tasks? (if successful task will be the last one)
All you need to do is create a TaskCompletionSource, add a continuation to each of your tasks, and set it when the first one finished successfully:
public static Task<T> FirstSuccessfulTask<T>(IEnumerable<Task<T>> tasks)
{
var taskList = tasks.ToList();
var tcs = new TaskCompletionSource<T>();
int remainingTasks = taskList.Count;
foreach (var task in taskList)
{
task.ContinueWith(t =>
{
if (task.Status == TaskStatus.RanToCompletion)
tcs.TrySetResult(t.Result);
else
if (Interlocked.Decrement(ref remainingTasks) == 0)
tcs.SetException(new AggregateException(tasks.SelectMany(t1 => t1.Exception.InnerExceptions)));
});
}
return tcs.Task;
}
And a version for tasks without a result:
public static Task FirstSuccessfulTask(IEnumerable<Task> tasks)
{
var taskList = tasks.ToList();
var tcs = new TaskCompletionSource<bool>();
int remainingTasks = taskList.Count;
foreach (var task in taskList)
{
task.ContinueWith(t =>
{
if (task.Status == TaskStatus.RanToCompletion)
tcs.TrySetResult(true);
else
if (Interlocked.Decrement(ref remainingTasks) == 0)
tcs.SetException(new AggregateException(
tasks.SelectMany(t1 => t1.Exception.InnerExceptions)));
});
}
return tcs.Task;
}
The problem with "the first successful task" is what to do if all tasks fail? It's a really bad idea to have a task that never completes.
I assume you'd want to propagate the last task's exception if they all fail. With that in mind, I would say something like this would be appropriate:
async Task<Task<T>> FirstSuccessfulTask(IEnumerable<Task<T>> tasks)
{
Task<T>[] ordered = tasks.OrderByCompletion();
for (int i = 0; i != ordered.Length; ++i)
{
var task = ordered[i];
try
{
await task.ConfigureAwait(false);
return task;
}
catch
{
if (i == ordered.Length - 1)
return task;
continue;
}
}
return null; // Never reached
}
This solution builds on the OrderByCompletion extension method that is part of my AsyncEx library; alternative implementations also exist by Jon Skeet and Stephen Toub.
As a straight forward solution is to wait for any task, check if it is in RanToCompletion state and if not, wait again for any task except the already finished one.
async Task<TResult> WaitForFirstCompleted<TResult>( IEnumerable<Task<TResult>> tasks )
{
var taskList = new List<Task<TResult>>( tasks );
while ( taskList.Count > 0 )
{
Task<TResult> firstCompleted = await Task.WhenAny( taskList ).ConfigureAwait(false);
if ( firstCompleted.Status == TaskStatus.RanToCompletion )
{
return firstCompleted.Result;
}
taskList.Remove( firstCompleted );
}
throw new InvalidOperationException( "No task completed successful" );
}
I have found many methods of using the TaskFactory but I could not find anything about starting more tasks and watching when one ends and starting another one.
I always want to have 10 tasks working.
I want something like this
int nTotalTasks=10;
int nCurrentTask=0;
Task<bool>[] tasks=new Task<bool>[nThreadsNum];
for (int i=0; i<1000; i++)
{
string param1="test";
string param2="test";
if (nCurrentTask<10) // if there are less than 10 tasks then start another one
tasks[nCurrentThread++] = Task.Factory.StartNew<bool>(() =>
{
MyClass cls = new MyClass();
bool bRet = cls.Method1(param1, param2, i); // takes up to 2 minutes to finish
return bRet;
});
// How can I stop the for loop until a new task is finished and start a new one?
}
Check out the Task.WaitAny method:
Waits for any of the provided Task objects to complete execution.
Example from the documentation:
var t1 = Task.Factory.StartNew(() => DoOperation1());
var t2 = Task.Factory.StartNew(() => DoOperation2());
Task.WaitAny(t1, t2)
I would use a combination of Microsoft's Reactive Framework (NuGet "Rx-Main") and TPL for this. It becomes very simple.
Here's the code:
int nTotalTasks=10;
string param1="test";
string param2="test";
IDisposable subscription =
Observable
.Range(0, 1000)
.Select(i => Observable.FromAsync(() => Task.Factory.StartNew<bool>(() =>
{
MyClass cls = new MyClass();
bool bRet = cls.Method1(param1, param2, i); // takes up to 2 minutes to finish
return bRet;
})))
.Merge(nTotalTasks)
.ToArray()
.Subscribe((bool[] results) =>
{
/* Do something with the results. */
});
The key part here is the .Merge(nTotalTasks) which limits the number of concurrent tasks.
If you need to stop the processing part way thru just call subscription.Dispose() and everything gets cleaned up for you.
If you want to process each result as they are produced you can change the code from the .Merge(...) like this:
.Merge(nTotalTasks)
.Subscribe((bool result) =>
{
/* Do something with each result. */
});
This should be all you need, not complete, but all you need to do is wait on the first to complete and then run the second.
Task.WaitAny(task to wait on);
Task.Factory.StartNew()
Have you seen the BlockingCollection class? It allows you to have multiple threads running in parallel and you can wait from results from one task to execute another. See more information here.
The answer depends on whether the tasks to be scheduled are CPU or I/O bound.
For CPU-intensive work I would use Parallel.For() API setting the number of thread/tasks through MaxDegreeOfParallelism property of ParallelOptions
For I/O bound work the number of concurrently executing tasks can be significantly larger than the number of available CPUs, so the strategy is to rely on async methods as much as possible, which reduces the total number of threads waiting for completion.
How can I stop the for loop until a new task is finished and start a
new one?
The loop can be throttled by using await:
static void Main(string[] args)
{
var task = DoWorkAsync();
task.Wait();
// handle results
// task.Result;
Console.WriteLine("Done.");
}
async static Task<bool> DoWorkAsync()
{
const int NUMBER_OF_SLOTS = 10;
string param1="test";
string param2="test";
var results = new bool[NUMBER_OF_SLOTS];
AsyncWorkScheduler ws = new AsyncWorkScheduler(NUMBER_OF_SLOTS);
for (int i = 0; i < 1000; ++i)
{
await ws.ScheduleAsync((slotNumber) => DoWorkAsync(i, slotNumber, param1, param2, results));
}
ws.Complete();
await ws.Completion;
}
async static Task DoWorkAsync(int index, int slotNumber, string param1, string param2, bool[] results)
{
results[slotNumber] = results[slotNumber} && await Task.Factory.StartNew<bool>(() =>
{
MyClass cls = new MyClass();
bool bRet = cls.Method1(param1, param2, i); // takes up to 2 minutes to finish
return bRet;
}));
}
A helper class AsyncWorkScheduler uses TPL.DataFlow components as well as Task.WhenAll():
class AsyncWorkScheduler
{
public AsyncWorkScheduler(int numberOfSlots)
{
m_slots = new Task[numberOfSlots];
m_availableSlots = new BufferBlock<int>();
m_errors = new List<Exception>();
m_tcs = new TaskCompletionSource<bool>();
m_completionPending = 0;
// Initial state: all slots are available
for(int i = 0; i < m_slots.Length; ++i)
{
m_slots[i] = Task.FromResult(false);
m_availableSlots.Post(i);
}
}
public async Task ScheduleAsync(Func<int, Task> action)
{
if (Volatile.Read(ref m_completionPending) != 0)
{
throw new InvalidOperationException("Unable to schedule new items.");
}
// Acquire a slot
int slotNumber = await m_availableSlots.ReceiveAsync().ConfigureAwait(false);
// Schedule a new task for a given slot
var task = action(slotNumber);
// Store a continuation on the task to handle completion events
m_slots[slotNumber] = task.ContinueWith(t => HandleCompletedTask(t, slotNumber), TaskContinuationOptions.ExecuteSynchronously);
}
public async void Complete()
{
if (Interlocked.CompareExchange(ref m_completionPending, 1, 0) != 0)
{
return;
}
// Signal the queue's completion
m_availableSlots.Complete();
await Task.WhenAll(m_slots).ConfigureAwait(false);
// Set completion
if (m_errors.Count != 0)
{
m_tcs.TrySetException(m_errors);
}
else
{
m_tcs.TrySetResult(true);
}
}
public Task Completion
{
get
{
return m_tcs.Task;
}
}
void SetFailed(Exception error)
{
lock(m_errors)
{
m_errors.Add(error);
}
}
void HandleCompletedTask(Task task, int slotNumber)
{
if (task.IsFaulted || task.IsCanceled)
{
SetFailed(task.Exception);
return;
}
if (Volatile.Read(ref m_completionPending) == 1)
{
return;
}
// Release a slot
m_availableSlots.Post(slotNumber);
}
int m_completionPending;
List<Exception> m_errors;
BufferBlock<int> m_availableSlots;
TaskCompletionSource<bool> m_tcs;
Task[] m_slots;
}