This is the code that I wrote to better understand asynchronous methods. I knew that an asynchronous method is not the same as multithreading, but it does not seem so in this particular scenario:
class Program
{
static void Main(string[] args)
{
Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo("en-US");
//the line above just makes sure that the console output uses . to represent doubles instead of ,
ExecuteAsync();
Console.ReadLine();
}
private static async Task ParallelAsyncMethod() //this is the method where async parallel execution is taking place
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Run(() => DownloadWebsite()));
}
var strings = await Task.WhenAll(tasks);
foreach (var str in strings)
{
Console.WriteLine(str);
}
}
private static string DownloadWebsite() //Imitating a website download
{
Thread.Sleep(1500); //making the thread sleep for 1500 miliseconds before returning
return "Download finished";
}
private static async void ExecuteAsync()
{
var watch = Stopwatch.StartNew();
await ParallelAsyncMethod();
watch.Stop();
Console.WriteLine($"It took the machine {watch.ElapsedMilliseconds} milliseconds" +
$" or {Convert.ToDouble(watch.ElapsedMilliseconds) / 1000} seconds to complete this task");
Console.ReadLine();
}
}
//OUTPUT:
/*
Download finished
Download finished
Download finished
Download finished
Download finished
It took the machine 1537 milliseconds or 1.537 seconds to complete this task
*/
As you can see, the DownloadWebsite method waits for 1.5 seconds and then returns "a". The method called ParallelAsyncMethod adds five of these methods into the "tasks" list and then starts the parallel asynchronous execution. As you can see, I also tracked the amount of time that it takes for the ExecuteAsync method to be executed. The result is always somewhere around 1540 milliseconds. Here is my question: if the DownloadWebsite method required a thread to sleep 5 times for 1500 milliseconds, does it mean that the parallel execution of these methods required 5 different threads? If not, then how come it only took the program 1540 milliseconds to be executed and not ~7500 ms?
I knew that an asynchronous method is not the same as multi-threading
That is correct, an asynchronous method releases the current thread whilst I/O occurs, and schedules a continuation after it's completion.
Async and threads are completely unrelated concepts.
but it does not seem so in this particular scenario
That is because you explicitly run DownloadWebsite on the ThreadPool using Task.Run, which imitates asynchronous code by returning a Task after instructing the provided delegate to run.
Because you are not waiting for each Task to complete before starting the next, multiple threads can be used simultaneously.
Currently each thread is being blocked, as you have used Thread.Sleep in the implementation of DownloadWebsite, meaning you are actually running 5 synchronous methods on the ThreadPool.
In production code your DownloadWebsite method should be written asynchronously, maybe using HttpClient.GetAsync:
private static async Task<string> DownloadWebsiteAsync()
{
//...
await httpClinet.GetAsync(//...
//...
}
In that case, GetAsync returns a Task, and releases the current thread whilst waiting for the HTTP response.
You can still run multiple async methods concurrently, but as the thread is released each time, this may well use less than 5 separate threads and may even use a single thread.
Ensure that you dont use Task.Run with an asynchronous method; this simply adds unnecessary overhead:
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(DownloadWebsiteAsync()); // No need for Task.Run
}
var strings = await Task.WhenAll(tasks);
As an aside, if you want to imitate an async operation, use Task.Delay instead of Thread.Sleep as the former is non-blocking:
private static async Task<string> DownloadWebsite() //Imitating a website download
{
await Task.Delay(1500); // Release the thread for ~1500ms before continuing
return "Download finished";
}
I create a task with continuationWith(anotherTask) as below. I want to find out the time taken for completing its work by the first task. I share the variable "task1StartedDateTime" between task1 and child task. Will this work without any issues?
public static void MyMethod()
{
var task1StartedDateTime = DateTime.Now;
var task1 = doWorkAsync();
task1.ContinueWith(t1 => {
var task1TookTime = DateTime.Now - task1StartedDateTime;
Console.WriteLine($"Task 1 took {task1TookTime}");
//Some other work of this child task
});
}
Yes it will work. However it should be better to make use of the StopWatch class, since this is a more accurate and efficient way of calculating elapsed time of a method, process whatever running on a machine. For more info related to the latter argument, please have a look here:
var stopwatch = StopWatch.StartNew();
var task1 = doWorkAsync();
task1.ContinueWith(t1 => {
stopwatch.Stop();
Console.WriteLine($"Task 1 took {stopwatch.EllapsedMilliseconds} ms.");
//Some other work of this child task
}
Yes, you can use captured variables in a lambda - captured variables closed over in this way will be promoted to an anonymous class instance, to ensure they can outlive the method they are declared in, and to allow sharing between the outer method and the continuation.
However, you should use aStopwatch for measuring time - it is more accurate.
In .Net 4.5 and later, you also have the option to replace the continuation in .ContinueWith to an awaited continuation - this has additional guarantees, and is easier to read:
public static async Task MyMethod()
{
var sw = new Stopwatch();
sw.Start();
await doWorkAsync();
var task1TookTime = sw.Elapsed;
Console.WriteLine($"Task 1 took {task1TookTime}");
//Some other work of this child task
}
(Although note that if MyMethod is awaited, that the Task will only complete once doWorkAsync and the timer logging is complete, which differs from your original implementation).
I have a slightly complex requirement of performing some tasks in parallel, and having to wait for some of them to finish before continuing. Now, I am encountering unexpected behavior, when I have a number of tasks, that I want executed in parallel, but inside a ContinueWith handler. I have whipped up a small sample to illustrate the problem:
var task1 = Task.Factory.StartNew(() =>
{
Console.WriteLine("11");
Thread.Sleep(1000);
Console.WriteLine("12");
}).ContinueWith(async t =>
{
Console.WriteLine("13");
var innerTasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
var j = i;
innerTasks.Add(Task.Factory.StartNew(() =>
{
Console.WriteLine("1_" + j + "_1");
Thread.Sleep(500);
Console.WriteLine("1_" + j + "_2");
}));
}
await Task.WhenAll(innerTasks.ToArray());
//Task.WaitAll(innerTasks.ToArray());
Thread.Sleep(1000);
Console.WriteLine("14");
});
var task2 = Task.Factory.StartNew(() =>
{
Console.WriteLine("21");
Thread.Sleep(1000);
Console.WriteLine("22");
}).ContinueWith(t =>
{
Console.WriteLine("23");
Thread.Sleep(1000);
Console.WriteLine("24");
});
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");
The basic pattern is:
- Task 1 should be executed in parallel with Task 2.
- Once the first part of part 1 is done, it should do some more things in parallel. I want to complete, once everything is done.
I expect the following result:
1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
14 <- The end of the task 1 continuation
2 <- The end
Instead, what happens, is that the await Task.WhenAll(innerTasks.ToArray()); does not "block" the continuation task from completing. So, the inner tasks execute after the outer await Task.WhenAll(task1, task2); has completed. The result is something like:
1 <- Start
11 / 21 <- The initial task start
12 / 22 <- The initial task end
13 / 23 <- The continuation task start
Some combinations of "1_[0..9]_[1..2]" and 24 <- the "inner" tasks of task 1 + the continuation of task 2 end
2 <- The end
Some more combinations of "1_[0..9]_[1..2]" <- the "inner" tasks of task 1
14 <- The end of the task 1 continuation
If, instead, I use Task.WaitAll(innerTasks.ToArray()), everything seems to work as expected. Of course, I would not want to use WaitAll, so I won't block any threads.
My questions are:
Why is this unexpected behavior occuring?
How can I remedy the situation without blocking any threads?
Thanks a lot in advance for any pointers!
You're using the wrong tools. Instead of StartNew, use Task.Run. Instead of ContinueWith, use await:
var task1 = Task1();
var task2 = Task2();
Console.WriteLine("1");
await Task.WhenAll(task1, task2);
Console.WriteLine("2");
private async Task Task1()
{
await Task.Run(() =>
{
Console.WriteLine("11");
Thread.Sleep(1000);
Console.WriteLine("12");
});
Console.WriteLine("13");
var innerTasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
innerTasks.Add(Task.Run(() =>
{
Console.WriteLine("1_" + i + "_1");
Thread.Sleep(500);
Console.WriteLine("1_" + i + "_2");
}));
await Task.WhenAll(innerTasks);
}
Thread.Sleep(1000);
Console.WriteLine("14");
}
private async Task Task2()
{
await Task.Run(() =>
{
Console.WriteLine("21");
Thread.Sleep(1000);
Console.WriteLine("22");
});
Console.WriteLine("23");
Thread.Sleep(1000);
Console.WriteLine("24");
}
Task.Run and await are superior here because they correct a lot of unexpected behavior in StartNew/ContinueWith. In particular, asynchronous delegates and (for Task.Run) always using the thread pool.
I have more detailed info on my blog regarding why you shouldn't use StartNew and why you shouldn't use ContinueWith.
As noted in the comments, what you're seeing is normal. The Task returned by ContinueWith() completes when the delegate passed to and invoked by ContinueWith() finishes executing. This happens the first time the anonymous method uses the await statement, and the delegate returns a Task object itself that represents the eventual completion of the entire anonymous method.
Since you are only waiting on the ContinueWith() task, and this task only represents the availability of the task that represents the anonymous method, not the completion of that task, your code doesn't wait.
From your example, it's not clear what the best fix is. But if you make this small change, it will do what you want:
await Task.WhenAll(await task1, task2);
I.e. in the WhenAll() call, don't wait on the ContinueWith() task itself, but rather on the task that task will eventually return. Use await here to avoid blocking the thread while you wait for that task to be available.
When using async methods/lambdas with StartNew, you either wait on the returned task and the contained task:
var task = Task.Factory.StartNew(async () => { /* ... */ });
task.Wait();
task.Result.Wait();
// consume task.Result.Result
Or you use the extension method Unwrap on the result of StartNew and wait on the task it returns.
var task = Task.Factory.StartNew(async () => { /* ... */ })
.Unwrap();
task.Wait();
// consume task.Result
The following discussion goes along the line that Task.Factory.StartNew and ContinueWith should be avoided in specific cases, such as when you don't provide creation or continuation options or when you don't provide a task scheduler.
I don't agree that Task.Factory.StartNew shouldn't be used, I agree that you should use (or consider using) Task.Run wherever you use a Task.Factory.StartNew method overload that doesn't take TaskCreationOptions or a TaskScheduler.
Note that this only applies to the default Task.Factory. I've used custom task factories where I chose to use the StartNew overloads without options and task scheduler, because I configured the factories specific defaults for my needs.
Likewise, I don't agree that ContinueWith shouldn't be used, I agree that you should use (or consider using) async/await wherever you use a ContinueWith method overload that doesn't take TaskContinuationOptions or a TaskScheduler.
For instance, up to C# 5, the most practical way to workaround the limitation of await not being supported in catch and finally blocks is to use ContinueWith.
C# 6:
try
{
return await something;
}
catch (SpecificException ex)
{
await somethingElse;
// throw;
}
finally
{
await cleanup;
}
Equivalent before C# 6:
return await something
.ContinueWith(async somethingTask =>
{
var ex = somethingTask.Exception.InnerException as SpecificException;
if (ex != null)
{
await somethingElse;
// await somethingTask;
}
},
CancellationToken.None,
TaskContinuationOptions.DenyChildAttach | TaskContinuationOptions.NotOnRanToCompletion,
TaskScheduler.Default)
.Unwrap()
.ContinueWith(async catchTask =>
{
await cleanup;
await catchTask;
},
CancellationToken.None,
TaskContinuationOptions.DenyChildAttach,
TaskScheduler.Default)
.Unwrap();
Since, as I told, in some cases I have a TaskFactory with specific defaults, I've defined a few extension methods that take a TaskFactory, reducing the error chance of not passing one of the arguments (I known I can always forget to pass the factory itself):
public static Task ContinueWhen(this TaskFactory taskFactory, Task task, Action<Task> continuationAction)
{
return task.ContinueWith(continuationAction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}
public static Task<TResult> ContinueWhen<TResult>(this TaskFactory taskFactory, Task task, Func<Task, TResult> continuationFunction)
{
return task.ContinueWith(continuationFunction, taskFactory.CancellationToken, taskFactory.ContinuationOptions, taskFactory.Scheduler);
}
// Repeat with argument combinations:
// - Task<TResult> task (instead of non-generic Task task)
// - object state
// - bool notOnRanToCompletion (useful in C# before 6)
Usage:
// using namespace that contains static task extensions class
var task = taskFactory.ContinueWhen(existsingTask, t => Continue(a, b, c));
var asyncTask = taskFactory.ContinueWhen(existingTask, async t => await ContinueAsync(a, b, c))
.Unwrap();
I decided not to mimic Task.Run by not overloading the same method name to unwrapping task-returning delegates, it's really not always what you want. Actually, I didn't even implement ContinueWhenAsync extension methods so you need to use Unwrap or two awaits.
Often, these continuations are I/O asynchronous operations, and the pre- and post-processing overhead should be so small that you shouldn't care if it starts running synchronously up to the first yielding point, or even if it completes synchronously (e.g. using an underlying MemoryStream or a mocked DB access). Also, most of them don't depend on a synchronization context.
Whenever you apply the Unwrap extension method or two awaits, you should check if the task falls in this category. If so, async/await is most probably a better choice than starting a task.
For asynchronous operations with a non-negligible synchronous overhead, starting a new task may be preferable. Even so, a notable exception where async/await is still a better choice is if your code is async from the start, such as an async method invoked by a framework or host (ASP.NET, WCF, NServiceBus 6+, etc.), as the overhead is your actual business. For long processing, you may consider using Task.Yield with care. One of the tenets of asynchronous code is to not be too fine grained, however, too coarse grained is just as bad: a set of heavy-duty tasks may prevent the processing of queued lightweight tasks.
If the asynchronous operation depends on a synchronization context, you can still use async/await if you're within that context (in this case, think twice or more before using .ConfigureAwait(false)), otherwise, start a new task using a task scheduler from the respective synchronization context.
TL;DR:
Is it possible to kick off a series of IO bound tasks using only one thread using async await?
In less brief:
Trying to learn async await. In this video ("Async Best Practices for C# and Visual Basic"), the speaker gives an example of using async await to kick off some IO bound work. He explicitly says (at 21m 40s) whilst explaining why parallel for loops are not optimal as they use up loads of threads:
We don't need more threads for this. We don't need two threads...
Can we really kick off multiple requests asyncronously without using more than one thread? How? Unfortunately, the speaker didn't provide all the code so here's my stab at it:
// Pretty much exactly the same as video
private async Task<List<string>> LoadHousesAsync()
{
// Running on the UI thread
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
Task<string> t = LoadHouseAsync(i);
tasks.Add(t);
}
string[] loadedHouses = await Task.WhenAll(tasks);
return loadedHouses.ToList();
}
// My guess of the LoadHouseAsync method
private Task<string> LoadHouseAsync(int i)
{
// Running on the UI thread
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
return Task.Run(() => LoadHouse(i));
}
// My guess of the LoadHouse method
private string LoadHouse(int i)
{
// **** This is on a different thread :( ****
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
Thread.Sleep(5000); // simulate I/O bound work
return "House" + i;
}
Here's the output.
Thread: 10
Thread: 10
Thread: 3
Thread: 10
Thread: 10
Thread: 11
Thread: 10
Thread: 12
Thread: 10
Thread: 13
Thread: 14
You can do it with async I/O. What you made is a very nice example of doing it wrong (unfortunately, it's also quite common).
Task.Run runs a method on a thread pool thread, Thread.Sleep blocks the thread. So your example simulates doing synchronous (blocking) I/O on multiple threads.
To correctly perform async I/O, you need to use async methods all the way down. Never use Task.Run for I/O. You can simulate an asynchronous I/O method using Task.Delay:
private async Task<string> LoadHouseAsync(int i)
{
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
await Task.Delay(5000); // simulate async I/O bound work
return "House" + i;
}
Can we really kick off multiple requests asyncronously without using more than one thread?
Yes, here is a real world example (pseudo asp.net-mvc and entity framework, where EF will make IO calls to Sql Server for example).
public async ActionResult()
{
var model = new Company();
using (var db1 = new DbContext)
using (var db2 = new DbContext)
{
var task1 = db1.Employees.ToListAsync();
var task2 = db1.Managers.ToListAsync();
await Task.WhenAll(task1, task2);
model.employees = task1.Result;
model.managers = task2.Result;
}
return View(model);
}
I learned to use Task easily than async/await. Now, I'm trying to use Task to learn async/await.
static void Main(string[] args)
{
Console.ReadKey(true);
//Magic1();
Magic2();
Console.WriteLine("{0}", DateTime.Now.ToString());
Console.ReadKey(true);
}
static async void Magic1()
{
var taskA = GetDataAsync();
var taskB = GetDataAsync();
var taskC = GetDataAsync();
Console.WriteLine("a: " + await taskA);
Console.WriteLine("b: " + await taskB);
Console.WriteLine("c: " + await taskC);
}
static Task Magic2()
{
return Task.Run(() =>
{
var taskA = GetDataAsync();
var taskB = GetDataAsync();
var taskC = GetDataAsync();
Task.WaitAll(new Task[] { taskA, taskB, taskC });
Console.WriteLine("a: " + taskA.Result);
Console.WriteLine("b: " + taskB.Result);
Console.WriteLine("c: " + taskC.Result);
});
}
static Task<string> GetDataAsync()
{
return Task.Run(() =>
{
var startTime = DateTime.Now;
for (var i = 0; i < 1000000000; i++)
{
}
var endTime = DateTime.Now;
return startTime.ToString() + " to " + endTime.ToString() + " is " + (endTime - startTime).ToString();
});
}
I created two methods that appears to do the same thing, my questions are:
1) is Magic1 and Magic2 the same under the hood?
2) if they are not the same, can I convert Magic1 to a method that does the same thing without using async and await keywords?
Now, I'm trying to use my knowledge in Task to learn async/await.
I really recommend you not do this. Although task-based parallelism (.NET 4.0) and the task-based asynchronous pattern (async/await) use the same type (Task), they are completely different. They solve different problems and have a different way of working.
Instead, I suggest you pretend that you know nothing about the Task type and start with my async intro. At the end of my async intro are several followup resources, including the MSDN docs which are quite good for this feature.
If you're really familiar with continuations, you can (mostly) think of await as meaning "rewrite the rest of this method as a continuation and schedule it using the current SynchronizationContext or TaskScheduler". But even that is only an approximation; there are plenty of edge cases. It is not at all the same as doing a Task.Wait[All] within a Task.Run.
Differences between the two methods:
Where their code runs. Magic2 will run on the threadpool instead of in the caller's synchronization context, because you (unnecessarily) used Task.Run. If you were writing UI code, this could cause Magic2 to fail.
Blocking. Magic1 blocks the caller until it is done. This is generally a mistake when writing async code. Magic2 won't block the caller, even if you remove the Task.Run.
Incremental work. If taskB takes significantly longer to complete, that won't delay Magic2 from printing the result of taskA. Magic1 waits for all three tasks to finish, so it won't print taskA until after taskB finished.