We Keep Coding

Using IProgress when reporting progress for async await code vs progress bar control

private static async Task FuncAsync(DataTable dt, DataRow dr)
{
try
{
await Task.Delay(3000); //assume this is an async http post request that takes 3 seconds to respond
Thread.Sleep(1000) //assume this is some synchronous code that takes 2 second
}
catch (Exception e)
{
Thread.Sleep(1000); //assume this is synchronous code that takes 1 second
}
}
private async void Button1_Click(object sender, EventArgs e)
{
List<Task> lstTasks = new List<Task>();
DataTable dt = (DataTable)gridview1.DataSource;
foreach (DataRow dr in dt.Rows)
{
lstTasks.Add(FuncAsync(dr["colname"].ToString());
}
while (lstTasks.Any())
{
Task finishedTask = await Task.WhenAny(lstTasks);
lstTasks.Remove(finishedTask);
await finishedTask;
progressbar1.ReportProgress();
}
}
Assuming the datatable has got 10000 rows.
In the code, on button click, at the 1st iteration of the for loop, an async api request is made. While it takes 3 seconds, the control immediately goes to the caller. So the for loop can make the next iteration, and so on.
When the api response arrives, the code below the await runs as a callback. Thus blocking the UI thread and any incomplete for loop iterations will be delayed until the callback completes irrespective of whether I use await WhenAny or WhenAll.
All code runs on the UI thread due to the presence of synchronization context. I can do ConfigureAwait false on Task.Delay so the callbacks run on separate threads in order to unblock the ui thread.
Say 1000 iterations are made when the 1st await returns and when the 1st iterations await call back runs the following iterations will have completed completed awaits so their callbacks will run. Effectively callbacks will run one after the other if configure await is true. If false then they will run in parallel on separate threads.
So I think that the progress bar that I am updating in the while loop is incorrect because - by the time the code reaches the while block, most of the initial for loop iterations will have been already completed. I hope that I have understood correctly so far.
I have the following options to report progress from inside the task:
using IProgress (I think this is more suitable to report progress from another thread [for example when using Task.Run], or in usual async await if the configure await is false, resulting in code below the await to run in separate thread otherwise it will not show the progress bar moving as the ui thread will be blocked running the callbacks. In my current example code always runs on the same UI thread). So I was thinking the below point may be more appropriate solution.
making the Task non-static so that I can access the progress bar from within the Task and do porgressbar1.PerformStep().
Another thing I have noticed is that await WhenAll doesn't guarantee that IProgress is fully executed.

The IProgress<T> implementation offered natively by the .NET platform, the Progress<T> class, has the interesting characteristic of notifying the captured SynchronizationContext asynchronously, by invoking its Post method. This characteristic sometimes results to unexpected behavior. For example can you guess what effect has the code below to the Label1 control?
IProgress<string> progress = new Progress<string>(s => Label1.Text = s);
progress.Report("Hello");
Label1.Text = "World";
What text will be eventually written to the label, "Hello" or "World"? The correct answer is: "Hello". The delegate s => Label1.Text = s is invoked asynchronously, so it runs after the execution of the Label1.Text = "World" line, which is invoked synchronously.
Implementing a synchronous version of the Progress<T> class is quite trivial. All you have to do is copy-paste Microsoft's source code, rename the class from Progress<T> to SynchronousProgress<T>, and change the line m_synchronizationContext.Post(... to m_synchronizationContext.Send(.... This way every time you invoke the progress.Report method, the call will block until the invocation of the delegate on the UI thread is completed. The unfortunate implication of this is that if the UI thread is blocked for some reason, for example because you used the .Wait() or the .Result to wait synchronously for the task to complete, your application will deadlock.
The asynchronous nature of the Progress<T> class is rarely a problem in practice, but if you want to avoid thinking about it you can just manipulate the ProgressBar1 control directly. After all you are not writing a library, you are just writing code in the event handler of a button to make some HTTP requests. My suggestion is to forget about the .ConfigureAwait(false) hackery, and just let the main workflow of your asynchronous event handler to stay on the UI thread from start to end. If you have synchronous blocking code that needs to be offloaded to a ThreadPool thread, use the Task.Run method to offload it. To create your tasks, instead of manually adding tasks to a List<Task>, use the handly LINQ Select operator to project each DataRow to a Task. Also add a reference to the System.Data.DataSetExtensions assembly, so that the DataTable.AsEnumerable extension method becomes available. Finally add a throttler (a SemaphoreSlim), so that your application makes efficient use of the available network bandwidth, and it doesn't overburden the target machine:
private async void Button1_Click(object sender, EventArgs e)
{
Button1.Enabled = false;
const int maximumConcurrency = 10;
var throttler = new SemaphoreSlim(maximumConcurrency, maximumConcurrency);
DataTable dataTable = (DataTable)GridView1.DataSource;
ProgressBar1.Minimum = 0;
ProgressBar1.Maximum = dataTable.Rows.Count;
ProgressBar1.Step = 1;
ProgressBar1.Value = 0;
Task[] tasks = dataTable.AsEnumerable().Select(async row =>
{
await throttler.WaitAsync();
try
{
await Task.Delay(3000); // Simulate an asynchronous HTTP request
await Task.Run(() => Thread.Sleep(2000)); // Simulate synchronous code
}
catch
{
await Task.Run(() => Thread.Sleep(1000)); // Simulate synchronous code
}
finally
{
throttler.Release();
}
ProgressBar1.PerformStep();
}).ToArray();
await Task.WhenAll(tasks);
Button1.Enabled = true;
}

You can simply add a wrapper function:
private IProgress<double> _progress;
private int _jobsFinished = 0;
private int _totalJobs = 1000;
private static async Task FuncAsync()
{
try
{
await Task.Delay(3000); //assume this is an async http post request that takes 3 seconds to respond
Thread.Sleep(1000); //assume this is some synchronous code that takes 2 second
}
catch (Exception e)
{
Thread.Sleep(1000); //assume this is synchronous code that takes 1 second
}
}
private async Task AwaitAndUpdateProgress()
{
await FuncAsync(); // Can also do Task.Run(FuncAsync) to run on a worker thread
_jobsFinished++;
_progress.Report((double) _jobsFinished / _totalJobs);
}
And then just WhenAll after adding all the calls.

Is parallel asynchronous execution where a thread sleeps using multiple threads?

This is the code that I wrote to better understand asynchronous methods. I knew that an asynchronous method is not the same as multithreading, but it does not seem so in this particular scenario:
class Program
{
static void Main(string[] args)
{
Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo("en-US");
//the line above just makes sure that the console output uses . to represent doubles instead of ,
ExecuteAsync();
Console.ReadLine();
}
private static async Task ParallelAsyncMethod() //this is the method where async parallel execution is taking place
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Run(() => DownloadWebsite()));
}
var strings = await Task.WhenAll(tasks);
foreach (var str in strings)
{
Console.WriteLine(str);
}
}
private static string DownloadWebsite() //Imitating a website download
{
Thread.Sleep(1500); //making the thread sleep for 1500 miliseconds before returning
return "Download finished";
}
private static async void ExecuteAsync()
{
var watch = Stopwatch.StartNew();
await ParallelAsyncMethod();
watch.Stop();
Console.WriteLine($"It took the machine {watch.ElapsedMilliseconds} milliseconds" +
$" or {Convert.ToDouble(watch.ElapsedMilliseconds) / 1000} seconds to complete this task");
Console.ReadLine();
}
}
//OUTPUT:
/*
Download finished
Download finished
Download finished
Download finished
Download finished
It took the machine 1537 milliseconds or 1.537 seconds to complete this task
*/
As you can see, the DownloadWebsite method waits for 1.5 seconds and then returns "a". The method called ParallelAsyncMethod adds five of these methods into the "tasks" list and then starts the parallel asynchronous execution. As you can see, I also tracked the amount of time that it takes for the ExecuteAsync method to be executed. The result is always somewhere around 1540 milliseconds. Here is my question: if the DownloadWebsite method required a thread to sleep 5 times for 1500 milliseconds, does it mean that the parallel execution of these methods required 5 different threads? If not, then how come it only took the program 1540 milliseconds to be executed and not ~7500 ms?

I knew that an asynchronous method is not the same as multi-threading
That is correct, an asynchronous method releases the current thread whilst I/O occurs, and schedules a continuation after it's completion.
Async and threads are completely unrelated concepts.
but it does not seem so in this particular scenario
That is because you explicitly run DownloadWebsite on the ThreadPool using Task.Run, which imitates asynchronous code by returning a Task after instructing the provided delegate to run.
Because you are not waiting for each Task to complete before starting the next, multiple threads can be used simultaneously.
Currently each thread is being blocked, as you have used Thread.Sleep in the implementation of DownloadWebsite, meaning you are actually running 5 synchronous methods on the ThreadPool.
In production code your DownloadWebsite method should be written asynchronously, maybe using HttpClient.GetAsync:
private static async Task<string> DownloadWebsiteAsync()
{
//...
await httpClinet.GetAsync(//...
//...
}
In that case, GetAsync returns a Task, and releases the current thread whilst waiting for the HTTP response.
You can still run multiple async methods concurrently, but as the thread is released each time, this may well use less than 5 separate threads and may even use a single thread.
Ensure that you dont use Task.Run with an asynchronous method; this simply adds unnecessary overhead:
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(DownloadWebsiteAsync()); // No need for Task.Run
}
var strings = await Task.WhenAll(tasks);
As an aside, if you want to imitate an async operation, use Task.Delay instead of Thread.Sleep as the former is non-blocking:
private static async Task<string> DownloadWebsite() //Imitating a website download
{
await Task.Delay(1500); // Release the thread for ~1500ms before continuing
return "Download finished";
}

Why is this MVC method not running in parallel manner?

Background
I have an MVC 5 application and wanted to test if the requests were running in parallel. To do so I used the code below, and opened multiple pages all making the same request.
Code
Below is a relatively simple method where I wanted to the parallel nature.
public async Task<ActionResult> Login(string returnUrl, string message = "")
{
var rng = new Random();
var wait = rng.Next(3, 10);
var threadGuid = Guid.NewGuid();
DebugHelper.WriteToDebugLog($"Thread {threadGuid} about to wait {wait} seconds");
await Task.Delay(wait * 1000);
DebugHelper.WriteToDebugLog($"Thread {threadGuid} finished");
return View();
}
The class DebugHelper is just used so that I can write to a file safely.
public static class DebugHelper
{
private static readonly object WriteLock = new object();
public static void WriteToDebugLog(string message, string path = "C:\\Temp\\Log.txt")
{
lock (WriteLock)
{
File.AppendAllLines(path, new string[] { "", GetDateString(), message });
}
}
}
Output
I'm consistently getting this type of output which suggests the threads are blocking each other.
2020-03-24T13:43:43.1431913Z
Thread 6e42a6c5-d3cb-4541-b8aa-34b290952973 about to wait 7 seconds
2020-03-24T13:43:50.1564077Z
Thread 6e42a6c5-d3cb-4541-b8aa-34b290952973 finished
2020-03-24T13:43:50.1853278Z
Thread 90923f55-befd-4224-bdd8-b67f787839fc about to wait 4 seconds
2020-03-24T13:43:54.1943271Z
Thread 90923f55-befd-4224-bdd8-b67f787839fc finished
2020-03-24T13:43:54.2312257Z
Thread fa2d8d30-b762-4262-b188-0b34da5f4f04 about to wait 3 seconds
2020-03-24T13:43:57.2370556Z
Thread fa2d8d30-b762-4262-b188-0b34da5f4f04 finished
2020-03-24T13:43:57.2679690Z
Thread 37311a0e-d19e-4563-b92a-5e5e3def379a about to wait 8 seconds
2020-03-24T13:44:05.2812367Z
Thread 37311a0e-d19e-4563-b92a-5e5e3def379a finished
Question
Why is this occurring?
I was under the impression that any ASP.NET application was multithreaded to begin with, so even in a situation where I don't have the async/await setup, I thought it would run these threads simultaneously.
Update
As suggested in the answers/comments, my methodology was wrong. After using the following code I could see quite clearly in the logs that it was indeed running in parallel.
var targetTime = DateTime.UtcNow + TimeSpan.FromSeconds(5);
while(DateTime.UtcNow < targetTime)
{
DebugHelper.WriteToDebugLog($"Thread {threadGuid} with ID {threadId} doing stuff");
await Task.Delay(1000);
}

It simply boils down to the fact that your debug logging with its WriteLock and synchronous File.AppendAllLines forces a synchronization lock onto all asynchronous functions that call it.
You would do far better to have an asynchronous write to debug process that would allow your tasks to continue running.
Product/consumer patter, semaphores, events, use of asynchronous file access APIs all spring to mind.

If you are using session at all it can lock the user to a single thread. Check for controller level, page level, or filter/attribute session use. If you are unsure try adding
[SessionState(System.Web.SessionState.SessionStateBehavior.ReadOnly)]
to the controller.
Also, await by default will continue on the same thread that began the await. Try using configureAwait(false) to allow it to be flexible in the threads it uses.
await Task.Delay(wait * 1000).ConfigureAwait(false);

C# async/await progress reporting is not in expected order

I am experimenting with async/await and progress reporting and therefore have written an async file copy method that reports progress after every copied MB:
public async Task CopyFileAsync(string sourceFile, string destFile, CancellationToken ct, IProgress<int> progress) {
var bufferSize = 1024*1024 ;
byte[] bytes = new byte[bufferSize];
using(var source = new FileStream(sourceFile, FileMode.Open, FileAccess.Read)){
using(var dest = new FileStream(destFile, FileMode.Create, FileAccess.Write)){
var totalBytes = source.Length;
var copiedBytes = 0;
var bytesRead = -1;
while ((bytesRead = await source.ReadAsync(bytes, 0, bufferSize, ct)) > 0)
{
await dest.WriteAsync(bytes, 0, bytesRead, ct);
copiedBytes += bytesRead;
progress?.Report((int)(copiedBytes * 100 / totalBytes));
}
}
}
}
In a console application a create I file with random content of 10MB and then copy it using the method above:
private void MainProgram(string[] args)
{
Console.WriteLine("Create File...");
var dir = Path.GetDirectoryName(typeof(MainClass).Assembly.Location);
var file = Path.Combine(dir, "file.txt");
var dest = Path.Combine(dir, "fileCopy.txt");
var rnd = new Random();
const string chars = ("ABCDEFGHIJKLMNOPQRSTUVWXYZ1234567890");
var str = new string(Enumerable
.Range(0, 1024*1024*10)
.Select(i => letters[rnd.Next(chars.Length -1)])
.ToArray());
File.WriteAllText(file, str);
var source = new CancellationTokenSource();
var token = source.Token;
var progress = new Progress<int>();
progress.ProgressChanged += (sender, percent) => Console.WriteLine($"Progress: {percent}%");
var task = CopyFileAsync(file, dest, token, progress);
Console.WriteLine("Start Copy...");
Console.ReadLine();
}
After the application has executed, both files are identical, so the copy process is carried out in the correct order. However, the Console output is something like:
Create File...
Start Copy...
Progress: 10%
Progress: 30%
Progress: 20%
Progress: 60%
Progress: 50%
Progress: 70%
Progress: 80%
Progress: 40%
Progress: 90%
Progress: 100%
The order differs every time I call the application. I don't understand this behaviour. If I put a Breakpoint to the event handler and check each value, they are in the correct order. Can anyone explain this to me?
I want to use this later in a GUI application with a progress bar and don't want to have it jumping back and forward all the time.

Progress<T> captures current SynchronizationContext when created. If there is no SynchronizationContext (like in console app) - progress callbacks will be scheduled to thread pool threads. That means multiple callbacks can even run in parallel, and of course order is not guaranteed.
In UI applications, posting to synchronization context is roughly equivalent to:
In WPF: Dispatcher.BeginInvoke()
In WinForms: Control.BeginInvoke
I'm not working with WinForms, but in WPF, multiple BeginInvoke with the same priority (and in this case they are with the same priority) are guaranteed to execute in order they were invoked:
multiple BeginInvoke calls are made at the same DispatcherPriority,
they will be executed in the order the calls were made.
I don't see why in WinForms Control.BeginInvoke might execute our of order, but I'm not aware of a proof like I provided above for WPF. So I think in both WPF and WinForms you can safely rely on your progress callbacks to be executed in order (provided that you created Progress<T> instance itself on UI thread so that context could be captured).
Site note: don't forget to add ConfigureAwait(false) to your ReadAsync and WriteAsync calls to prevent returning to UI thread in UI applications every time after those awaits.

Number of threads required for IO bound async work

TL;DR:
Is it possible to kick off a series of IO bound tasks using only one thread using async await?
In less brief:
Trying to learn async await. In this video ("Async Best Practices for C# and Visual Basic"), the speaker gives an example of using async await to kick off some IO bound work. He explicitly says (at 21m 40s) whilst explaining why parallel for loops are not optimal as they use up loads of threads:
We don't need more threads for this. We don't need two threads...
Can we really kick off multiple requests asyncronously without using more than one thread? How? Unfortunately, the speaker didn't provide all the code so here's my stab at it:
// Pretty much exactly the same as video
private async Task<List<string>> LoadHousesAsync()
{
// Running on the UI thread
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
Task<string> t = LoadHouseAsync(i);
tasks.Add(t);
}
string[] loadedHouses = await Task.WhenAll(tasks);
return loadedHouses.ToList();
}
// My guess of the LoadHouseAsync method
private Task<string> LoadHouseAsync(int i)
{
// Running on the UI thread
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
return Task.Run(() => LoadHouse(i));
}
// My guess of the LoadHouse method
private string LoadHouse(int i)
{
// **** This is on a different thread :( ****
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
Thread.Sleep(5000); // simulate I/O bound work
return "House" + i;
}
Here's the output.
Thread: 10
Thread: 10
Thread: 3
Thread: 10
Thread: 10
Thread: 11
Thread: 10
Thread: 12
Thread: 10
Thread: 13
Thread: 14

You can do it with async I/O. What you made is a very nice example of doing it wrong (unfortunately, it's also quite common).
Task.Run runs a method on a thread pool thread, Thread.Sleep blocks the thread. So your example simulates doing synchronous (blocking) I/O on multiple threads.
To correctly perform async I/O, you need to use async methods all the way down. Never use Task.Run for I/O. You can simulate an asynchronous I/O method using Task.Delay:
private async Task<string> LoadHouseAsync(int i)
{
Debug.Print("Thread: " + Thread.CurrentThread.ManagedThreadId);
await Task.Delay(5000); // simulate async I/O bound work
return "House" + i;
}

Can we really kick off multiple requests asyncronously without using more than one thread?
Yes, here is a real world example (pseudo asp.net-mvc and entity framework, where EF will make IO calls to Sql Server for example).
public async ActionResult()
{
var model = new Company();
using (var db1 = new DbContext)
using (var db2 = new DbContext)
{
var task1 = db1.Employees.ToListAsync();
var task2 = db1.Managers.ToListAsync();
await Task.WhenAll(task1, task2);
model.employees = task1.Result;
model.managers = task2.Result;
}
return View(model);
}