We Keep Coding

How to synchronize a shared IProgress<int>

I have an asynchronous method DoStuffAsync that spawns two tasks with Task.Run, and both tasks report their progress using a single IProgress<int> object. From the user's perspective there is only one operation, so showing two progress bars (one for each Task) wouldn't make any sense. This is why the IProgress<int> is shared. The problem is that sometimes the UI receives the progress notifications in incorrect order. Here is my code:
private async void Button1_Click(object sender, EventArgs e)
{
TextBox1.Clear();
var progress = new Progress<int>(x => TextBox1.AppendText($"Progress: {x}\r\n"));
await DoStuffAsync(progress);
}
async Task DoStuffAsync(IProgress<int> progress)
{
int totalPercentDone = 0;
Task[] tasks = Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
var localPercentDone = Interlocked.Add(ref totalPercentDone, 10);
progress.Report(localPercentDone);
}
})).ToArray();
await Task.WhenAll(tasks);
}
Most of the time the notifications are in the correct order, but sometimes they are not:
This causes the ProgressBar control (not shown in the above screenshot) to jump awkwardly back and forth.
As a temporary solution I have added a lock inside the DoStuffAsync method, that includes the invocation of the IProgress.Report method:
async Task DoStuffAsync(IProgress<int> progress)
{
int totalPercentDone = 0;
object locker = new object();
Task[] tasks = Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
lock (locker)
{
totalPercentDone += 10;
progress.Report(totalPercentDone);
};
}
})).ToArray();
await Task.WhenAll(tasks);
}
Although this solves the problem, it causes me anxiety because I invoke arbitrary code while holding a lock. The DoStuffAsync method is actually part of a library, and could be called with a whatever IProgress<int> implementation as argument. This opens the possibility for deadlock scenarios. Is there a better way to implement the DoStuffAsync method, without using a lock, but with the desired behavior regarding the ordering of the notifications?

Your problem is that you need the increment of totalPercentDone AND the call to Report to be atomic.
There's nothing wrong with using a lock here. After all, you need some way to make the two operations atomic. If you really don't want to use lock then you could use a SemaphoireSlim:
async Task DoStuffAsync(IProgress<int> progress)
{
int totalPercentDone = 0;
var semaphore = new SemaphoreSlim(1,1);
Task[] tasks = Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
await semaphore.WaitAsync();
try
{
totalPercentDone += 10;
progress.Report(totalPercentDone);
}
finally
{
semaphore.Release();
}
}
})).ToArray();
await Task.WhenAll(tasks);
}

You could just report the deltas and let the handling handle them:
private async void Button1_Click(object sender, EventArgs e)
{
TextBox1.Clear();
var totalPercentDone = 0;
var progress = new Progress<int>(x =>
{
totalPercentDone += x;
TextBox1.AppendText($"Progress: {totalPercentDone}\r\n"));
}
await DoStuffAsync(progress);
}
async Task DoStuffAsync(IProgress<int> progress)
{
await Task.WhenAll(Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
progress.Report(10);
}
})));
}

Instead of using one int for both tasks you could use two individual ints, and take the smallest of them. Each Task need to report to 100, and not 50.
async Task DoStuffAsync(IProgress<int> progress)
{
int[] totalPercentDone = new int[2];
Task[] tasks = Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
totalPercentDone[n - 1] += 10;
progress.Report(totalPercentDone.Min());
}
})).ToArray();
await Task.WhenAll(tasks);
}

This is to extend my comments under question
Basically, progress is usually a forward-only value. With regards to reporting progress, it is likely that you never need to report a progress made in the past. Even you do, in most cases the client / event handler side would still drop such values received.
The problem here / why you need to synchronize reporting is mainly because you are reporting a progress of value type, whose value got copied when Report(T) is called.
You can simply avoid locking by reporting a reference type instance with the latest progress made:
public class DoStuffProgress
{
private volatile int _percentage;
public int Percentage => _percentage;
internal void IncrementBy(int increment)
{
Interlocked.Add(ref _percentage, increment);
}
}
Now your code looks like:
async Task DoStuffAsync(IProgress<DoStuffProgress> progress)
{
DoStuffProgress totalPercentDone = new DoStuffProgress();
Task[] tasks = Enumerable.Range(1, 2).Select(n => Task.Run(async () =>
{
for (int i = 0; i < 5; i++)
{
await Task.Delay(100); // Simulate an I/O operation
totalPercentDone.IncrementBy(10);
// Report reference type object
progress.Report(totalPercentDone);
}
})).ToArray();
await Task.WhenAll(tasks);
}
The client, however, may receive notification with duplicate value:
Progress: 20
Progress: 20
Progress: 40
Progress: 40
Progress: 60
Progress: 60
Progress: 80
Progress: 80
Progress: 90
Progress: 100
But, the values should never be out of order.

MultiThreading - For loop doesnt waits on Await

I have a console application with have two threads as:
public static async void Thread1()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Thread1 " + i);
await MyFunc();
}
}
public static async void Thread2()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Thread2 " + i);
await MyFunc();
}
}
public static void Main(string[] args)
{
MainAsync(args).GetAwaiter().GetResult();
}
private static async Task MainAsync(string[] args)
{
Console.WriteLine("Before start thread");
Thread tid1 = new Thread(Thread1);
Thread tid2 = new Thread(Thread2);
tid1.Start();
tid2.Start();
}
public static async Task MyFunc()
{
//do something
}
However, when the application run and terminates, it seems that only each thread is run just once as I see only below things in output:
Before start thread
Thread1 0
Thread2 0
//some thing from MyFunc
I expect or rather wannt to run each thread till the for loop.It seems to me that the for loop continues to run despite the await.
If yes, what could be other probable approach.
Any leads would be helpful.

You aren't doing anything to wait for the threads. The main routine will just continue on until it returns to the O/S, which will kill the process and any child threads. Since you aren't doing anything else, this happens almost immediately, cutting both threads' lives short.
If you want to wait for the threads to finish, you can refer to this answer and write some variation of
while (thread1.IsAlive || thread2.IsAlive)
{
//Do something to wait
}
...before exiting.
That being said, you should probably using Tasks instead of threads, e.g.
public static async Task Task1()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Task1 " + i);
await MyFunc();
}
}
public static async Task Task2()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Task2 " + i);
await MyFunc();
}
}
And then to execute and wait for both of them:
Task.WaitAll
(
new[]
{
Task1(),
Task2()
}
);
See this code in action on DotNetFiddle
See also What is the difference between tasks and threads?

You seem to have a lot of confusion about the role of threads and tasks, so it's a good idea to read up about it. Steven Cleary has a nice write-up about this. "There Is No Thread"
From the comments, it seems that your actual intention here is to run two async tasks in parallel, then to wait until they are both finished.
If you want to wait for two async tasks to complete in parallel, make sure your async methods actually return Task then:
Task task1 = DoSomethingAsync(); //don't await
Task task2 = DoSomethingElseAsync(); //don't await
then you can wait asynchronously for Task.WhenAll:
await Task.WhenAll(task1,task2);
You really don't need to be involving Thread at all.

Use async Task instead of async void
private static async Task MainAsync(string[] args)
{
Console.WriteLine("Before start thread");
var task1 = Thread1();
var task2 = Thread2();
var taskList = new [] { task1, task2 };
Task.WaitAll(taskList);
}

Threading in Forloop C#

I got a requirement where I need to process two Threads simultaneously in a ForLoop.
For Example :
private void BtnThreading_Click(object sender, EventArgs e)
{
for (int i = 0; i < 20; i++)
{
Thread thread1 = new Thread(ProcessA);
thread1.Start();
thread1.Join();
Thread thread2 = new Thread(ProcessB);
thread2.Start();
}
}
private void ProcessA()
{
Thread.Sleep(10000);
}
private void ProcessB()
{
Thread.Sleep(20000);
}
For the First Time in the for-loop,Let's say the ProcessA() takes 10 Seconds to complete and I need to wait till the ProcessA() to finish using thread1.Join(); to Start Processing ProcessB().Later the ProcessB() starts and it will take 20 seconds to finish.
So,In the mean while ProcessA() again starts and thread1.Join(); statement will wait until ProcessA() finishes.Here I need to wait also for the previous ProcessB() to finish.
so finally,I want the ProcessB() to wait for the Previous Thread of ProcessB()
Sorry for my Bad English !!! :)

Unless your really need to do low level programming you should not use Thread objects directly. Each thread is quite expensive in terms of resources. Instead .NET provides several high level abstractions to do asynchronous and parallel programming. Your loop can be built using async and await and will then execute on thread pool threads. Here is an example:
Instead of using Thread.Sleep to simulate delay you have to use Task.Delay:
async Task ProcessA() {
await Task.Delay(10000);
}
async Task ProcessB() {
await Task.Delay(20000);
}
The loop:
var task = Task.Delay(0); // No operation task to simplify loop.
for (var i = 0; i < 20; i += 1) {
await ProcessA();
await task;
task = ProcessB();
}
await task;

It looks like you don't really need ProcessA to execute in a different thread at all - but you do need to keep track of your previous ProcessB thread. So something like:
Thread previousThread = null;
for (int i = 0; i < 20; i++)
{
ProcessA();
if (previousThread != null)
{
previousThread.Join();
}
previousThread = new Thread(ProcessB);
previousThread.Start();
}
// Possibly join on previousThread here too
Note that your method name suggests that you're doing this in a UI thread - which you really, really shouldn't. Don't block the UI thread for any length of time - and remember that Join is a blocking call.

You can give this code a try, but again it is not production ready..!!!
Thread thread1 = null;
Thread thread2 = null;
private void BtnThreading_Click(object sender, EventArgs e)
{
for (int i = 0; i < 20; i++)
{
thread1 = new Thread(ProcessA);
thread1.Start();
thread1.Join();
if (thread2 != null)
thread2.Join();
thread2 = new Thread(ProcessB);
thread2.Start();
}
}

Not slowing down a loop of tasks, and know when result is returned

Task<string> RunList(int client)
{
return pages[client];
}
private async void Form1_DoubleClick(object sender, EventArgs e)
{
for (int x = 0; x < listBox1.Items.Count; x++)
{
RunList(x);
}
}
This will fly through the loop of tasks, but how do you know when the results are all in without compromising speed of the loop?

You can await on the result of WhenAll to ensure that all of the tasks have completed at that point in code. (It's important not to use WaitAll here, that would block the UI thread.)
private async void Form1_DoubleClick(object sender, EventArgs e)
{
var tasks = new List<Task<string>>();
for (int x = 0; x < listBox1.Items.Count; x++)
{
tasks.Add(RunList(x));
}
await Task.WhenAll(tasks);
}
The basic idea here is to simply start the tasks before calling await on them. Here is a simpler example with just two Tasks:
await Task.Delay(1000);
await Task.Delay(1000);
This will perform the first task and then the second task.
var task1 = Task.Delay(1000);
var task2 = Task.Delay(1000);
await task1;
await task2;
This will start both tasks and then continue on after both tasks have finished, allowing then to run concurrently.

Task synchronization without a UI thread

In the code below I want to syncronize the reporting of the results of a list of tasks. This is working now because task.Result blocks until the task completes. However, task id = 3 takes a long time to complete and blocks all of the other finished tasks from reporting their status.
I think that I can do this by moving the reporting (Console.Write) into a .ContinueWith instruction but I don't have a UI thread so how do I get a TaskScheduler to syncronize the .ContinueWith tasks?
What I have now:
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
var tasks = new List<Task<int>>();
for (var i = 0; i < 10; i++)
{
var num = i;
var t = Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 5000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
});
tasks.Add(t);
}
foreach (var task in tasks)
{
Console.WriteLine("Completed {0} on {1}", task.Result, Thread.CurrentThread.ManagedThreadId);
}
Console.WriteLine("End of Main");
Console.ReadKey();
}
I would like to move to this or something similar but I need the Console.Write("Completed...") to all happen on the same thread:
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
for (var i = 0; i < 10; i++)
{
var num = i;
Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
}).ContinueWith(value =>
{
Console.WriteLine("Completed {0} on {1}", value.Result, Thread.CurrentThread.ManagedThreadId);
}
/* need syncronization context */);
}
Console.WriteLine("End of Main");
Console.ReadKey();
}
-- SOLUTION --
After getting some comments and reading some of the solutions this is the complete solution that does what I want. The goal here is to process severl long running tasks as fast as possible and then do something with the results of each task one at a time.
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
var results = new BlockingCollection<int>();
Task.Factory.StartNew(() =>
{
while (!results.IsCompleted)
{
try
{
var x = results.Take();
Console.WriteLine("Completed {0} on {1}", x, Thread.CurrentThread.ManagedThreadId);
}
catch (InvalidOperationException)
{
}
}
Console.WriteLine("\r\nNo more items to take.");
});
var tasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
var num = i;
var t = Task.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
results.Add(num);
});
tasks.Add(t);
}
Task.Factory.ContinueWhenAll(tasks.ToArray(), _ => results.CompleteAdding());
Console.WriteLine("End of Main");
Console.ReadKey();
}

You'll have to create a writer task of some sort, however, keep in mind even this task can be rescheduled onto another native or managed thread! Using the default scheduler in TPL you have no control over which managed thread receives the work.
public class ConcurrentConsole
{
private static BlockingCollection<string> output
= new BlockingCollection<string>();
public static Task CreateWriterTask(CancellationToken token)
{
return new Task(
() =>
{
while (!token.IsCancellationRequested)
{
string nextLine = output.Take(token);
Console.WriteLine(nextLine);
}
},
token);
}
public static void WriteLine(Func<string> writeLine)
{
output.Add(writeLine());
}
}
When I switched your code to use this I received the following output:
End of Main
Done 1 on 6
Completed 1 on 6
Done 5 on 9
Completed 5 on 9
Done 0 on 4
Completed 0 on 4
Done 2 on 5
Completed 2 on 13
Done 7 on 10
Completed 7 on 10
Done 4 on 8
Completed 4 on 5
Done 9 on 12
Completed 9 on 9
Done 6 on 6
Completed 6 on 5
Done 8 on 11
Completed 8 on 4
Done 3 on 7
Completed 3 on 7
Even with your code sending () => String.Format("Completed {0} on {1}"... to ConcurrentConsole.WriteLine, ensuring the ManagedThreadId would be picked up on the ConcurrentConsole Task, it still would alter which thread it ran on. Although with less variability than the executing tasks.

You can use OrderedTaskScheduler to ensure only one task completion is run at a time; however, they will run on a threadpool thread (not necessarily all on the same thread).
If you really need them all on the same thread (not just one at a time), then you can use ActionThread from the Nito.Async library. It provides a SynchronizationContext for its code, which can be picked up by FromCurrentSynchronizationContext.

I would suggest:
1) Creating a lock object
2) Create a list of strings to be written
3) Spawn a thread that loops, sleeping for a bit, then locking the list of strings, then if it isn't empty, writing all of them and emptying the list
4) Other threads then lock the list, add their status, unlock and continue.
object writeListLocker = new object();
List<string> linesToWrite = new List<string>();
// Main thread loop
for (; ; )
{
lock (writerListLocker)
{
foreach (string nextLine in linesToWrite)
Console.WriteLine(nextLine);
linesToWrite.Clear();
}
Thread.Sleep(500);
}
// Reporting threads
lock (writerListLocker)
{
linesToWrite.Add("Completed (etc.)");
}

I think you expect a result like the following.
Starting on 8
Done 1 on 11
Completed 1 on 9
Done 5 on 11
Completed 5 on 9
Done 0 on 10
Completed 0 on 9
Done 2 on 12
Completed 2 on 9
Done 7 on 16
Completed 7 on 9
Done 4 on 14
Completed 4 on 9
Done 9 on 18
Completed 9 on 9
Done 6 on 15
Completed 6 on 9
Done 8 on 17
Completed 8 on 9
Done 3 on 13
Completed 3 on 9
As below, I used the StaSynchronizationContext in my code from the Understanding SynchronizationContext where a synchronized call in one thread is explained well. Please, refer to it.
My code snippet is:
static void Main(string[] args)
{
StaSynchronizationContext context = new StaSynchronizationContext();
StaSynchronizationContext.SetSynchronizationContext(context);
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
for (var i = 0; i < 10; i++)
{
var num = i;
Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
}).ContinueWith(
value =>
{
Console.WriteLine("Completed {0} on {1}", value.Result, Thread.CurrentThread.ManagedThreadId);
}
,TaskScheduler.FromCurrentSynchronizationContext());
}
Console.WriteLine("End of Main");
Console.ReadKey();
}