We Keep Coding

Using IProgress when reporting progress for async await code vs progress bar control

private static async Task FuncAsync(DataTable dt, DataRow dr)
{
try
{
await Task.Delay(3000); //assume this is an async http post request that takes 3 seconds to respond
Thread.Sleep(1000) //assume this is some synchronous code that takes 2 second
}
catch (Exception e)
{
Thread.Sleep(1000); //assume this is synchronous code that takes 1 second
}
}
private async void Button1_Click(object sender, EventArgs e)
{
List<Task> lstTasks = new List<Task>();
DataTable dt = (DataTable)gridview1.DataSource;
foreach (DataRow dr in dt.Rows)
{
lstTasks.Add(FuncAsync(dr["colname"].ToString());
}
while (lstTasks.Any())
{
Task finishedTask = await Task.WhenAny(lstTasks);
lstTasks.Remove(finishedTask);
await finishedTask;
progressbar1.ReportProgress();
}
}
Assuming the datatable has got 10000 rows.
In the code, on button click, at the 1st iteration of the for loop, an async api request is made. While it takes 3 seconds, the control immediately goes to the caller. So the for loop can make the next iteration, and so on.
When the api response arrives, the code below the await runs as a callback. Thus blocking the UI thread and any incomplete for loop iterations will be delayed until the callback completes irrespective of whether I use await WhenAny or WhenAll.
All code runs on the UI thread due to the presence of synchronization context. I can do ConfigureAwait false on Task.Delay so the callbacks run on separate threads in order to unblock the ui thread.
Say 1000 iterations are made when the 1st await returns and when the 1st iterations await call back runs the following iterations will have completed completed awaits so their callbacks will run. Effectively callbacks will run one after the other if configure await is true. If false then they will run in parallel on separate threads.
So I think that the progress bar that I am updating in the while loop is incorrect because - by the time the code reaches the while block, most of the initial for loop iterations will have been already completed. I hope that I have understood correctly so far.
I have the following options to report progress from inside the task:
using IProgress (I think this is more suitable to report progress from another thread [for example when using Task.Run], or in usual async await if the configure await is false, resulting in code below the await to run in separate thread otherwise it will not show the progress bar moving as the ui thread will be blocked running the callbacks. In my current example code always runs on the same UI thread). So I was thinking the below point may be more appropriate solution.
making the Task non-static so that I can access the progress bar from within the Task and do porgressbar1.PerformStep().
Another thing I have noticed is that await WhenAll doesn't guarantee that IProgress is fully executed.

The IProgress<T> implementation offered natively by the .NET platform, the Progress<T> class, has the interesting characteristic of notifying the captured SynchronizationContext asynchronously, by invoking its Post method. This characteristic sometimes results to unexpected behavior. For example can you guess what effect has the code below to the Label1 control?
IProgress<string> progress = new Progress<string>(s => Label1.Text = s);
progress.Report("Hello");
Label1.Text = "World";
What text will be eventually written to the label, "Hello" or "World"? The correct answer is: "Hello". The delegate s => Label1.Text = s is invoked asynchronously, so it runs after the execution of the Label1.Text = "World" line, which is invoked synchronously.
Implementing a synchronous version of the Progress<T> class is quite trivial. All you have to do is copy-paste Microsoft's source code, rename the class from Progress<T> to SynchronousProgress<T>, and change the line m_synchronizationContext.Post(... to m_synchronizationContext.Send(.... This way every time you invoke the progress.Report method, the call will block until the invocation of the delegate on the UI thread is completed. The unfortunate implication of this is that if the UI thread is blocked for some reason, for example because you used the .Wait() or the .Result to wait synchronously for the task to complete, your application will deadlock.
The asynchronous nature of the Progress<T> class is rarely a problem in practice, but if you want to avoid thinking about it you can just manipulate the ProgressBar1 control directly. After all you are not writing a library, you are just writing code in the event handler of a button to make some HTTP requests. My suggestion is to forget about the .ConfigureAwait(false) hackery, and just let the main workflow of your asynchronous event handler to stay on the UI thread from start to end. If you have synchronous blocking code that needs to be offloaded to a ThreadPool thread, use the Task.Run method to offload it. To create your tasks, instead of manually adding tasks to a List<Task>, use the handly LINQ Select operator to project each DataRow to a Task. Also add a reference to the System.Data.DataSetExtensions assembly, so that the DataTable.AsEnumerable extension method becomes available. Finally add a throttler (a SemaphoreSlim), so that your application makes efficient use of the available network bandwidth, and it doesn't overburden the target machine:
private async void Button1_Click(object sender, EventArgs e)
{
Button1.Enabled = false;
const int maximumConcurrency = 10;
var throttler = new SemaphoreSlim(maximumConcurrency, maximumConcurrency);
DataTable dataTable = (DataTable)GridView1.DataSource;
ProgressBar1.Minimum = 0;
ProgressBar1.Maximum = dataTable.Rows.Count;
ProgressBar1.Step = 1;
ProgressBar1.Value = 0;
Task[] tasks = dataTable.AsEnumerable().Select(async row =>
{
await throttler.WaitAsync();
try
{
await Task.Delay(3000); // Simulate an asynchronous HTTP request
await Task.Run(() => Thread.Sleep(2000)); // Simulate synchronous code
}
catch
{
await Task.Run(() => Thread.Sleep(1000)); // Simulate synchronous code
}
finally
{
throttler.Release();
}
ProgressBar1.PerformStep();
}).ToArray();
await Task.WhenAll(tasks);
Button1.Enabled = true;
}

You can simply add a wrapper function:
private IProgress<double> _progress;
private int _jobsFinished = 0;
private int _totalJobs = 1000;
private static async Task FuncAsync()
{
try
{
await Task.Delay(3000); //assume this is an async http post request that takes 3 seconds to respond
Thread.Sleep(1000); //assume this is some synchronous code that takes 2 second
}
catch (Exception e)
{
Thread.Sleep(1000); //assume this is synchronous code that takes 1 second
}
}
private async Task AwaitAndUpdateProgress()
{
await FuncAsync(); // Can also do Task.Run(FuncAsync) to run on a worker thread
_jobsFinished++;
_progress.Report((double) _jobsFinished / _totalJobs);
}
And then just WhenAll after adding all the calls.

Locking issue with LimitedConcurrencyLevelTaskScheduler and aync/await

I'm struggling to understand what's happening in this simple program.
In the example below I have a task factory that uses the LimitedConcurrencyLevelTaskScheduler from ParallelExtensionsExtras with maxDegreeOfParallelism set to 2.
I then start 2 tasks that each call an async method (e.g. an async Http request), then gets the awaiter and the result of the completed task.
The problem seem to be that Task.Delay(2000) never completes. If I set maxDegreeOfParallelism to 3 (or greater) it completes. But with maxDegreeOfParallelism = 2 (or less) my guess is that there is no thread available to complete the task. Why is that?
It seems to be related to async/await since if I remove it and simply do Task.Delay(2000).GetAwaiter().GetResult() in DoWork it works perfectly. Does async/await somehow use the parent task's task scheduler, or how is it connected?
using System;
using System.Linq;
using System.Threading.Tasks;
using System.Threading.Tasks.Schedulers;
namespace LimitedConcurrency
{
class Program
{
static void Main(string[] args)
{
var test = new TaskSchedulerTest();
test.Run();
}
}
class TaskSchedulerTest
{
public void Run()
{
var scheduler = new LimitedConcurrencyLevelTaskScheduler(2);
var taskFactory = new TaskFactory(scheduler);
var tasks = Enumerable.Range(1, 2).Select(id => taskFactory.StartNew(() => DoWork(id)));
Task.WaitAll(tasks.ToArray());
}
private void DoWork(int id)
{
Console.WriteLine($"Starting Work {id}");
HttpClientGetAsync().GetAwaiter().GetResult();
Console.WriteLine($"Finished Work {id}");
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}
}
Thanks in advance for any help

await by default captures the current context and uses that to resume the async method. This context is SynchronizationContext.Current, unless it is null, in which case it is TaskScheduler.Current.
In this case, await is capturing the LimitedConcurrencyLevelTaskScheduler used to execute DoWork. So, after starting the Task.Delay both times, both of those threads are blocked (due to the GetAwaiter().GetResult()). When the Task.Delay completes, the await schedules the remainder of the HttpClientGetAsync method to its context. However, the context will not run it since it already has 2 threads.
So you end up with threads blocked in the context until their async methods complete, but the async methods cannot complete until there is a free thread in the context; thus a deadlock. Very similar to the standard "don't block on async code" style of deadlock, just with n threads instead of one.
Clarifications:
The problem seem to be that Task.Delay(2000) never completes.
Task.Delay is completing, but the await cannot continue executing the async method.
If I set maxDegreeOfParallelism to 3 (or greater) it completes. But with maxDegreeOfParallelism = 2 (or less) my guess is that there is no thread available to complete the task. Why is that?
There are plenty of threads available. But the LimitedConcurrencyTaskScheduler only allows 2 threads at a time to run in its context.
It seems to be related to async/await since if I remove it and simply do Task.Delay(2000).GetAwaiter().GetResult() in DoWork it works perfectly.
Yes; it's the await that is capturing the context. Task.Delay does not capture a context internally, so it can complete without needing to enter the LimitedConcurrencyTaskScheduler.
Solution:
Task schedulers in general do not work very well with asynchronous code. This is because task schedulers were designed for Parallel Tasks rather than asynchronous tasks. So they only apply when code is running (or blocked). In this case, LimitedConcurrencyLevelTaskScheduler only "counts" code that's running; if you have a method that's doing an await, it won't "count" against that concurrency limit.
So, your code has ended up in a situation where it has the sync-over-async antipattern, probably because someone was trying to avoid the problem of await not working as expected with limited concurrency task schedulers. This sync-over-async antipattern has then caused the deadlock problem.
Now, you could add in more hacks by using ConfigureAwait(false) everywhere and continue blocking on asynchronous code, or you could fix it better.
A more proper fix would be to do asynchronous throttling. Toss out the LimitedConcurrencyLevelTaskScheduler completely; concurrency-limiting task schedulers only work with synchronous code, and your code is asynchronous. You can do asynchronous throttling using SemaphoreSlim, as such:
class TaskSchedulerTest
{
private readonly SemaphoreSlim _mutex = new SemaphoreSlim(2);
public async Task RunAsync()
{
var tasks = Enumerable.Range(1, 2).Select(id => DoWorkAsync(id));
await Task.WhenAll(tasks);
}
private async Task DoWorkAsync(int id)
{
await _mutex.WaitAsync();
try
{
Console.WriteLine($"Starting Work {id}");
await HttpClientGetAsync();
Console.WriteLine($"Finished Work {id}");
}
finally
{
_mutex.Release();
}
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}

I think you are encountering a sync deadlock. You are waiting for a thread to complete that is waiting for your thread to complete. Never going to happen. If you make your DoWork method async so you can await the HttpClientGetAsync() call, and you'll avoid the deadlock.
using MassTransit.Util;
using System;
using System.Linq;
using System.Threading.Tasks;
//using System.Threading.Tasks.Schedulers;
namespace LimitedConcurrency
{
class Program
{
static void Main(string[] args)
{
var test = new TaskSchedulerTest();
test.Run();
}
}
class TaskSchedulerTest
{
public void Run()
{
var scheduler = new LimitedConcurrencyLevelTaskScheduler(2);
var taskFactory = new TaskFactory(scheduler);
var tasks = Enumerable.Range(1, 2).Select(id => taskFactory.StartNew(() => DoWork(id)));
Task.WaitAll(tasks.ToArray());
}
private async Task DoWork(int id)
{
Console.WriteLine($"Starting Work {id}");
await HttpClientGetAsync();
Console.WriteLine($"Finished Work {id}");
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}
}
https://medium.com/rubrikkgroup/understanding-async-avoiding-deadlocks-e41f8f2c6f5d
TLDR never call .result, which I'm sure .GetResult(); was doing

What causes a Task to complete?

I'm trying to find out how to use WhenAll to let two methods run at once, and once they both finish, collect the results without blocking by using .Result
I have this little console app test:
using System.Diagnostics;
using System.Threading.Tasks;
namespace ConsoleApplication2
{
class Program
{
public static void Main(string[] args)
{
var run = TaskRunner();
Debug.WriteLine(run);
if (run.IsCompleted)
{
Debug.WriteLine("this worked!");
} else
{
Debug.WriteLine("this failed!");
}
}
public static async Task<string> TaskRunner()
{
var taskOne = OneAsync();
var taskTwo = TwoAsync();
var tasks = await Task.WhenAll(taskOne, taskTwo);
var retval = tasks[0] + tasks[1];
return retval;
}
public static Task<string> OneAsync()
{
return Task.Run(() =>
{
return "test1";
});
}
public static Task<string> TwoAsync()
{
return Task.Run(() =>
{
return "test2";
});
}
}
}
This currently prints this worked! to my Output window... However, if I comment out Debug.WriteLine(run); it prints this failed!... Why does the Task complete simply by being logged to the output window?
I'm trying to understand a huge problem in a complex piece of code and this little test is my MCVE to hopefully shed some light on what is happening behind the scenes.

This happens just by pure chance. The way you are starting your task is with Task.Run. This essentially creates a new thread on which the (synchronous) action is executed. It returns a task for the completion of that thread.
So OneAsync and TwoAsync will each spawn a new thread that then immediately returns a string. This will happen very quickly but there’s still some overhead for creating those threads which means that it won’t be instantaneous.
TaskRunner then calls both those methods (spawning the threads), and then asynchronously waits for both threads to finish. Since the threads are not completely instantly, this TaskRunner method also won’t complete instantly.
Now, in your main, you are starting the asynchronous TaskRunner, which we figured will take “a very short moment”. You do not await the task, so the execution continues immediately. Debug.WriteLine is executed to print something (it probably doesn’t really matter that it’s the task in question that is being printed), and then you are checking the state of the task.
Since printing stuff is relatively slow (compared to other operations), this is probably the reason why the tasks ends up being completed. And when you remove the printing, the if is just reached too quickly for the task to finish.
As you likely noticed, working like that with asynchronous tasks does not appear to be a good idea. That’s why you should always await the task when you depend on its result.
// note the *async* here; async main methods are supported with C# 7.1
public static async void Main(string[] args)
{
var run = TaskRunner();
// await the task
await run;
if (run.IsCompleted)
{
Debug.WriteLine("this worked!");
}
else
{
Debug.WriteLine("this failed!");
}
}

TPL inside Windows Service

I need to perform few tasks inside a Windows Service I am writing in parallel. I am using VS2013, .NET 4.5 and this thread Basic design pattern for using TPL inside windows service for C# shows that TPL is the way to go.
Below is my implementation. I was wondering if anyone can tell me if I have done it correctly!
public partial class FtpLink : ServiceBase
{
private readonly CancellationTokenSource _cancellationTokenSource = new CancellationTokenSource();
private readonly ManualResetEvent _runCompleteEvent = new ManualResetEvent(false);
public FtpLink()
{
InitializeComponent();
// Load configuration
WebEnvironment.Instance.Initialise();
}
protected override void OnStart(string[] args)
{
Trace.TraceInformation("DatabaseToFtp is running");
try
{
RunAsync(_cancellationTokenSource.Token).Wait();
}
finally
{
_runCompleteEvent.Set();
}
}
protected override void OnStop()
{
Trace.TraceInformation("DatabaseToFtp is stopping");
_cancellationTokenSource.Cancel();
_runCompleteEvent.WaitOne();
Trace.TraceInformation("DatabaseToFtp has stopped");
}
private async Task RunAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
Trace.TraceInformation("Working");
// Do the actual work
var tasks = new List<Task>
{
Task.Factory.StartNew(() => new Processor().ProcessMessageFiles(), cancellationToken),
Task.Factory.StartNew(() => new Processor().ProcessFirmware(), cancellationToken)
};
Task.WaitAll(tasks.ToArray(), cancellationToken);
// Delay the loop for a certain time
await Task.Delay(WebEnvironment.Instance.DatabasePollInterval, cancellationToken);
}
}
}

There are a few things i would do differently:
OnStart should execute in a timely fashion. Common practice is to defer work to a background thread which is in charge of doing the actual work. You're actually doing that but blocking that thread with a call to Task.Wait, which kind of makes the offloading to a background thread useless, because execution becomes synchronous again.
You're using the sync over async anti-pattern, this should be mostly avoided. Let the calling method invoke the work in parallel.
I think you might be using the ManualResetEvent the other way around. You're wrapping your RunAsync method in a try-finally block, but you're only calling WaitOne from OnStop. I'm not really sure you need a lock here at all, it doesn't seem (from your current code) that this code is being invoked in parallel. Instead, you can store the Task returned by RunAsync in a field and wait on it to complete.
You're using the blocking version, WaitAll. Instead, you could use the asynchronous version, Task.WhenAll, which can be asynchronously waited.

Set thread lifetime at startup

Is there a way to set a value for how long a thread should (maximally) be alive when you start the thread?
Said in another way, with "pseudocode", is there anything like this:
Thread t = new Thread();
t.start();
t.abort_after_x_seconds(30);
which would make the thread abort if it lived more than 30 seconds.
Edit: I still can't get it to work, what I originally had is:
while(true)
{
if(...)
{
Thread t = new Thread(new ThreadStart(startMethod));
t.start();
}
Thread.sleep(...);
}
the problem is that sometimes the threads will hang (I'm not implementing what the threads do so I don't know exactly why (it's a school project, we're noobs at organizing)), so I want to kill those threads. I tried using Tasks and CancellationTokens as in the examples below, but when the Task hangs
it can't check if a cancellation request has occured.

Most of the time, you shouldn't be using Threads, use Tasks instead. They are more convenient and more efficient.
Aborting something is not safe, you should use cooperative cancellation instead. If you're calling a method that supports cancellation, then just pass it a cancellation token that will be cancelled after 30 seconds.
So your code could look like this (using .Net 4.5):
var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30)));
var task = Task.Run(() => YourMethod(cts.Token), cts.Token);

[EDIT: My response was far too slow. But I'll leave this here for the sample code.]
You should use co-operative cancellation for this purpose. The thread itself will need to detect when it should exit, and respond appropriately.
There's a thing called a CancellationToken produced from a CancellationTokenSource that you can use for this purpose.
There's even a CancellationTokenSource constructor which lets you set a timeout.
Here's some sample code to demonstrate its use:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace Demo
{
class Program
{
private void run()
{
using (var tokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(30)))
{
var task = Task.Run(() => exampleOne(tokenSource.Token));
task.Wait();
}
using (var tokenSource = new CancellationTokenSource(TimeSpan.FromSeconds(30)))
{
var task = Task.Run(() => exampleTwo(tokenSource.Token));
task.Wait();
}
Console.WriteLine("Done.");
}
static void exampleZero()
{
Console.WriteLine("Starting exampleZero()");
try
{
Thread.Sleep(10000); // Simulate work.
}
catch (OperationCanceledException)
{
Console.WriteLine("Operation cancelled.");
}
Console.WriteLine("Exiting exampleZero()");
}
static void exampleOne(CancellationToken cancellation)
{
Console.WriteLine("Starting exampleOne()");
// Busy loop processing.
while (!cancellation.IsCancellationRequested)
{
// Do some work.
}
Console.WriteLine("Exiting exampleOne()");
}
static void exampleTwo(CancellationToken cancellation)
{
Console.WriteLine("Starting exampleTwo()");
while (!cancellation.WaitHandle.WaitOne(100)) // Wait 100ms between work.
{
// Do some work.
}
Console.WriteLine("Exiting exampleTwo()");
}
static void Main()
{
new Program().run();
}
}
}

As commenters have said, using Abort is bad practice and not guaranteed to abort immediately.
Why would you want to keep the thread alive? The thread will be released back to the pool when the task assigned to it is completed. The next time the task is run on the thread will automatically be given from the pool of threads either by creating another new one or re-using one that is available in the threadpool.
Sounds like your logic/code is bad and needs to be fixed rather than waiting for something for x seconds then terminating it, which in itself will cause knock on problems.
Perhaps you need a timer instead which can tick after 30 seconds then you can disable the timer and kill the task at hand.