We Keep Coding

How to implement a token system for limiting concurrency of a processor/IO heavy multithreading Tasks in C#?

This is a follow up question from this question: How do you run a variable number of concurrent parametrizable infinite loop type of threads in C#?
Say I have a value taskLimit (assume 20), of no of simultaneous Tasks "MyTask" which are created in the RunAsync method below:
protected override async Task RunAsync(CancellationToken cancellationToken)
{
var tasks = new List<Task>();
try
{
for (int i = 0; i < taskLimit; i++)
{
tasks.Add(MyTask(cancellationToken, i);
}
await Task.WhenAll(tasks);
}
catch (Exception e)
{
//Exception Handling
}
}
public async Task MyTask(CancellationToken cancellationToken, int a)
{
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
try
{
//long running code, if possible check for cancellation using the token
//Do something useful here. Very Processor and IO heavy. Takes 5-10 minutes to complete.
//SomeHeavyTask can only concurrently run upto a limit of say 5. Implement a token system of sorts
while(freeTokens<1)
{
await Task.Delay(TimeSpan.FromMinutes(1), cancellationToken);
}
freeTokens = freeTokens-1;
SomeHeavyTask(cancellationToken);
freeTokens = freeTokens+1;
//sleep for an independently parameterizable period, then wake up and repeat
await Task.Delay(TimeSpan.FromHours(parametrizableTaskDelay[i]), cancellationToken);
}
catch (Exception e)
{
//Exception Handling
}
}
}
Is it possible to do such a thing? Is there a better more formal approach supported natively in C# to achieve the same thing? Please note the essence of this question is that there are substantially less freeTokens than the taskLimit. And that each MyTask spends only 10% time in SomeHeavyTask(), and most of the time is spent in await Task.Delay().

You should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
int taskLimit = 500;
int maxConcurrent = 5;
IObservable<Unit> query =
Observable
.Range(0, taskLimit)
.Select(x => Observable.FromAsync(ct => SomeHeavyTask(ct)))
.Merge(maxConcurrent);
await query;
That's a lot easier to work with in my book.

Another option:
var block = new ActionBlock<int>(x => SomeHeavyTask(cancellationToken, x),
new ExecutionDataflowBlockOptions()
{
MaxDegreeOfParallelism = 20,
CancellationToken = cancellationToken
});
for (int i = 0; i < 100; i++)
await block.SendAsync(i, cancellationToken);
block.Complete();
await block.Completion;

You could use a SemaphoreSlim to limit the number of tasks that are working concurrently (you will still have taskLimit Tasks active, but only a limited number of those will be doing the heavy work simultaneously; I assume this is what you want).
This is best demonstrated with a sample console app. If you run this you'll see from the output that a maximum of 5 "heavy tasks" are active simultaneously.
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace Demo
{
class Program
{
static async Task Main()
{
Console.WriteLine("Starting");
// Cancel after 30 seconds for demo purposes.
using var source = new CancellationTokenSource(TimeSpan.FromSeconds(30));
await RunAsync(source.Token);
Console.WriteLine("Stopped.");
Console.ReadLine();
}
public static async Task RunAsync(CancellationToken cancellationToken)
{
int taskLimit = 20;
int concurrencyLimit = 5;
var sem = new SemaphoreSlim(concurrencyLimit);
var tasks = new List<Task>();
try
{
for (int i = 0; i < taskLimit; i++)
{
int p = i; // Prevent modified closure.
tasks.Add(Task.Run(() => MyTask(cancellationToken, p, sem)));
}
await Task.WhenAll(tasks);
}
catch (OperationCanceledException)
{
Console.WriteLine("Task(s) were cancelled.");
}
catch (Exception e)
{
// Exception Handling
}
}
public static async Task MyTask(CancellationToken cancellationToken, int a, SemaphoreSlim sem)
{
while (true)
{
cancellationToken.ThrowIfCancellationRequested();
try
{
await sem.WaitAsync(cancellationToken);
try
{
someHeavyTask(cancellationToken, a);
}
finally
{
sem.Release();
}
}
catch (OperationCanceledException)
{
Console.WriteLine("Task was cancelled.");
return;
}
catch (Exception e)
{
//Exception Handling
}
}
}
static int heavyTaskCount;
static void someHeavyTask(CancellationToken cancel, int a)
{
int n = Interlocked.Increment(ref heavyTaskCount);
Console.WriteLine("Starting heavy task. Number of simultaneous heavy tasks = " + n);
// Simulate work. Make the work for each task take varying time by using 'a' for the sleep.
for (int i = 0; i < 20 && !cancel.IsCancellationRequested; ++i)
{
Thread.Sleep(100 + a*10);
}
n = Interlocked.Decrement(ref heavyTaskCount);
Console.WriteLine("Finishing heavy task. Number of simultaneous heavy tasks = " + n);
}
}
}
The core of this is controlled by the semaphore in the code here:
await sem.WaitAsync(cancellationToken);
try
{
someHeavyTask(cancellationToken, a);
}
finally
{
sem.Release();
}

Like #mjwills said, you can use a C# semaphore to manage concurrent access to resources. (random example)
I do recommend looking at existing solutions first though. For example, Hangfire.
You can store its state inside SF if needed.

Is there a way to cancel execution of method in cancelled task?

I have a problem, that I could not find any answear yet. And this is my first project with use of threading and Tasks. When my task is cancelled, it keeps executing time consuming method.
Right now I have not idea how to stop execution of the method together with the task.
Below is a loop, that runs tasks. Every single task is running a ParseHorseData method, that runs also several other methods. Execution of them takes sometimes a lot of time.
After the task is cancelled, before await Task.WhenAll(tasks); is completed, it takes a lot of time.
So, as in the question, is there a way to cancel execution of method in cancelled task?
List<Task> tasks = new List<Task>();
int loopCounter = 0;
int taskCounter = 0;
//for all races in the file
for (int i = 0; i < _allRaces.Count; i ++)
{
int j = i;
if (TaskCancellation == true)
{
break;
}
Task task = Task.Run(async () =>
{
while (!_cancellationToken.IsCancellationRequested)
{
loopCounter++;
ProgressBarTick("Requesting historic data", loopCounter, _allRaces.Count, 0);
//if the race is from 2018
if (_allRaces[j].RaceDate.Year == 2018)
{
Category = _allRaces[j].RaceCategory;
Distance = _allRaces[j].RaceDistance.ToString();
//for all horses in the race
for (int h = 0; h < _allRaces[j].HorseList.Count; h++)
{
HorseDataWrapper horse = new HorseDataWrapper();
//TIME CONSUMING
horse = ParseHorseData(_allRaces[j].HorseList[h], _allRaces[j].RaceDate);
_allRaces[j].HorseList[h] = horse;
}
}
taskCounter++;
if (loopCounter >= _allRaces.Count)
{
ProgressBarTick("Testing on historic data", taskCounter, _allRaces.Count, 0);
}
}
}, _tokenSource.Token);
tasks.Add(task);
}
try
{
await Task.WhenAll(tasks);
}
catch (TaskCanceledException)
{
//
}
finally
{
_tokenSource.Dispose();
}

is there a way to cancel execution of method in cancelled task?
All cancellation is cooperative. The method being executed must pass the CancellationToken into the methods that it calls, and either:
Periodically poll for cancellation using ThrowIfCancellationRequested. This approach is more appropriate for CPU-bound loops.
Take an action on cancellation using Register. This approach is more appropriate for interfacing with non-CancellationToken-based cancellation systems.
In this case, it sounds like polling is appropriate. I strongly recommend polling via ThrowIfCancellationRequested and not IsCancellationRequested, because when a task is canceled, it should throw an OperationCanceledException when awaited. This is how the calling code knows it has been canceled.
Example:
Task task = Task.Run(async () =>
{
while (true)
{
_cancellationToken.ThrowIfCancellationRequested();
...
//for all horses in the race
for (int h = 0; h < _allRaces[j].HorseList.Count; h++)
{
_cancellationToken.ThrowIfCancellationRequested();
HorseDataWrapper horse = new HorseDataWrapper();
horse = ParseHorseData(_allRaces[j].HorseList[h], _allRaces[j].RaceDate);
_allRaces[j].HorseList[h] = horse;
}
...
}
});

MultiThreading - For loop doesnt waits on Await

I have a console application with have two threads as:
public static async void Thread1()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Thread1 " + i);
await MyFunc();
}
}
public static async void Thread2()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Thread2 " + i);
await MyFunc();
}
}
public static void Main(string[] args)
{
MainAsync(args).GetAwaiter().GetResult();
}
private static async Task MainAsync(string[] args)
{
Console.WriteLine("Before start thread");
Thread tid1 = new Thread(Thread1);
Thread tid2 = new Thread(Thread2);
tid1.Start();
tid2.Start();
}
public static async Task MyFunc()
{
//do something
}
However, when the application run and terminates, it seems that only each thread is run just once as I see only below things in output:
Before start thread
Thread1 0
Thread2 0
//some thing from MyFunc
I expect or rather wannt to run each thread till the for loop.It seems to me that the for loop continues to run despite the await.
If yes, what could be other probable approach.
Any leads would be helpful.

You aren't doing anything to wait for the threads. The main routine will just continue on until it returns to the O/S, which will kill the process and any child threads. Since you aren't doing anything else, this happens almost immediately, cutting both threads' lives short.
If you want to wait for the threads to finish, you can refer to this answer and write some variation of
while (thread1.IsAlive || thread2.IsAlive)
{
//Do something to wait
}
...before exiting.
That being said, you should probably using Tasks instead of threads, e.g.
public static async Task Task1()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Task1 " + i);
await MyFunc();
}
}
public static async Task Task2()
{
for (int i = 0; i < 100; i++)
{
Debug.WriteLine("Task2 " + i);
await MyFunc();
}
}
And then to execute and wait for both of them:
Task.WaitAll
(
new[]
{
Task1(),
Task2()
}
);
See this code in action on DotNetFiddle
See also What is the difference between tasks and threads?

You seem to have a lot of confusion about the role of threads and tasks, so it's a good idea to read up about it. Steven Cleary has a nice write-up about this. "There Is No Thread"
From the comments, it seems that your actual intention here is to run two async tasks in parallel, then to wait until they are both finished.
If you want to wait for two async tasks to complete in parallel, make sure your async methods actually return Task then:
Task task1 = DoSomethingAsync(); //don't await
Task task2 = DoSomethingElseAsync(); //don't await
then you can wait asynchronously for Task.WhenAll:
await Task.WhenAll(task1,task2);
You really don't need to be involving Thread at all.

Use async Task instead of async void
private static async Task MainAsync(string[] args)
{
Console.WriteLine("Before start thread");
var task1 = Thread1();
var task2 = Thread2();
var taskList = new [] { task1, task2 };
Task.WaitAll(taskList);
}

how to continue completed tasks?

public int CalcGroup(CancellationTokenSource cts)
{
try
{
CancellationToken ct = cts.Token;
if (cts.IsCancellationRequested == true && TaskWorkStatus.IsContinue == false) return 0;
for (int i = 0; i < _paralellTaskCount; i++)
{
int counter = CheckCounter(message);
if (counter >= 0)
{
var myTask = new Task<long>(() => CalcSingle(_personnelIds[counter].Item1), ct, TaskCreationOptions.LongRunning);
if(myTask.IsCompleted)
myTask.ContinueWith(t => CalcSingle(_personnelIds[counter].Item1), ct);
else
myTask.Start();
}
}
}
catch (Exception)
{
return 0;
}
return 1;
}
In above code block I want to run tasks that are Canceled and Completed, but it doesn't work.
What is my mistake?

0) Creating a new Task via the constructor is a bad practice since running it will later incur extra overhead(you can google the why's and how's later). Use Task.Run or Task.Factory.StartNew, depending on your .NET version.. (I've recently learned that Task.Run is a better practice for .NET 4.5 and later).
1) Creating a task does not start it, so the current logic will always skip the first condition, start the task and no continuation will ever occur.
2) even if you start the task, there is no guarantee that the task will finish in time for the condition checking.. this is why Task.ContinueWith has an overload which accepts a TaskContinuationOptions enumeration.. you have OnlyOnRanToCompletion, OnlyOnFaulted, etc..
3) To sum it up, here is your code, after a little fine tuning
for (int i = 0; i < _paralellTaskCount; i++)
{
object message;
int counter = CheckCounter(message);
if (counter >= 0)
{
var task = Task.Run(() => CalcSingle(_personnelIds[counter].Item1));
var continuation = task.ContinueWith((antecedent) => CalcSingle(_personnelIds[counter].Item1),
TaskContinuationOptions.OnlyOnRanToCompletion);
}
}
(Note that the continuation receives the antecedent task, which is the task it continues..
4) You do not seem to check if an exception has occured in any task, in any of the many ways provided.. note that wrapping the statement in a try catch clause will not transfer the exception to your thread.. (You need to access the Result or Exception properties, invoke Wait() or use the await keyword in order to accomplish that.
5) Here is an async implementation which does the same, and checks for exceptions
for (int i = 0; i < _paralellTaskCount; i++)
{
try
{
object message;
int counter = CheckCounter(message);
if (counter >= 0)
{
long res1 = await Task.Run(() => CalcSingle(_personnelIds[counter].Item1));
long res2 = await Task.Run(() => CalcSingle(_personnelIds[counter].Item1));
}
}
catch (AggregateException e)
{
//TODO handle
}
}
6) Please do read some articles which regard TPL, as it seems that you missed a few very important notions.

How can I insert a delay between a List of Task<int>?

I have a loop that creates 5 Tasks. How can I insert a Delay of 5 seconds between each Task. I don't know how to fit Task.Delay(5000) in there.
var tasks = new List<Task<int>>();
for (var i = 0; i < 5; i++)
{
tasks.Add(ProcessQueueAsync());
}
await Task.WhenAll(tasks);
My ProcessQueAsync method calls a server, retrieves data and returns and int.
private async Task<int> ProcessQueAsync()
{
var result = await CallToServer();
return result.Count;
}

for (var i = 0; i < 5; i++)
{
tasks.Add(ProcessQueueAsync());
await Task.Delay(5000);
}
Or:
for (var i = 0; i < 5; i++)
{
await ProcessQueueAsync();
await Task.Delay(5000);
}
Depending on that you want.

If you want the tasks to run one after the other, with a 5 second delay, you should perhaps look at Task.ContinueWith instead of using Task.WhenAll. This would allow you to run tasks in serial rather than in parallel.