We Keep Coding

Stop foreach in asynchronous task on key press passing a cancellation token

I'm running a foreach and I would like to cancel its execution on a key press.
While I succeeded doing it integrating a single if (keypress) within the loop, now I'm trying to achieve the same using a CancellationToken while the task for listening for a key stroke is running.
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task.Factory.StartNew(() =>
{
while (true)
{
foreach (var station in stations)
{
/*if (Console.KeyAvailable)
{
break;
}*/
Console.WriteLine(station.name + " ");
Thread.Sleep(100);
}
Thread.Sleep(100);
if (ct.IsCancellationRequested)
{
// another thread decided to cancel
Console.WriteLine("task canceled");
break;
}
}
}, ct);
ts.Cancel();
Console.ReadLine();
I came from this answer How do I abort/cancel TPL Tasks? which helped me a lot.
However, while it works without the foreach, right now the foreach has to end before the task is cancelled.
Obviously, it looks like the iteration has to end before proceeding to the next step and what I don't understand is how I can make the foreach stop.

Cancellation is co-operative.
You need to check inside the foreach() and at any other appropriate point in your routines, to see if cancellation has been requested.
If it has, then you could prematurely exit out of the foreach(), (if your logic allows it), while ensuring that you clean up any resources and complete any actions that need to be completed.
Also, I suggest, instead of Thread.Sleep(), use await Task.Delay(100);.

It seems as though it's as simple as repeating your if:
var ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
Task.Factory.StartNew(() =>
{
while (true)
{
foreach (var station in stations)
{
/*if (Console.KeyAvailable)
{
break;
}*/
if (ct.IsCancellationRequested)
{
// another thread decided to cancel
break;
}
Console.WriteLine(station.name + " ");
Thread.Sleep(100);
}
Thread.Sleep(100);
if (ct.IsCancellationRequested)
{
// another thread decided to cancel
Console.WriteLine("task canceled");
break;
}
}
}, ct);
ts.Cancel();
Console.ReadLine();

As I commented before, I think you're trying to achieve something like this:
public class Program
{
static ConcurrentQueue<int> queue = new(); // >= .NET 5 / C# 9 only
public static void Main(string[] args)
{
var ts = new CancellationTokenSource();
var rand = new Random();
Task activeTask = null;
while (true)
{
var keyPressed = Console.ReadKey(true).Key;
if (keyPressed == ConsoleKey.Q)
{
for (int i = 0; i < 1000; i++)
queue.Enqueue(rand.Next(0, 1000));
Console.WriteLine("Random elements enqueued");
}
if (keyPressed == ConsoleKey.D)
{
if(activeTask == null)
{
ts.Dispose();
ts = new CancellationTokenSource();
CancellationToken ct = ts.Token;
activeTask = BackgroundLoopAction(ct);
}
else
Console.WriteLine("Background loop task already running");
}
if (keyPressed == ConsoleKey.X)
{
ts.Cancel();
activeTask = null;
}
if (keyPressed == ConsoleKey.Escape)
break;
}
}
private static Task BackgroundLoopAction(CancellationToken ct)
{
return Task.Run(() =>
{
while (queue.Count > 0)
{
queue.TryDequeue(out int q);
Console.WriteLine($"Dequeued element: {q}");
Thread.Sleep(100);
if (ct.IsCancellationRequested)
break;
}
Console.WriteLine(ct.IsCancellationRequested ? "Task canceled by user event" : "Task completed");
}, ct);
}
}

Using threadpool with thread completion updates inside a lock

Following several tutorials on threaded app development, I've implemented a small practice utility that uses both Task.Run, and Threadpool.QueueUserWorkItem to recursively index a folder and generate a checksum manifest of all contained files.
While the application works well, I'm unable to get the UI to update during the portion that specifically uses a threadpool. The UI does update correctly using await Application.Current.Dispatcher.BeginInvoke when using Task.Run.
The desired behavior is for the progress to continue to update within ProcessChecksums as each of the threadpool's threads finish their individual task.
int queuedThreads = 0;
object locker = new object();
CancellationTokenSource cancellationTokenSource;
List<string> lstFilePaths = new List<string>();
Manifest manifestData;
event Action<double> OnProgressChange;
event Action<string> OnStatusChange;
async void GenerateManifest(Object sender, RoutedEventArgs e) {
status = "Indexing Files";
progress = 1;
cancellationTokenSource = new CancellationTokenSource();
await Task.Run(() => Async_GenerateManifest(cancellationTokenSource.Token), cancellationTokenSource.Token);
if (!cancellationTokenSource.Token.IsCancellationRequested) {
Finished();
}else{
Cancelled();
}
}
async Task Async_GenerateManifest(CancellationToken cancellationToken) {
if (cancellationToken.IsCancellationRequested) { return; }
Async_IndexFiles(initialpath, cancellationToken);
//Works Perfectly
await Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Background, new Action(() => {
OnStatusChange("Generating Checksums");
OnProgressChange(10);
}));
ProcessChecksums(cancellationToken);
//Works Perfectly
await Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Background, new Action(() => {
OnStatusChange("Saving Manifest");
OnProgressChange(90);
}));
SaveManifestFile(cancellationToken);
}
void ProcessChecksums(CancellationToken cancellationToken) {
List<FileData> lstFileData = new List<FileData>();
for (int i = 0; i < lstFilePaths.Count; i++) {
if (cancellationToken.IsCancellationRequested) { return; }
string s = lstFilePaths[i];
lock (locker) queuedThreads++;
ThreadPool.QueueUserWorkItem( x => {
manifestData.AddFileData(new FileData(s, GenerateChecksum(s)));
});
}
lock (locker) {
while (queuedThreads > 0) {
if (cancellationToken.IsCancellationRequested) { return; }=
Monitor.Wait(locker);
//Can't use await Dispatcher.BeginInvoke as is inside a lock, doesn't update GUI while waiting.
OnProgressChange((((queuedThreads - lstFilePaths.Count) * -1) / lstFilePaths.Count) - 0.2);
}
}
}
string GenerateChecksum(string filePath) {
//Time-consuming checksum generation
//...
lock (locker) {
queuedThreads--;
Monitor.Pulse(locker);
}
return BitConverter.ToString(checksum);
}

The standard pattern for updating the UI with progress updates from a background thread is to use IProgress<T>/Progress<T>. The more modern approach has several benefits over direct use of Dispatcher.
// You'd want to set these while on the UI thread.
// E.g., in your constructor:
// _percentProgress = new Progress<double>(value => ...);
// _statusProgress = new Progress<string>(value => ...);
IProgress<double> _percentProgress;
IProgress<string> _statusProgress;
async void GenerateManifest(Object sender, RoutedEventArgs e) {
status = "Indexing Files";
progress = 1;
cancellationTokenSource = new CancellationTokenSource();
await Task.Run(() => GenerateManifest(cancellationTokenSource.Token));
if (!cancellationTokenSource.Token.IsCancellationRequested) {
Finished();
}else{
Cancelled();
}
}
void GenerateManifest(CancellationToken cancellationToken) {
if (cancellationToken.IsCancellationRequested) { return; }
Async_IndexFiles(initialpath, cancellationToken);
_statusProgress?.Report("Generating Checksums");
_percentProgress?.Report(10);
ProcessChecksums(cancellationToken);
_statusProgress?.Report("Saving Manifest");
_percentProgress?.Report(90);
SaveManifestFile(cancellationToken);
}
void ProcessChecksums(CancellationToken cancellationToken) {
List<FileData> lstFileData = new List<FileData>();
for (int i = 0; i < lstFilePaths.Count; i++) {
if (cancellationToken.IsCancellationRequested) { return; }
string s = lstFilePaths[i];
lock (locker) queuedThreads++;
ThreadPool.QueueUserWorkItem( x => {
manifestData.AddFileData(new FileData(s, GenerateChecksum(s)));
});
}
lock (locker) {
while (queuedThreads > 0) {
if (cancellationToken.IsCancellationRequested) { return; }
Monitor.Wait(locker);
_percentProgress?.Report((((queuedThreads - lstFilePaths.Count) * -1) / lstFilePaths.Count) - 0.2);
}
}
}
There are a few other problems with the code, too. QueueUserWorkItem should probably be replaced with Task.Run and an await Task.WhenAll to ensure exceptions don't tear down the process. The if (...) Finished(); else Cancelled(); is probably best represented by changing the return type to Task. Use ThrowIfCancellationRequested instead of IsCancellationRequested. And there's a kind of Monitor.Wait without a Pulse that is just used for progress updates, which is odd; it would be cleaner to allow that code to report its own progress. If you take a look at each of these individually, you should find your code ends up cleaner.

Wait for all threads to release semaphore?

How do I close down and wait for a semaphore to be fully released?
private SemaphoreSlim _processSemaphore = new SemaphoreSlim(10);
public async Task<Modification> Process(IList<Command> commands)
{
Assert.IsFalse(_shuttingDown, "Server is in shutdown phase");
await _processSemaphore.WaitAsync();
try
{
// threads that have reached this far must be allowed to complete
return _database.Process(commands);
}
finally
{
_processSemaphore.Release();
}
}
public async Task StopAsync()
{
_shuttingDown = true;
// how wait for threads to complete without cancelling?
await ?
}

private SemaphoreSlim _processSemaphore = new SemaphoreSlim(10);
private int _concurrency;
private TaskCompletionSource<int> _source;
private ManualResetEvent _awaitor;
public void Start()
{
//solution 1
_concurrency = 0;
_source = new TaskCompletionSource<int>();
_shuttingDown = false;
//solution 2
_awaitor = new ManualResetEvent(false);
//your code
}
public async Task<Modification> Process(IList<Command> commands)
{
Interlocked.Increment(ref _concurrency);
Assert.IsFalse(_shuttingDown, "Server is in shutdown phase");
await _processSemaphore.WaitAsync();
try
{
// threads that have reached this far must be allowed to complete
return _database.Process(commands);
}
finally
{
_processSemaphore.Release();
//check and release
int concurrency = Interlocked.Decrement(ref _concurrency);
if (_shuttingDown && concurrency == 0)
{
//solution 1
_source.TrySetResult(0);
//solution 2
_awaitor.Set();
}
}
}
public async Task StopAsync()
{
_shuttingDown = true;
// how wait for threads to complete without cancelling?
if (Interlocked.CompareExchange(ref _concurrency, 0, 0) != 0)
{
await _source.Task;//solution 1
_awaitor.WaitOne();//solution 2
}
}

Might not be exactly what you need, but I had a similar case and I solved it with the CountdownEvent class
private CountdownEvent _countdownEvent = new CountdownEvent(1);
process_method() {
//if the count is zero means that we already called finalize
if (_countdownEvent.IsSet)
return;
try
{
//this can throw and exception if we try to add when the countdown has already reached 0.
//this exception happens when one process_method B has passed the _counddownEvent.IsSet check and context switched to
//to another process_method A that was the last one (after finalize waits for 0) and sets the countdown to 0. Which
//triggers finalization and should not allow new process_method, so process_methodB not continuing is good (finalization is
//in progress).
_countdownEvent.AddCount(1);
} catch
{
return;
}
try
{
//your process
}
finally
{
_countdownEvent.Signal();
}
}
And then when you are ready to wait for the count to be zero:
finalize() {
_countdownEvent.Signal();
_countdownEvent.Wait(_finalizationSafetyTimeout, cancellationToken)
}

How to properly use Async, Await and ManualResetEvents to control an infinite while loop

So what i am trying to do here is:
Make the engine loop and work on an object if the queue is not empty.
If the queue is empty i call the manualresetevent to make the thread sleep.
When a item is added and the loop is not active i set the manualresetevent.
To make it faster i pick up atmost 5 items from the list and perform operation on them asynchronously and wait for all of them to finish.
Problem:
The clear methods on the two lists are called as soon as a new call to the AddToUpdateQueueMethod is called.
In my head as i am waiting for Task.WhenAll(tasks), so thread should wait for its completion before moving ahead, hence the clear on the lists should only be called on after Task.WhenAll(tasks) returns.
What am i missing here, or what will be a better way to achieve this.
public async Task ThumbnailUpdaterEngine()
{
int count;
List<Task<bool>> tasks = new List<Task<bool>>();
List<Content> candidateContents = new List<Content>();
while (true)
{
for (int i = 0; i < 5; i++)
{
Content nextContent = GetNextFromInternalQueue();
if (nextContent == null)
break;
else
candidateContents.Add(nextContent);
}
foreach (var candidateContent in candidateContents)
{
foreach (var provider in interactionProviders)
{
if (provider.IsServiceSupported(candidateContent.ServiceType))
{
Task<bool> task = provider.UpdateThumbnail(candidateContent);
tasks.Add(task);
break;
}
}
}
var results = await Task.WhenAll(tasks);
tasks.Clear();
foreach (var candidateContent in candidateContents)
{
if (candidateContent.ThumbnailLink != null && !candidateContent.ThumbnailLink.Equals(candidateContent.FileIconLink, StringComparison.CurrentCultureIgnoreCase))
{
Task<bool> task = DownloadAndUpdateThumbnailCache(candidateContent);
tasks.Add(task);
}
}
await Task.WhenAll(tasks);
//Clean up for next time the loop comes in.
tasks.Clear();
candidateContents.Clear();
lock (syncObject)
{
count = internalQueue.Count;
if (count == 0)
{
isQueueControllerRunning = false;
monitorEvent.Reset();
}
}
await Task.Run(() => monitorEvent.WaitOne());
}
}
private Content GetNextFromInternalQueue()
{
lock (syncObject)
{
Content nextContent = null;
if (internalQueue.Count > 0)
{
nextContent = internalQueue[0];
internalQueue.Remove(nextContent);
}
return nextContent;
}
}
public void AddToUpdateQueue(Content content)
{
lock (syncObject)
{
internalQueue.Add(content);
if (!isQueueControllerRunning)
{
isQueueControllerRunning = true;
monitorEvent.Set();
}
}
}

You should simply use TPL Dataflow. It's an actor framework on top of the TPL with an async support. Use an ActionBlock with an async action and MaxDegreeOfParallelism of 5:
var block = new ActionBlock<Content>(
async content =>
{
var tasks = interactionProviders.
Where(provider => provider.IsServiceSupported(content.ServiceType)).
Select(provider => provider.UpdateThumbnail(content));
await Task.WhenAll(tasks);
if (content.ThumbnailLink != null && !content.ThumbnailLink.Equals(
content.FileIconLink,
StringComparison.CurrentCultureIgnoreCase))
{
await DownloadAndUpdateThumbnailCache(content);
}
}, new ExecutionDataflowBlockOptions { MaxDegreeOfParallelism = 5});
foreach (var content in GetContent())
{
block.Post(content);
}
block.Complete();
await block.Completion

What is the best way to cancel a task that is in a blocking state?

I have tasks running that call a method that reads from RabbitMQ. When there is nothing in the queue, the method simply blocks. So the tasks have a "running" status, but aren't actually doing anything. Is there any way to gracefully end these tasks?
The code that accesses the queue is as follows:
private void FindWork(CancellationToken ct)
{
if (ct.IsCancellationRequested)
return;
bool result = false;
bool process = false;
bool queueResult = false;
Work_Work work = null;
try
{
using (Queue workQueue = new Queue(_workQueue))
{
// Look for work on the work queue
workQueue.Open(Queue.Mode.Consume);
work = workQueue.ConsumeWithBlocking<Work_Work>();
// Do some work with the message ...
return;
The tasks are created as follows:
private void Run()
{
while (!_stop)
{
// Remove and stopped tasks from the pool
List<int> removeThreads = new List<int>();
lock (_tasks)
{
foreach (KeyValuePair<int, Task> task in _tasks)
{
if (task.Value.Status != TaskStatus.Running)
{
task.Value.Wait();
removeThreads.Add(task.Value.Id);
}
}
foreach (int taskID in removeThreads)
_tasks.Remove(taskID);
}
CancellationToken ct = _cts.Token;
TaskFactory factory = new TaskFactory(ct, TaskCreationOptions.LongRunning, TaskContinuationOptions.LongRunning, null);
// Create new tasks if we have room in the pool
while (_tasks.Count < _runningMax)
{
Task task = factory.StartNew(() => FindWork(ct));
lock (_tasks)
_tasks.Add(task.Id, task);
}
// Take a rest so we don't run the CPU to death
Thread.Sleep(1000);
}
}
Currently I have changed my task creation code to look like the following so that I can abort the tasks. I know this is not a good solution, but I don't know what else to do.
while (_tasks.Count < _runningMax)
{
Task task = factory.StartNew(() =>
{
try
{
using (_cts.Token.Register(Thread.CurrentThread.Abort))
{
FindWork(ct);
}
}
catch (ThreadAbortException)
{
return;
}
}, _cts.Token);
_tasks.Add(task.Id, task);
}

Could the following work in your scenario?
Instead of spawning multiple threads and having them waiting in the queue, I would have a single thread in an infinite polling loop and having that one spawn a new thread when a new piede of work comes in. You can add a semaphore to limit the number of threads you create. Check sample code below, I've used a BlockingCollection instead of the RabbitMQ .
public class QueueManager
{
public BlockingCollection<Work> blockingCollection = new BlockingCollection<Work>();
private const int _maxRunningTasks = 3;
static SemaphoreSlim _sem = new SemaphoreSlim(_maxRunningTasks);
public void Queue()
{
blockingCollection.Add(new Work());
}
public void Consume()
{
while (true)
{
Work work = blockingCollection.Take();
_sem.Wait();
Task t = Task.Factory.StartNew(work.DoWork);
}
}
public class Work
{
public void DoWork()
{
Thread.Sleep(5000);
_sem.Release();
Console.WriteLine("Finished work");
}
}
}
and my testing class
class Test
{
static void Main(string[] args)
{
Consumer c = new Consumer();
Task t = Task.Factory.StartNew(c.Consume);
c.Queue();
c.Queue();
c.Queue();
c.Queue();
c.Queue();
Thread.Sleep(1000);
Console.ReadLine();
}
}

To make this work, you'd need to change ConsumeWithBlocking to support cancellation. I'm not familiar with RabbitMQ, but apparently it supports cancellation on the consumer channel.
So, instead of doing Thread.CurrentThread.Abort from Token.Register callback, do the right thing and cancel the operation gracefully via proper RabbitMQ API.
On a side note, the thread you're currently trying to abort is most likely not the one which is blocked by ConsumeWithBlocking.