It's a little hard for me to understand the actual behavior in this scenario. What is actually happening to not execute the task when expected but later when SemaphoreSlim has been disposed? It throws the following exception-
System.ObjectDisposedException {"The semaphore has been disposed."}
I have a class library like -
public class ParallelProcessor
{
private Action[] actions;
private int maxConcurrency;
public ParallelProcessor(Action[] actionList, int maxConcurrency)
{
this.actions = actionList;
this.maxConcurrency = maxConcurrency;
}
public void RunAllActions()
{
if (Utility.IsNullOrEmpty<Action>(actions))
throw new Exception("No Action Found!");
using (SemaphoreSlim concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
{
foreach (Action action in actions)
{
Task.Factory.StartNew(() =>
{
concurrencySemaphore.Wait();
try
{
action();
}
finally
{
concurrencySemaphore.Release();
}
});
}
}
}
}
And I'm using it like-
class Program
{
static void Main(string[] args)
{
int maxConcurrency = 3;
Action[] actions = new Action[] { () => Console.WriteLine(1), () => Console.WriteLine(2), () => Console.WriteLine(3) }; //Array.Empty<Action>();
ParallelProcessor processor = new ParallelProcessor(actions, maxConcurrency);
processor.RunAllActions();
Console.ReadLine();
}
}
Could anybody please shed some light on this? Thanks in advance.
The problem is your using statement. This is how things are happening:
Create the semaphore
Start tasks running in the background
Dispose of the semaphore
Tasks try to use the semaphore... but can't, because it's disposed
Options:
Just remove the using statement (so you don't dispose of the semaphore, but that's unlikely to be a problem unless you're using this really heavily)
Change your method to block (inside the using statement) until all the tasks have completed, e.g. by using Parallel.ForEach instead of calling Task.Factory.StartNew directly
Change your code to dispose of the semaphore in a task which will only execute after all the other tasks have completed
Your semaphore is disposed at the end of the using block, but used by the still running Task created inside it.
I would recommend moving the semaphore up to the class level:
public class ParallelProcessor
{
private Action[] actions;
private SemaphoreSlim concurrencySemaphore;
public ParallelProcessor(Action[] actionList, int maxConcurrency)
{
this.actions = actionList;
concurrencySemaphore = new SemaphoreSlim(maxConcurrency);
}
public void RunAllActions()
{
if (Utility.IsNullOrEmpty<Action>(actions))
throw new Exception("No Action Found!");
foreach (Action action in actions)
{
Task.Factory.StartNew(() =>
{
concurrencySemaphore.Wait();
try
{
action();
}
finally
{
concurrencySemaphore.Release();
}
});
}
}
}
or an alternative approach, where RunAllActions will block until all are done:
public class ParallelProcessor
{
private Action[] actions;
private int maxConcurrency;
public ParallelProcessor(Action[] actionList, int maxConcurrency)
{
this.actions = actionList;
this.maxConcurrency = maxConcurrency;
}
public void RunAllActions()
{
if (Utility.IsNullOrEmpty<Action>(actions))
throw new Exception("No Action Found!");
using (var concurrencySemaphore = new SemaphoreSlim(maxConcurrency))
{
Task.WaitAll(actions.Select(a => Task.Run(() =>
{
concurrencySemaphore.Wait();
try { a(); }
finally { concurrencySemaphore.Release(); }
})).ToArray());
}
}
}
I believe the problem is dispose the concurrencySemaphore which is already in the using statement.
The main use of Using is, it will automatically add try and finally and in finally it will dispose the object that is under using.
https://www.codeproject.com/Articles/6564/Understanding-the-using-statement-in-C
The solution for your case is either remove the using or remove the finally statement
Related
I have a problem which is very similar to this one. However, in my case I create a ReactiveCommand that calls an async method on execution. The ThrownExceptions observable doesn't seem to pipe any exceptions no matter where they are thrown (directly in the method or in the task started by it).
I have a written a minimum example to demonstrate that. I know that ThrownExceptions doesn't catch everything but I don't know for which cases it is not designed to work or how to handle these exceptions correctly.
using ReactiveUI;
using System;
using System.Reactive;
using System.Threading.Tasks;
namespace ConsoleApp3
{
class Program
{
static void Main()
{
var test = new TestClass();
while (Console.ReadKey().Key == ConsoleKey.Enter)
{
test.Command.Execute().Subscribe();
}
Console.ReadKey();
}
}
public class TestClass
{
public TestClass()
{
Command = ReactiveCommand.Create(() => RunCommand());
Command.ThrownExceptions.Subscribe(ex => HandleException(ex));
}
public ReactiveCommand<Unit, Task> Command { get; private set; }
private async Task RunCommand()
{
//throw new Exception("will not be handled");
//await Task.Run(() => throw new Exception("will also not be handled"));
}
private void HandleException(Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
To answer my own question: commands that call async methods have to be created using the CreateFromTask method instead of the Create method.
In addition, the exceptions are not caught if the command is executed by subscribing to it's Execute observable. The Execute method of the ICommand interface has to be used instead (commands should be exposed to the public using this interface anyway).
I have changed my demo project as below:
class Program
{
static void Main()
{
var test = new TestClass();
while (Console.ReadKey().Key == ConsoleKey.Enter)
{
(test.Command as ICommand).Execute(null);
}
Console.ReadKey();
}
}
public class TestClass
{
public TestClass()
{
Command = ReactiveCommand.CreateFromTask(RunCommand);
Command.ThrownExceptions.Subscribe(ex => HandleException(ex));
}
public ReactiveCommand<Unit, Unit> Command { get; private set; }
private async Task RunCommand()
{
//throw new Exception("will be handled");
await Task.Run(() =>
{
throw new Exception("will also be handled");
});
}
private void HandleException(Exception ex)
{
Console.WriteLine(ex.Message);
}
}
I have following problem: Service is being run on the server. It has timer that ticks every 5 seconds. It is being modeled to produce items for a consumer. Items are being consumed in multi thread.
I've added logging but I cannot find out when and where any problem occurs and this just blocks. No exception or error. My goal is to keep getting requests to be processed from db and consume them. Timer is being a producer.
public class CustomProducerConsumer<T> : IDisposable
{
private readonly BlockingCollection<T> blockingCollection;
private readonly Action<T> consumeItem;
private readonly Task[] workers;
public CustomProducerConsumer(Action<T> consumeItem,
int degreeOfParallelism,
int capacity = 1024)
{
this.consumeItem = consumeItem;
this.blockingCollection = new BlockingCollection<T>(capacity);
this.workers = Enumerable.Range(1, degreeOfParallelism)
.Select(_ => Task.Factory.StartNew(Worker,
TaskCreationOptions.LongRunning))
.ToArray();
}
public void Dispose()
{
// Unblock all workers even if the client
// didn't call CompleteProcessing
if (!this.blockingCollection.IsAddingCompleted)
{
this.blockingCollection.CompleteAdding();
}
Task.WaitAll(this.workers);
this.blockingCollection.Dispose();
}
public void Process(T item)
{
this.blockingCollection.TryAdd(item);
}
private void Worker()
{
foreach (var item in this.blockingCollection.GetConsumingEnumerable())
{
this.consumeItem(item);
}
}
}
Here is my code from service:
private readonly BlockingCollection<StitchingRequestProcessingModel> requestsToBeProcessed =
new BlockingCollection<StitchingRequestProcessingModel>(10);
private readonly BlockingCollection<Dictionary<int, StitchingRequest[]>> pendingRequests =
new BlockingCollection<Dictionary<int, StitchingRequest[]>>(10);
private readonly Timer timer;
public Service()
{
InitializeComponent();
this.produceConsumer =
new CustomProducerConsumer<StitchingRequestModel>(this.ProcessItems,
Environment.ProcessorCount);
this.timer = new Timer(o =>
{
this.TimerElapsed();
this.timer.Change(TimeSpan.FromSeconds(5), Timeout.InfiniteTimeSpan);
}, null, TimeSpan.Zero, Timeout.InfiniteTimeSpan);
this.ConsumeRequests();
}
public void TimerElapsed()
{
try
{
//this just adds into the list an item to ping the db for pending requests when available
this.requestsToBeProcessed.Add(new StitchingRequestProcessingModel());
this.pendingRequests.TryTake(out Dictionary<int, Request[]> requests);
if (requests == null)
{
return;
}
foreach (KeyValuePair<int, Request[]> request in requests)
{
this.produceConsumer.Process(new StitchingRequestModel(request));
}
}
catch (Exception exception)
{
this.errorLogger.Error(exception.Message);
}
}
private void ConsumeRequests()
{
Task.Factory.StartNew(() =>
{
while (!this.requestsToBeProcessed.IsCompleted)
{
if (this.tokenSource.Token.IsCancellationRequested)
{
break;
}
StitchingRequestProcessingModel data = null;
try
{
data = this.requestsToBeProcessed.Take();
}
catch (InvalidOperationException)
{
}
if (data == null)
{
continue;
}
try
{
// this just executes sql query to get those request from db
var requests = this.requestService.GetPendingRequests();
this.pendingRequests.Add(requests);
}
catch (Exception exception)
{
this.errorLogger.Error(exception.Message, "Failed to get pending requests");
}
}
},
this.tokenSource.Token,
TaskCreationOptions.LongRunning, TaskScheduler.Current);
}
private void ProcessItems(StitchingRequestModel model)
{
foreach (StitchingRequest request in model.Requests)
{
this.requestsToBeProcessed.Add(new StitchingRequestProcessingModel(request);
}
}
Main reason why I placed consuming items into blocking collection is Nhibernate. It is giving me issues while doing multithreading. No ideas what else to try, nor why this approach is not working. I do not want to call CompleteAdding on blocking collection since I need requests to be added and just processed in first available thread.
Timer on each elapsed event will try to create an a pending request that will be added into blocking collection and processed on first available turn. Service runs for 2~3h and just stops. ProcessItems method can be long running. CPU is 8 core.
UPDATE
Added cancelation for consumer task.
Problem was solved with not working with entity object between consumer and producer. Created dto for info needed.
I need to execute a kind of LongRunning task after a delay.
Each Task can be cancelled. I prefer TPL with cancellationToken.
Since my task is long running and before starting a task it has to be placed in dictionary I have to use new Task(). But I've faced different behavior - when task is created using new Task() after Cancel() it throws TaskCanceledException whereas a task created with Task.Run doesn't throw an exception.
Generally I need to recognize the difference and not get TaskCanceledException.
It's my code:
internal sealed class Worker : IDisposable
{
private readonly IDictionary<Guid, (Task task, CancellationTokenSource cts)> _tasks =
new Dictionary<Guid, (Task task, CancellationTokenSource cts)>();
public void ExecuteAfter(Action action, TimeSpan waitBeforeExecute, out Guid cancellationId)
{
var cts = new CancellationTokenSource();
var task = new Task(async () =>
{
await Task.Delay(waitBeforeExecute, cts.Token);
action();
}, cts.Token, TaskCreationOptions.LongRunning);
cancellationId = Guid.NewGuid();
_tasks.Add(cancellationId, (task, cts));
task.Start(TaskScheduler.Default);
}
public void ExecuteAfter2(Action action, TimeSpan waitBeforeExecute, out Guid cancellationId)
{
var cts = new CancellationTokenSource();
cancellationId = Guid.NewGuid();
_tasks.Add(cancellationId, (Task.Run(async () =>
{
await Task.Delay(waitBeforeExecute, cts.Token);
action();
}, cts.Token), cts));
}
public void Abort(Guid cancellationId)
{
if (_tasks.TryGetValue(cancellationId, out var value))
{
value.cts.Cancel();
//value.task.Wait();
_tasks.Remove(cancellationId);
Dispose(value.cts);
Dispose(value.task);
}
}
public void Dispose()
{
if (_tasks.Count > 0)
{
foreach (var t in _tasks)
{
Dispose(t.Value.cts);
Dispose(t.Value.task);
}
_tasks.Clear();
}
}
private static void Dispose(IDisposable obj)
{
if (obj == null)
{
return;
}
try
{
obj.Dispose();
}
catch (Exception ex)
{
//Log.Exception(ex);
}
}
}
internal class Program
{
private static void Main(string[] args)
{
Action act = () => Console.WriteLine("......");
Console.WriteLine("Started");
using (var w = new Worker())
{
w.ExecuteAfter(act, TimeSpan.FromMilliseconds(10000), out var id);
//w.ExecuteAfter2(act, TimeSpan.FromMilliseconds(10000), out var id);
Thread.Sleep(3000);
w.Abort(id);
}
Console.WriteLine("Enter to exit");
Console.ReadKey();
}
}
UPD:
This approach also works without exception
public void ExecuteAfter3(Action action, TimeSpan waitBeforeExecute, out Guid cancellationId)
{
var cts = new CancellationTokenSource();
cancellationId = Guid.NewGuid();
_tasks.Add(cancellationId, (Task.Factory.StartNew(async () =>
{
await Task.Delay(waitBeforeExecute, cts.Token);
action();
}, cts.Token, TaskCreationOptions.LongRunning, TaskScheduler.Default), cts)); ;
}
The reason of the inconsistent behavior is fundamentally incorrect usage of an async delegate in the first case. The Task constructors just don't receive Func<Task> and your asynchronous delegate is always interpreted as async void not async Task in case of using with constructor. If an exception is raised in an async Task method it's caught and placed into Task object which isn't true for an async void method, in that case exception just bubbles up out of the method to a synchronization context and goes under category of unhandled exceptions (you can familiarize with details in this Stephen Cleary article). So what happens in case of using constructor: a task which is supposed to initiate asynchronous flow is created and started. Once it reaches point when Task.Delay(...) returns a promise, the task completes and it has no more relationship to anything which happens in Task.Delay continuation (you can easily check in debugger by setting a breakpoint to value.cts.Cancel() that the task object in the _tasks dictionary has status RanToCompletetion while however the task delegate essentially is still running). When a cancellation is requested the exception is raised inside the Task.Delay method and without existence of any promise object is being promoted to app domain.
In case of Task.Run the situation is different because there are overloads of this method which are able to accept Func<Task> or Func<Task<T>> and unwrap the tasks internally in order to return underlying promise instead of wrapped task which ensures proper task object inside the _tasks dictionary and proper error handling.
The third scenario despite the fact that it doesn't throw an exception it is partially correct. Unlike Task.Run, Task.Factory.StartNew doesn't unwrap underlying task to return promise, so task stored in the _tasks is just wrapper task, like in the case with constructor (again you can check its state with debugger). It however is able to understand Func<Task> parameters, so asynchronous delegate has async Task signature which allows at least to handle and store exception in the underlying task. In order to get this underlying task with Task.Factory.StartNew you need to unwrap the task by yourself with Unwrap() extension method.
The Task.Factory.StartNew isn't considered as a beast practice of creating tasks because of certain dangers related to its application (see there). It however can be used with some caveats if you need to apply specific options like LongRunning which cannot be directly applied with Task.Run.
I don't know why I got down votes here but it's inspired me to update my answer.
UPDATED
My full approach:
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp4
{
internal class Program
{
private static void Main(string[] args)
{
using (var delayedWorker = new DelayedWorker())
{
delayedWorker.ProcessWithDelay(() => { Console.WriteLine("100"); }, TimeSpan.FromSeconds(5), out var cancellationId_1);
delayedWorker.ProcessWithDelay(() => { Console.WriteLine("200"); }, TimeSpan.FromSeconds(10), out var cancellationId_2);
delayedWorker.ProcessWithDelay(() => { Console.WriteLine("300"); }, TimeSpan.FromSeconds(15), out var cancellationId_3);
Cancel_3(delayedWorker, cancellationId_3);
Console.ReadKey();
}
}
private static void Cancel_3(DelayedWorker delayedWorker, Guid cancellationId_3)
{
Task.Run(() => { delayedWorker.Abort(cancellationId_3); }).Wait();
}
internal sealed class DelayedWorker : IDisposable
{
private readonly object _locker = new object();
private readonly object _disposeLocker = new object();
private readonly IDictionary<Guid, (Task task, CancellationTokenSource cts)> _tasks = new Dictionary<Guid, (Task task, CancellationTokenSource cts)>();
private bool _disposing;
public void ProcessWithDelay(Action action, TimeSpan waitBeforeExecute, out Guid cancellationId)
{
Console.WriteLine("Creating delayed action...");
CancellationTokenSource tempCts = null;
CancellationTokenSource cts = null;
try
{
var id = cancellationId = Guid.NewGuid();
tempCts = new CancellationTokenSource();
cts = tempCts;
var task = new Task(() => { Process(action, waitBeforeExecute, cts); }, TaskCreationOptions.LongRunning);
_tasks.Add(cancellationId, (task, cts));
tempCts = null;
task.ContinueWith(t =>
{
lock (_disposeLocker)
{
if (!_disposing)
{
TryRemove(id);
}
}
}, TaskContinuationOptions.ExecuteSynchronously);
Console.WriteLine($"Created(cancellationId: {cancellationId})");
task.Start(TaskScheduler.Default);
}
finally
{
if (tempCts != null)
{
tempCts.Dispose();
}
}
}
private void Process(Action action, TimeSpan waitBeforeExecute, CancellationTokenSource cts)
{
Console.WriteLine("Starting delayed action...");
cts.Token.WaitHandle.WaitOne(waitBeforeExecute);
if (cts.Token.IsCancellationRequested)
{
return;
}
lock (_locker)
{
Console.WriteLine("Performing action...");
action();
}
}
public bool Abort(Guid cancellationId)
{
Console.WriteLine($"Aborting(cancellationId: {cancellationId})...");
lock (_locker)
{
if (_tasks.TryGetValue(cancellationId, out var value))
{
if (value.task.IsCompleted)
{
Console.WriteLine("too late");
return false;
}
value.cts.Cancel();
value.task.Wait();
Console.WriteLine("Aborted");
return true;
}
Console.WriteLine("Either too late or wrong cancellation id");
return true;
}
}
private void TryRemove(Guid id)
{
if (_tasks.TryGetValue(id, out var value))
{
Remove(id, value.task, value.cts);
}
}
private void Remove(Guid id, Task task, CancellationTokenSource cts)
{
_tasks.Remove(id);
Dispose(cts);
Dispose(task);
}
public void Dispose()
{
lock (_disposeLocker)
{
_disposing = true;
}
if (_tasks.Count > 0)
{
foreach (var t in _tasks)
{
t.Value.cts.Cancel();
t.Value.task.Wait();
Dispose(t.Value.cts);
Dispose(t.Value.task);
}
_tasks.Clear();
}
}
private static void Dispose(IDisposable obj)
{
if (obj == null)
{
return;
}
try
{
obj.Dispose();
}
catch (Exception ex)
{
//log ex
}
}
}
}
}
I have a situation where I must call an async method synchronously, and it is done so as follows:
obj.asyncMethod().Wait(myCancelToken)
If the cancellation token is switched the disposable's within the task will not get disposed despite being activated via a using statement.
The below program illustrates the problem:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace LeakTest {
class Program {
static void Main(string[] args) {
try {
var timeout = new CancellationTokenSource(TimeSpan.FromMilliseconds(100));
LongRunningTask().Wait(timeout.Token);
} catch (OperationCanceledException error) {
// handling timeout is logically okay, but expect nothing to be leaked
}
Console.WriteLine("Leaked Instances = {0}", DisposableResource.Instances);
Console.ReadKey();
}
static async Task LongRunningTask() {
using (var resource = new DisposableResource()) {
await Task.Run( () => Thread.Sleep(1000));
}
}
public class DisposableResource : IDisposable {
public static int Instances = 0;
public DisposableResource() {
Instances++;
}
public void Dispose() {
Instances--;
}
}
}
}
It seems Wait method just kills the task thread on cancellation instead of triggering an exception within that thread and letting it terminate naturally. Question is why?
You've cancelled the task returned by Wait(timeout.Token) not the one returned from LongRunningTask, if you want to cancel that one pass the token to Task.Run and also use await Task.Delay instead of Thread.Sleep and pass the token there as well.
static void Main(string[] args)
{
try
{
var timeout = new CancellationTokenSource(TimeSpan.FromMilliseconds(100));
LongRunningTask(timeout.Token).Wait();
}
catch (AggregateException error)
{
// handling timeout is logically okay, but expect nothing to be leaked
}
Console.WriteLine("Leaked Instances = {0}", DisposableResource.Instances);
Console.ReadLine();
}
static async Task LongRunningTask(CancellationToken token)
{
using (var resource = new DisposableResource())
{
await Task.Run(async () => await Task.Delay(1000, token), token);
}
}
public class DisposableResource : IDisposable
{
public static int Instances = 0;
public DisposableResource()
{
Instances++;
}
public void Dispose()
{
Instances--;
}
}
Note that the using statment will still dispose of the resource once the long running operation finishes. Run this example:
static void Main(string[] args)
{
try {
var timeout = new CancellationTokenSource(TimeSpan.FromMilliseconds(100));
LongRunningTask().Wait(timeout.Token);
} catch (OperationCanceledException error) {
// handling timeout is logically okay, but expect nothing to be leaked
}
Console.WriteLine("Leaked Instances = {0}", DisposableResource.Instances);
Console.ReadKey();
}
static async Task LongRunningTask()
{
using (var resource = new DisposableResource())
{
await Task.Run(() => Thread.Sleep(1000));
}
}
public class DisposableResource : IDisposable
{
public static int Instances = 0;
public DisposableResource()
{
Instances++;
}
public void Dispose()
{
Instances--;
Console.WriteLine("Disposed resource. Leaked Instances = {0}", Instances);
}
}
Output
Leaked Instances = 1
Disposed resource. Leaked Instances = 0
It seems Wait method just kills the task thread on cancellation instead of triggering an exception within that thread
You are incorrect, on when you cancel the only thing that happens is you stop waiting for Wait(myCancelToken) to complete, the task is still running in the background.
In order to cancel the background task you must pass the cancelation token into all of the methods down the chain. If you want the innermost layer (the long running one) to stop early that code must call token.ThrowIfCancellationRequested() throughout its code.
I have the following (simplified) code:
MailMessage message = GetMailMessage();
ProcessEmail(() => SendEmail(message));
private void ProcessEmail(Action Method) {
try {
ThreadPool.QueueUserWorkItem(new WaitCallback(?));
} catch (Exception ex) {
}
}
private static void SendEmail(object message) {
// send email
}
My question is, within ProcessEmail, can I pass the Action parameter named Method to the ThreadPool.QueueUserWorkItem method?
Any help much appreciated.
No, but you can do this:
ThreadPool.QueueUserWorkItem(new WaitCallback(state=>{Method();}));
or more succinctly:
ThreadPool.QueueUserWorkItem(state=>{Method();});
Essentially, you are supplying a new anonymous function callback that adapts the call to your Method() callback.
I prefer this technique:
ThreadPool.QueueUserWorkItem(nameOfAnonymousMethod =>
{
Method(...);
});
I think you will need to use autoresetevent or manualresetevent.
Simple Code:
public class Test
{
private AutoResetEvent _eventWaitThread = new AutoResetEvent(false);
private void Job()
{
Action act = () =>
{
try
{
// do work...
}
finally
{
_eventWaitThread.Set();
}
};
ThreadPool.QueueUserWorkItem(x => act());
_eventWaitThread.WaitOne(10 * 1000 * 60);
}
}