I am trying to block RequestHandler.ParseAll() with await ConsumerTask;, but when i set a breakpoint there, i ALWAYS get the "Done..." output first... and then Parse2() fails with a NullReferenceException. (thats my guess: "the GC starts cleaning up because _handler got out of scope")
Anyway, I can't figure out why that happens.
class MainClass
{
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
/* fill mUrls here with values */
await Task.Run(() => _handler.ParseSpecific(mUrls));
Console.WriteLine("Done...");
}
}
static class Parser
{
public static async Task<IEnumerable<string>> QueryWebPage(string url) { /*Query the url*/ }
public static async Task Parse1(Query query)
{
Parallel.ForEach(/*Process data here*/);
}
public static async Task Parse2(Query query)
{
foreach(string line in query.WebPage)
/* Here i get a NullReference exception because query.WebPage == null */
}
}
sealed class RequestHandler
{
private BlockingCollection<Query> Queue;
private Task ConsumerTask = Task.Run(() => /* call consume() for each elem in the queue*/);
private async void Consume(Query obj)
{
await (obj.BoolField ? Parser.Parse1(obj) : Parser.Parse2(obj));
}
public async void ParseSpecific(string[] urls)
{
foreach(string v in urls)
Queue.Add(new Query(await QueryWebPage(v), BoolField: false));
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
private async Task ParseAll(bool onlySome)
{
ReInit();
Parallel.ForEach(mCollection, v => Queue.Add(new Query(url, BoolField:false)));
Queue.CompleteAdding();
await ConsumerTask;
/* Process stuff further */
}
}
struct Query
{
public readonly string[] WebPage;
public readonly bool BoolField;
public Query(uint e, IEnumerable<string> page, bool b) : this()
{
Webpage = page.ToArray();
BoolField = b;
}
}
CodesInChaos has spotted the problem in comments. It stems from having async methods returning void, which you should almost never do - it means you've got no way to track them.
Instead, if your async methods don't have any actual value to return, you should just make them return Task.
What's happening is that ParseSpecific is only running synchronously until the first await QueryWebPage(v) that doesn't complete immediately. It's then returning... so the task started here:
await Task.Run(() => _handler.ParseSpecific(mUrls));
... completes immediately, and "Done" gets printed.
Once you've made all your async methods return Task, you can await them. You also won't need Task.Run at all. So you'd have:
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
await _handler.ParseSpecific(mUrls);
Console.WriteLine("Done...");
}
...
public async TaskParseSpecific(string[] urls)
{
foreach(string v in urls)
{
// Refactored for readability, although I'm not sure it really
// makes sense now that it's clearer! Are you sure this is what
// you want?
var page = await QueryWebPage(v);
Queue.Add(new Query(page, false);
}
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
Your Reinit method also needs changing, as currently the ConsumerTask will basically complete almost immediately, as Consume will return immediately as it's another async method returning void.
To be honest, what you've got looks very complex, without a proper understanding of async/await. I would read up more on async/await and then probably start from scratch. I strongly suspect you can make this much, much simpler. You might also want to read up on TPL Dataflow which is designed to make producer/consumer scenarios simpler.
Related
I have a problem with async/await in C#, i need it to get some object called Trades, after i get it, it needs to SAVE it. Problem is, with async/await, it is doing the SAVE first, and then go and get my trade objects. How do i ensure i get the objects first, and then does the saving.... here is my code...
private async void OnRefresh()
{
try
{
var trades = await ExchangeServiceInstance.GetTrades("");
mmTrades = new ObservableCollection<EEtrade>(trades);
tradeListView.ItemsSource = mmTrades;
}
catch { }
}
public async void OnSignalReceived()
{
// THIS NEEDS TO FINISH FIRST, BUT IT DOESN'T
await tradeListView.Dispatcher.InvokeAsync((Action)async delegate
{
if (ExchangeServiceInstance.SelectedTabIndex == CURRENT_TAB_INDEX_ITEM)
{
await Task.Delay(MMConfig.DELAYMILLISEC);
OnRefresh();
}
});
// SOMEHOW THIS GETS CALLED FIRST BEFORE THE ABOVE GETS TO FINISH!
await OnSaveTrades();
}
public async Task<int> OnSaveTrades()
{
foreach (var trade in mmTrades)
{
await ExchangeServiceInstance.OnInsertDoneTrade(trade);
}
return mmTrades.Count;
}
Any ideas guys? Thanks!
The problem is your OnRefresh method. Because the return type is void the method is not awaited [Check out this answer]. In addition you dont even try to await for the method inside your delegate
Changing the method to the following:
private async Task OnRefresh()
{
try
{
var trades = await ExchangeServiceInstance.GetTrades("");
mmTrades = new ObservableCollection<EEtrade>(trades);
tradeListView.ItemsSource = mmTrades;
}
catch { }
}
And await this method inside your delegate, should solve your problem:
public async void OnSignalReceived()
{
// THIS NEEDS TO FINISH FIRST, BUT IT DOESN'T
await tradeListView.Dispatcher.InvokeAsync((Action)async delegate
{
if (ExchangeServiceInstance.SelectedTabIndex == CURRENT_TAB_INDEX_ITEM)
{
await Task.Delay(MMConfig.DELAYMILLISEC);
await OnRefresh();
}
});
// SOMEHOW THIS GETS CALLED FIRST BEFORE THE ABOVE GETS TO FINISH!
await OnSaveTrades();
}
The use of (Action)async is basically the same as async void, and async void is almost always a mistake. Specifically, the consumer cannot know the outcome (unless it faults synchronously). The dispatcher here isn't really thinking of async.
If we assume that you must use the dispatcher here, perhaps a workaround might be to use something like a SemaphoreSlim (or maybe a TaskCompletionSource<something>) that you signal at the end of your async work (even in the exception case), and then await that; untested, but:
var tcs = new TaskCompletionSource<bool>();
await tradeListView.Dispatcher.InvokeAsync((Action)async delegate
{
try {
if (ExchangeServiceInstance.SelectedTabIndex == CURRENT_TAB_INDEX_ITEM)
{
await Task.Delay(MMConfig.DELAYMILLISEC);
OnRefresh();
}
tcs.TrySetResult(true);
} catch (Exception ex) {
tcs.TrySetException(ex);
}
});
await tcs.Task; // ensure the async work is complete
await OnSaveTrades();
First of all, you are using the async void pattern a lot. This is really bad practice for a number of reasons. You should stop doing that.
The problem here is that OnRefresh is again an async void method that can't be awaited but should be:
private async Task OnRefresh()
{
try
{
var trades = await ExchangeServiceInstance.GetTrades("");
mmTrades = new ObservableCollection<EEtrade>(trades);
tradeListView.ItemsSource = mmTrades;
}
catch { }
}
In your OnSignalReceived method change the call to OnRefresh(); to await OnRefresh();
I have a Task which starts a win process, which generates file if its not created yet and returns it. The problem is that the action is called more than once. To be more precisely its src attribute of a <track> element.
I have ConcurrentDictionary<Guid, Task<string>>
which keeps track of for which Id a process is currently running
public async Task<string> GenerateVTTFile(Guid Id)
{
if (_currentGenerators.TryGetValue(id, out Task<string> task))
{
return await task; // problem is here?
}
var t = Run(); // Task
_currentGenerators.TryAdd(id, t);
return await t;
}
In the action method of the controller
var path = await _svc.GetSomePath();
if (string.IsNullOrEmpty(path))
{
var path = await svc.GenerateVTTFile(id);
return PhysicalFile(path, "text/vtt");
}
return PhysicalFile(path, "text/vtt");
Run() method is just starting Process and waits it.
process.WaitForExit();
What I want to achieve is to return the result of the same task for the same Id. It seems that if the Id already exists in the dictionary and I await it starts another process (calls Run method again).
Is there a way to achieve that?
You can make the method atomic to protect the "dangerzone":
private SemaphoreSlim _sem = new SemaphoreSlim(1);
public Task<string> GenerateVTTFile(Guid Id)
{
_sem.Wait();
try
{
if (_currentGenerators.TryGetValue(Id, out Task<string> task))
{
return task;
}
var t = Run(); // Task
_currentGenerators.TryAdd(Id, t); // While Thread 1 is here,
// Thread 2 can already be past the check above ...
// unless we make the method atomic like here.
return t;
}
finally
{
_sem.Release();
}
}
Drawback here is, that also calls with different ids have to wait. So that makes for a bottleneck. Of course, you could make an effort but hey: the dotnet guys did it for you:
Preferably, you can use GetOrAdd to do the same with only ConcurrentDictionary's methods:
public Task<string> GenerateVTTFile(Guid Id)
{
// EDIT: This overload vv is actually NOT atomic!
// DO NOT USE:
//return _currentGenerators.GetOrAdd(Id, () => Run());
// Instead:
return _currentGenerators.GetOrAdd(Id,
_ => new Lazy<Task<string>>(() => Run(id))).Value;
// Fix "stolen" from Theodore Zoulias' Answer. Link to his answer below.
// If you find this helped you, please upvote HIS answer.
}
Yes, it's really a "one-liner".
Please see this answer: https://stackoverflow.com/a/61372518/982149 from which I took the fix for my flawed answer.
As pointed out already by João Reis, using simply the GetOrAdd method is not enough to ensure that a Task will be created only once per key. From the documentation:
If you call GetOrAdd simultaneously on different threads, valueFactory may be called multiple times, but only one key/value pair will be added to the dictionary.
The quick and lazy way to deal with this problem is to use the Lazy class. Instead of storing Task objects in the dictionary, you could store Lazy<Task> wrappers. This way even if a wrapper is created multiple times per key, all extraneous wrappers will be discarded without their Value property requested, and so without duplicate tasks created.
private ConcurrentDictionary<Guid, <Lazy<Task<string>>> _currentGenerators;
public Task<string> GenerateVTTFileAsync(Guid id)
{
return _currentGenerators.GetOrAdd(id,
_ => new Lazy<Task<string>>(() => Run(id))).Value;
}
In order to have multiple concurrent calls of that method but only one for each id, you need to use ConcurrentDictionary.GetOrAdd with SemaphoreSlim.
GetOrAdd is not enough because the factory parameter might be executed more than once, see "Remarks" here https://learn.microsoft.com/en-us/dotnet/api/system.collections.concurrent.concurrentdictionary-2.getoradd?view=netframework-4.8
Here is an example:
private ConcurrentDictionary<Guid, Generator> _currentGenerators =
new ConcurrentDictionary<Guid, Generator>();
public async Task<string> GenerateVTTFile(Guid id)
{
var generator = _currentGenerators.GetOrAdd(id, _ => new Generator());
return await generator.RunGenerator().ConfigureAwait(false);
}
public class Generator
{
private int _started = 0;
private Task<string> _task;
private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1);
public async Task<string> RunGenerator()
{
if (!IsInitialized())
{
await Initialize().ConfigureAwait(false);
}
return await Interlocked.CompareExchange(ref _task, null, null).ConfigureAwait(false);
}
private async Task Initialize()
{
await _semaphore.WaitAsync().ConfigureAwait(false);
try
{
// check again after acquiring the lock
if (IsInitialized())
{
return;
}
var task = Run();
_ = Interlocked.Exchange(ref _task, task);
Interlocked.Exchange(ref _started, 1);
}
finally
{
_semaphore.Release();
}
}
private bool IsInitialized()
{
return Interlocked.CompareExchange(ref _started, 0, 0) == 1;
}
private async Task<string> Run()
{
// your implementation here
}
}
I've made a queue, which contains tasks to do. After creating some tasks manually with new Task() in Returns method, my whole application hangs - await current;. The body of the task is not even triggered.
ConfigureAwait(false) didn't help.
The first task in the queue, which is not created by me, but other framework is executing successfully and returning a value. Mine - doesn't. I've tried add Task.CompletedTask and then it has worked. I don't understand why I can't even reach the body of the task containing _output assignment.
IDE debugger code screenshot
---UPDATE---
The code works when I use code below. With await it doesn't. Any ideas?
current.Start();
current.Wait();
Original code
private readonly Queue<Task> _pipe;
public IPipeBuilder<TOutput> Returns(Func<IEnumerable<IExecutionResult>, TOutput> outputBuilder)
{
_pipe.Enqueue(new Task(() => // this task causes a problem and breakpoint isn't hit inside
{
_output = outputBuilder(_results);
}));
return this;
}
public async Task<TOutput> Execute()
{
Task current;
while (_pipe.TryDequeue(out current))
{
if (current.IsCommandExecution())
{
IExecutionResult result = await (Task<IExecutionResult>)current; // this awaits successfully
_results.Add(result);
}
else
{
await current; // hangs here
}
}
return await Task.FromResult(_output);
}
Usage
[HttpGet("eventflow/pipe/issue/add/{title}")]
public async Task<IActionResult> PipeAction(string title)
=> Ok(
await Pipe<IExecutionResult>()
.Validate(title)
.Handle<AddIssueCommand>(IssueId.New, title)
.Returns(results => results.First())
.Execute());
You should never use the Task constructor. This goes double on ASP.NET, since constructed tasks are always Delegate Tasks, which interfere with the ASP.NET usage of the thread pool. The actual reason that the await hangs is because manually-created tasks need to be started.
If you have synchronous work that you need to wrap into a Task to work alongside asynchronous tasks, then you should use Task.CompletedTask and Task.FromException:
private static Task SynchronousWork(Func<IEnumerable<IExecutionResult>, TOutput> outputBuilder)
{
try { _output = outputBuilder(_results); return Task.CompletedTask; }
catch (Exception ex) { return Task.FromException(ex); }
}
public IPipeBuilder<TOutput> Returns(Func<IEnumerable<IExecutionResult>, TOutput> outputBuilder)
{
_pipe.Enqueue(SynchronousWork(outputBuilder));
return this;
}
However, note that this executes outputBuilder immediately, which may not be desirable due to its side effects on _results and _output. If you want a delayed execution queue, then the type in the queue needs to be changed from Task to Func<Task>. Then you can add to it as such:
public IPipeBuilder<TOutput> Returns(Func<IEnumerable<IExecutionResult>, TOutput> outputBuilder)
{
_pipe.Enqueue(() =>
{
try { _output = outputBuilder(_results); return Task.CompletedTask; }
catch (Exception ex) { return Task.FromException(ex); }
});
return this;
}
and you would consume it by calling each delegate one at a time and inspecting the task it returns:
public async Task<TOutput> Execute()
{
while (_pipe.TryDequeue(out var currentFunc))
{
var currentTask = currentFunc();
if (currentTask.IsCommandExecution())
{
IExecutionResult result = await (Task<IExecutionResult>)currentTask;
_results.Add(result);
}
else
{
await currentTask;
}
}
return _output;
}
Okay, thank you. I've ended up with such class and Queue<Func<Task>> like you said.
public sealed class SyncTaskWrapper
{
private Func<Task> _action;
public SyncTaskWrapper(Action action)
=> _action = CreateFunc(action);
private static Func<Task> CreateFunc(Action action)
=> () =>
{
try
{
action();
return Task.CompletedTask;
}
catch (Exception exception)
{
return Task.FromException(exception);
}
};
public static implicit operator Func<Task>(SyncTaskWrapper #this)
=> #this._action;
}
with usage
_pipe.Enqueue(new SyncTaskWrapper(() =>
_output = outputBuilder(_results)));
I'm trying to understand async actions and I'm a bit confused.
Actions are just glorified Delegates. Given the Actions
Action act = null;
act += () => { Console.WriteLine("Sync"); };
act += async () => { await File.AppendAllLinesAsync("C:/Test.Txt",
new[] { "Async File Operation" });
};
How can we invoke this async seeing as one of the delegates is async and the other is not. I've seen some extension methods in other SO answers simplified for the example would look like so:
public static void InvokeAsync(this Action action, AsyncCallback ar, object userObject = null)
{
var listeners = action.GetInvocationList();
foreach (var t in listeners)
{
var handler = (Action)t;
handler.BeginInvoke(ar, userObject);
}
}
I'm concerned if this even works because it looks like it invokes your callback for each listener which doesn't make sense.
I've only been using async with the more friendly version async/await so I do not understand this syntax as much. (I'm assuming the callback would be everything after the await and the userObject is equivalent to the dreadful SyncronizationContext that causes deadlocks if when calling sync without ConfigureAwait(false), but that is just a guess)
This is syntax inconvenient so I would perfer to use async await syntax, since async/await is called using duck-typing. I've read a blog about using async with delegates which for the example
public static class DelegateExtensions
{
public static TaskAwaiter GetAwaiter(this Action action)
{
Task task = new Task(action);
task.Start();
return task.GetAwaiter();
}
}
This too concerns me for a few reason, this looks much like an anti pattern.
Isn't this just creating a task which will run my action synchronous on a seperate thread? I also don't see this run through the invocation list.
Are either of these methods proper for invoking run delegates asynchronously?
Is there a way I can invoke an async delegate with the await syntax while still fully leveraging async?
What is the proper way to invoke async delegates with multiple functions in the invocation list?
I think Eric Lippert's comment have clarified the situation more than I could ever.
Overall, if you need to act on the return type of a method, you shouldn't use multicast delegates. If you still have to, at least use a Func<Task> signature, then you can iterate on each individual delegate using GetInvocationList, as explained here.
But would it be really impossible to work your way out of a multicast delegate with async void method?
It turns out that you can be notified of beginning and end of async void methods by using a custom synchronization context and overriding the OperationStarted and OperationCompleted methods. We can also override the Post method to set the synchronization context of child operations, to capture subsequent async void calls.
Piecing it together, you could come with something like:
class Program
{
static async Task Main(string[] args)
{
Action act = null;
act += () => { Console.WriteLine("Sync"); };
act += async () =>
{
Callback();
await Task.Delay(1000);
Console.WriteLine("Async");
};
await AwaitAction(act);
Console.WriteLine("Done");
Console.ReadLine();
}
static async void Callback()
{
await Task.Delay(2000);
Console.WriteLine("Async2");
}
static Task AwaitAction(Action action)
{
var delegates = action.GetInvocationList();
var oldSynchronizationContext = SynchronizationContext.Current;
var asyncVoidSynchronizationContext = new AsyncVoidSynchronizationContext();
try
{
SynchronizationContext.SetSynchronizationContext(asyncVoidSynchronizationContext);
var tasks = new Task[delegates.Length];
for (int i = 0; i < delegates.Length; i++)
{
((Action)delegates[i]).Invoke();
tasks[i] = asyncVoidSynchronizationContext.GetTaskForLastOperation();
}
return Task.WhenAll(tasks);
}
finally
{
SynchronizationContext.SetSynchronizationContext(oldSynchronizationContext);
}
}
}
public class AsyncVoidSynchronizationContext : SynchronizationContext
{
private TaskCompletionSource<object> _tcs;
private Task _latestTask;
private int _operationCount;
public Task GetTaskForLastOperation()
{
if (_latestTask != null)
{
var task = _latestTask;
_latestTask = null;
return task;
}
return Task.CompletedTask;
}
public override void Post(SendOrPostCallback d, object state)
{
Task.Run(() =>
{
SynchronizationContext.SetSynchronizationContext(this);
d(state);
});
}
public override void OperationStarted()
{
if (Interlocked.Increment(ref _operationCount) == 1)
{
// First operation
_tcs = new TaskCompletionSource<object>();
_latestTask = _tcs.Task;
}
base.OperationStarted();
}
public override void OperationCompleted()
{
if (Interlocked.Decrement(ref _operationCount) == 0)
{
// Last operation
_tcs.TrySetResult(null);
}
base.OperationCompleted();
}
}
The output would be:
Sync
Async
Async2
Done
Of course, this code is provided just for recreational purpose. There's plenty of limitations, such as the fact the fact that it wouldn't work as-is if you're already using a synchronization context (such as the WPF one). I'm also certain that it has a few subtle bugs and concurrency issues here and there.
I am trying to generate different objects and insert each object into it's respective List using await and async using the code below:
static List<ClassA> classAList = new List<ClassA>();
static List<ClassB> classBList = new List<ClassB>();
public async void GenerateUsers()
{
await GenerateClassA();
await GenerateClassB();
}
private static Task GenerateClassA()
{
while(true)
{
Thread.Sleep(3000);
classAList .Add(new ClassA());
Console.WriteLine(classAList.Count);
}
}
private static Task GenerateClassB()
{
while (true)
{
Thread.Sleep(6000);
classBList .Add(new ClassB());
Console.WriteLine(classBList.Count)
}
}
When I call GenerateUsers like
UserContainer uc = new UserContainer(); //Class GenerateUsers is defined
uc.GenerateUsers();
Console.WriteLine("Generating.....");
The program does not exit GenerateClassA and Generating..... is not printed to the screen.
How can I generate different objects in an infinite loop using await and async for each infinite loop.
You are using await GenerateClassA();, but GenerateClassA doesn't do anything continuation related - it simply run to completion promising to return a Task. Except: it can't run to completion, because while(true). So yes, this won't work.
Something doesn't become async just beacause you add async. That enables genuinely async operations to be coordinated: nothing more, nothing less.
The methods you are calling are not async, and so they execute synchronously. Indeed, only the first method, GenerateClassA(), is ever even called. That method never returns, so you never get as far as calling the second. Probably you wanted something more like this:
public void GenerateUsers()
{
GenerateClassA();
GenerateClassB();
}
private static async void GenerateClassA()
{
while(true)
{
await Task.Delay(3000);
classAList.Add(new ClassA());
Console.WriteLine(classAList.Count);
}
}
private static async void GenerateClassB()
{
while (true)
{
await Task.Delay(6000);
classBList.Add(new ClassB());
Console.WriteLine(classBList.Count)
}
}
Note that the void return types prevent you from observing the tasks, such as exceptions that might occur. It's not advised. But it's consistent with the code you originally posted.