I'm wondering if there is a way to create either IAsyncEnumerable<T> or IAsyncEnumerator<T> via a Source object, rather like TaskCompletionSource allows one to do for tasks. In particular, TaskCompletionSource can be passed around like any other parameter.
Maybe something like this:
public class AsyncEnumerables {
public Task HandlerTask { get; set; }
public async Task<string> ParentMethod() {
var source = new AsyncEnumerableSource<int>();
IAsyncEnumerable asyncEnumerable = source.GetAsyncEnumerable();
HandlerTask = Task.Run(() => handleAsyncResultsAsTheyHappen(asyncEnumerable));
int n = await someOtherTask();
source.YieldReturn(n);
var r = await ChildMethod(source);
source.Complete(); // this call would cause the HandlerTask to complete.
return r;
}
private async Task<string> ChildMethod(AsyncEnumerableSource<int> source) {
source.YieldReturn(5);
await SomeOtherCall();
source.YieldReturn(10);
return "hello";
}
}
With the above code, the handleAsyncResultsAsTheyHappen task would see whatever values got passed into YieldReturn. So it would see the n from the above code, as well as the 5 and the 10 from ChildMethod.
Here is another implementation of the AsyncEnumerableSource class, that doesn't depend on the Rx library. This one depends instead on the Channel<T>, class, which is natively available in the .NET standard libraries. It has identical behavior to the Rx-based implementation.
The class AsyncEnumerableSource can propagate notifications to multiple subscribers. Each subscriber can enumerate these notifications at its own pace. This is possible because each subscription has its own dedicated Channel<T> as underlying storage. The lifetime of a subscription is practically tied to the lifetime of a single await foreach loop. Breaking early from a loop for any reason (including thrown exceptions), ends immediately the subscription.
In technical terms a new subscription is created the first time that the MoveNextAsync method of an IAsyncEnumerator<T> is invoked. Calling the method GetAsyncEnumerable alone doesn't create a subscription, nor calling the GetAsyncEnumerator method does. The subscription ends when the associated IAsyncEnumerator<T> is disposed.
public class AsyncEnumerableSource<T>
{
private readonly List<Channel<T>> _channels = new();
private bool _completed;
private Exception _exception;
public async IAsyncEnumerable<T> GetAsyncEnumerable(
[EnumeratorCancellation] CancellationToken cancellationToken = default)
{
Channel<T> channel;
lock (_channels)
{
if (_exception != null) throw _exception;
if (_completed) yield break;
channel = Channel.CreateUnbounded<T>(
new() { SingleWriter = true, SingleReader = true });
_channels.Add(channel);
}
try
{
await foreach (var item in channel.Reader.ReadAllAsync()
.WithCancellation(cancellationToken).ConfigureAwait(false))
{
yield return item;
cancellationToken.ThrowIfCancellationRequested();
}
}
finally { lock (_channels) _channels.Remove(channel); }
}
public void YieldReturn(T value)
{
lock (_channels)
{
if (_completed) return;
foreach (var channel in _channels) channel.Writer.TryWrite(value);
}
}
public void Complete()
{
lock (_channels)
{
if (_completed) return;
foreach (var channel in _channels) channel.Writer.TryComplete();
_completed = true;
}
}
public void Fault(Exception error)
{
lock (_channels)
{
if (_completed) return;
foreach (var channel in _channels) channel.Writer.TryComplete(error);
_completed = true;
_exception = error;
}
}
}
The reason for the cancellationToken.ThrowIfCancellationRequested(); is because of this issue: ChannelReader.ReadAllAsync(CancellationToken) not actually cancelled mid-iteration.
Caution: in case you start propagating values with YieldReturn before any consumer has subscribed to the AsyncEnumerableSource, these values are going to be lost. No subscriber is going to observe them. To prevent this scenario you should make sure that all consumers have subscribed before starting the producers. The easiest way to do it is for the consumers to be async methods, with the await foreach being the first await inside the async method:
// Correct, synchronous subscription
async Task Consume()
{
await foreach (var item in source.GetAsyncEnumerable())
{
//...
}
}
Task consumer = Consume();
Avoid the temptation to use the Task.Run method, because in this case the subscription will occur asynchronously on a ThreadPool thread, and not synchronously with the creation of the consumer:
// Wrong, delayed subscription (possibility for unobserved values)
Task consumer = Task.Run(async () =>
{
await foreach (var item in source.GetAsyncEnumerable())
{
//...
}
});
In case that you don't want to do the subscriptions synchronously, it is possible to offload them to the ThreadPool, and await them to be established before starting the producers:
// Correct, awaited subscription
Task consumer = await Task.Factory.StartNew(async () =>
{
HeavySynchronousComputation();
await foreach (var item in source.GetAsyncEnumerable())
{
//...
}
}, default, TaskCreationOptions.DenyChildAttach, TaskScheduler.Default);
The Task.Factory.StartNew(async method creates a nested Task<Task>. The outer task represents the subscription, and the inner task represents the consuming loop.
You're much better off if you can structure your code to take advantage of yield return and await foreach. E.g., this code does almost the same thing:
public async Task Consume()
{
var source = ParentMethod();
HandlerTask = Task.Run(async () => { await foreach (var item in source) { Console.WriteLine(item); } });
}
public async IAsyncEnumerable<int> ParentMethod()
{
await Task.Yield();
yield return 13;
await foreach (var item in ChildMethod())
yield return item;
}
private async IAsyncEnumerable<int> ChildMethod()
{
yield return 5;
await Task.Yield();
yield return 10;
}
However, if you really need an "async enumerable source", you need to first recognize one thing. TaskCompletionSource<T> holds the results, i.e., the T (or exception). It's acting as a container. The result can be set before the task is awaited. It's the same thing with the "async enumerable source" - you'd need it to be able to hold results before any items are taken from it. The "async enumerable source" would need to hold multiple results - in this case, a collection.
So what you're actually asking for is "a collection that can be consumed as an asynchronous enumerable". There are a few possibilities here, but the one I'd recommend is a Channel:
public async Task<string> ParentMethod()
{
var source = Channel.CreateUnbounded<int>();
var sourceWriter = source.Writer;
IAsyncEnumerable<int> asyncEnumerable = source.Reader.ReadAllAsync();
HandlerTask = Task.Run(async () => { await foreach (var item in asyncEnumerable) Console.WriteLine(item); });
await Task.Yield();
await sourceWriter.WriteAsync(13);
var r = await ChildMethod(sourceWriter);
sourceWriter.Complete();
return r;
}
private async Task<string> ChildMethod(ChannelWriter<int> sourceWriter)
{
await sourceWriter.WriteAsync(5);
await Task.Yield();
await sourceWriter.WriteAsync(10);
return "hello";
}
AFAIK the .NET platform has no built-in AsyncEnumerableSource class, but it is easy to implement one by using the System.Reactive and System.Linq.Async libraries. The System.Reactive library contains the class Subject which is a combination of an IObservable and IObserver. This is a convenient class, because you can send notifications to the IObserver interface, and subscribe independently any number of times to the IObservable interface to receive these notifications back. Actually it's not required to subscribe manually, because the System.Linq.Async library contains the handy extension method ToAsyncEnumerable, that converts an IObservable to IAsyncEnumerable automatically.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Reactive.Subjects;
public class AsyncEnumerableSource<T>
{
private readonly Subject<T> _subject = new Subject<T>();
public IAsyncEnumerable<T> GetAsyncEnumerable() => _subject.ToAsyncEnumerable();
public void YieldReturn(T value) => _subject.OnNext(value);
public void Complete() => _subject.OnCompleted();
public void Fault(Exception ex) => _subject.OnError(ex);
}
This implementation will send to the subscribers only the notifications that occurred after their subscription. If you want to ensure that late joiners will get the early messages, you could replace the Subject with a ReplaySubject. This one buffers the notifications it receives, so it comes with memory usage considerations: it accepts an int bufferSize argument in its constructor.
Note: The above implementation is thread-safe, although the Subject<T> class is not synchronized, and in general calling OnNext from multiple threads in parallel breaks the Rx contract. That's because the ToAsyncEnumerable operator does not depend on the Rx contract for its correctness, and synchronizes the incoming notifications. It's not a particularly efficient implementation though. Channel<T>-based implementations are significantly more efficient under heavy load.
Related
Is it possible to use Async when using ForEach? Below is the code I am trying:
using (DataContext db = new DataLayer.DataContext())
{
db.Groups.ToList().ForEach(i => async {
await GetAdminsFromGroup(i.Gid);
});
}
I am getting the error:
The name 'Async' does not exist in the current context
The method the using statement is enclosed in is set to async.
List<T>.ForEach doesn't play particularly well with async (neither does LINQ-to-objects, for the same reasons).
In this case, I recommend projecting each element into an asynchronous operation, and you can then (asynchronously) wait for them all to complete.
using (DataContext db = new DataLayer.DataContext())
{
var tasks = db.Groups.ToList().Select(i => GetAdminsFromGroupAsync(i.Gid));
var results = await Task.WhenAll(tasks);
}
The benefits of this approach over giving an async delegate to ForEach are:
Error handling is more proper. Exceptions from async void cannot be caught with catch; this approach will propagate exceptions at the await Task.WhenAll line, allowing natural exception handling.
You know that the tasks are complete at the end of this method, since it does an await Task.WhenAll. If you use async void, you cannot easily tell when the operations have completed.
This approach has a natural syntax for retrieving the results. GetAdminsFromGroupAsync sounds like it's an operation that produces a result (the admins), and such code is more natural if such operations can return their results rather than setting a value as a side effect.
This little extension method should give you exception-safe async iteration:
public static async Task ForEachAsync<T>(this List<T> list, Func<T, Task> func)
{
foreach (var value in list)
{
await func(value);
}
}
Since we're changing the return type of the lambda from void to Task, exceptions will propagate up correctly. This will allow you to write something like this in practice:
await db.Groups.ToList().ForEachAsync(async i => {
await GetAdminsFromGroup(i.Gid);
});
Starting with C# 8.0, you can create and consume streams asynchronously.
private async void button1_Click(object sender, EventArgs e)
{
IAsyncEnumerable<int> enumerable = GenerateSequence();
await foreach (var i in enumerable)
{
Debug.WriteLine(i);
}
}
public static async IAsyncEnumerable<int> GenerateSequence()
{
for (int i = 0; i < 20; i++)
{
await Task.Delay(100);
yield return i;
}
}
More
The simple answer is to use the foreach keyword instead of the ForEach() method of List().
using (DataContext db = new DataLayer.DataContext())
{
foreach(var i in db.Groups)
{
await GetAdminsFromGroup(i.Gid);
}
}
Here is an actual working version of the above async foreach variants with sequential processing:
public static async Task ForEachAsync<T>(this List<T> enumerable, Action<T> action)
{
foreach (var item in enumerable)
await Task.Run(() => { action(item); }).ConfigureAwait(false);
}
Here is the implementation:
public async void SequentialAsync()
{
var list = new List<Action>();
Action action1 = () => {
//do stuff 1
};
Action action2 = () => {
//do stuff 2
};
list.Add(action1);
list.Add(action2);
await list.ForEachAsync();
}
What's the key difference? .ConfigureAwait(false); which keeps the context of main thread while async sequential processing of each task.
This is not an old question, but .Net 6 introduced Parallel.ForeachAsync:
var collectionToIterate = db.Groups.ToList();
await Parallel.ForEachAsync(collectionToIterate, async (i, token) =>
{
await GetAdminsFromGroup(i);
});
ForeachAsync also accepts a ParallelOptions object, but usually you don't want to mess with the MaxDegreeOfParallelism property:
ParallelOptions parallelOptions = new ParallelOptions { MaxDegreeOfParallelism = 4 };
var collectionToIterate = db.Groups.ToList();
await Parallel.ForEachAsync(collectionToIterate, parallelOptions , async (i, token) =>
{
await GetAdminsFromGroup(i);
});
From Microsoft Docs: https://learn.microsoft.com/en-us/dotnet/api/system.threading.tasks.paralleloptions.maxdegreeofparallelism?view=net-6.0
By default, For and ForEach will utilize however many threads the underlying scheduler provides, so changing MaxDegreeOfParallelism from the default only limits how many concurrent tasks will be used.
Generally, you do not need to modify this setting....
Add this extension method
public static class ForEachAsyncExtension
{
public static Task ForEachAsync<T>(this IEnumerable<T> source, int dop, Func<T, Task> body)
{
return Task.WhenAll(from partition in Partitioner.Create(source).GetPartitions(dop)
select Task.Run(async delegate
{
using (partition)
while (partition.MoveNext())
await body(partition.Current).ConfigureAwait(false);
}));
}
}
And then use like so:
Task.Run(async () =>
{
var s3 = new AmazonS3Client(Config.Instance.Aws.Credentials, Config.Instance.Aws.RegionEndpoint);
var buckets = await s3.ListBucketsAsync();
foreach (var s3Bucket in buckets.Buckets)
{
if (s3Bucket.BucketName.StartsWith("mybucket-"))
{
log.Information("Bucket => {BucketName}", s3Bucket.BucketName);
ListObjectsResponse objects;
try
{
objects = await s3.ListObjectsAsync(s3Bucket.BucketName);
}
catch
{
log.Error("Error getting objects. Bucket => {BucketName}", s3Bucket.BucketName);
continue;
}
// ForEachAsync (4 is how many tasks you want to run in parallel)
await objects.S3Objects.ForEachAsync(4, async s3Object =>
{
try
{
log.Information("Bucket => {BucketName} => {Key}", s3Bucket.BucketName, s3Object.Key);
await s3.DeleteObjectAsync(s3Bucket.BucketName, s3Object.Key);
}
catch
{
log.Error("Error deleting bucket {BucketName} object {Key}", s3Bucket.BucketName, s3Object.Key);
}
});
try
{
await s3.DeleteBucketAsync(s3Bucket.BucketName);
}
catch
{
log.Error("Error deleting bucket {BucketName}", s3Bucket.BucketName);
}
}
}
}).Wait();
If you are using EntityFramework.Core there is an extension method ForEachAsync.
The example usage looks like this:
using Microsoft.EntityFrameworkCore;
using System.Threading.Tasks;
public class Example
{
private readonly DbContext _dbContext;
public Example(DbContext dbContext)
{
_dbContext = dbContext;
}
public async void LogicMethod()
{
await _dbContext.Set<dbTable>().ForEachAsync(async x =>
{
//logic
await AsyncTask(x);
});
}
public async Task<bool> AsyncTask(object x)
{
//other logic
return await Task.FromResult<bool>(true);
}
}
I would like to add that there is a Parallel class with ForEach function built in that can be used for this purpose.
The problem was that the async keyword needs to appear before the lambda, not before the body:
db.Groups.ToList().ForEach(async (i) => {
await GetAdminsFromGroup(i.Gid);
});
This is method I created to handle async scenarios with ForEach.
If one of tasks fails then other tasks will continue their execution.
You have ability to add function that will be executed on every exception.
Exceptions are being collected as aggregateException at the end and are available for you.
Can handle CancellationToken
public static class ParallelExecutor
{
/// <summary>
/// Executes asynchronously given function on all elements of given enumerable with task count restriction.
/// Executor will continue starting new tasks even if one of the tasks throws. If at least one of the tasks throwed exception then <see cref="AggregateException"/> is throwed at the end of the method run.
/// </summary>
/// <typeparam name="T">Type of elements in enumerable</typeparam>
/// <param name="maxTaskCount">The maximum task count.</param>
/// <param name="enumerable">The enumerable.</param>
/// <param name="asyncFunc">asynchronous function that will be executed on every element of the enumerable. MUST be thread safe.</param>
/// <param name="onException">Acton that will be executed on every exception that would be thrown by asyncFunc. CAN be thread unsafe.</param>
/// <param name="cancellationToken">The cancellation token.</param>
public static async Task ForEachAsync<T>(int maxTaskCount, IEnumerable<T> enumerable, Func<T, Task> asyncFunc, Action<Exception> onException = null, CancellationToken cancellationToken = default)
{
using var semaphore = new SemaphoreSlim(initialCount: maxTaskCount, maxCount: maxTaskCount);
// This `lockObject` is used only in `catch { }` block.
object lockObject = new object();
var exceptions = new List<Exception>();
var tasks = new Task[enumerable.Count()];
int i = 0;
try
{
foreach (var t in enumerable)
{
await semaphore.WaitAsync(cancellationToken);
tasks[i++] = Task.Run(
async () =>
{
try
{
await asyncFunc(t);
}
catch (Exception e)
{
if (onException != null)
{
lock (lockObject)
{
onException.Invoke(e);
}
}
// This exception will be swallowed here but it will be collected at the end of ForEachAsync method in order to generate AggregateException.
throw;
}
finally
{
semaphore.Release();
}
}, cancellationToken);
if (cancellationToken.IsCancellationRequested)
{
break;
}
}
}
catch (OperationCanceledException e)
{
exceptions.Add(e);
}
foreach (var t in tasks)
{
if (cancellationToken.IsCancellationRequested)
{
break;
}
// Exception handling in this case is actually pretty fast.
// https://gist.github.com/shoter/d943500eda37c7d99461ce3dace42141
try
{
await t;
}
#pragma warning disable CA1031 // Do not catch general exception types - we want to throw that exception later as aggregate exception. Nothing wrong here.
catch (Exception e)
#pragma warning restore CA1031 // Do not catch general exception types
{
exceptions.Add(e);
}
}
if (exceptions.Any())
{
throw new AggregateException(exceptions);
}
}
}
I'm new to System.Threading.Channels. I have the following consumer code:
await foreach (var thing in this.Reader.ReadAllAsync(cancellationToken)
.ConfigureAwait(false))
{
await this.HandleThingAsync(thing, cancellationToken).ConfigureAwait(false);
}
That seems to work fine when consuming things produced by a single producer like this:
var things = await this.GetThingsAsync(cancellationToken).ConfigureAwait(false);
await foreach (var thing in things.WithCancellation(cancellationToken)
.ConfigureAwait(false))
{
await this.Writer.WriteAsync(thing, cancellationToken).ConfigureAwait(false);
}
this.Writer.Complete();
But when I try to add in a second producer of the same general form, as soon as one of the two producers is done (and calls this.Writer.Complete()), anything that the other producer still needs to add will be rejected because the channel is already closed. This is a problem because I want the reader to read everything, not merely everything up until the point that any one producer has nothing more to produce.
How does one deal with this situation? Is there some built-in or otherwise "standard" way? For example, perhaps a "condenser" channel which exposes multiple Channel.Writer objects (one for each "real" producer), and a single Channel.Reader (for the single "real" consumer)?
I don't think that there is a way that you could call "standard". A Channel<T> is a tool that can be used in many different ways, much like a Task or a SemaphoreSlim. In your case, you could propagate the completion of all producers by using a counter like this:
int producersCount = X;
//...
await foreach (var thing in things)
await channel.Writer.WriteAsync(thing);
if (Interlocked.Decrement(ref producersCount) == 0) channel.Writer.Complete();
Alternatively, if each producer is a Task, you could attach a continuation to all these tasks combined like this:
var producers = new List<Task>();
//...
_ = Task.WhenAll(producers).ContinueWith(_ => channel.Writer.Complete(),
default, TaskContinuationOptions.ExecuteSynchronously, TaskScheduler.Default);
The discard (_) above has been used in order to communicate that the ContinueWith continuation has been launched in a fire-and-forget fashion. If you dislike throwing unobserved tasks in the wind like me, you can handle the completion of the producers in an async void method like this:
var producers = new List<Task>();
//...
HandleProducersCompletion();
//...
async void HandleProducersCompletion()
{
try { await Task.WhenAll(producers); }
finally { channel.Writer.Complete(); }
}
This way an exception thrown by the channel.Writer.Complete(); invocation will be unhandled and will crash the process. Which is arguably a good thing, considering the alternative which is a process that has been deadlocked for no apparent reason.
I wound up making this class, based on my "channel condenser" idea that I mentioned in my original question. It may or may not be horrible and/or bug-ridden, but at least so far it seems to do the job in a way that seems fairly natural and unobtrusive to me:
using Nito.AsyncEx;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Channels;
using System.Threading.Tasks;
namespace Rwv37.System.Threading.Channels
{
public class ChannelCondenser<T>
{
private bool IsGoing { get; set; }
private AsyncLock IsGoingLock { get; init; }
private ConcurrentBag<Channel<T>> IncomingChannel { get; init; }
private Channel<T> OutgoingChannel { get; init; }
public ChannelCondenser()
{
this.IsGoingLock = new AsyncLock();
this.IncomingChannel = new();
this.OutgoingChannel = Channel.CreateUnbounded<T>();
}
public async Task GoAsync(CancellationToken cancellationToken = default)
{
using (await this.IsGoingLock.LockAsync(cancellationToken).ConfigureAwait(false))
{
if (this.IsGoing)
{
throw new System.InvalidOperationException("Cannot go - already going!");
}
this.IsGoing = true;
}
List<Task> tasks = new();
foreach (var incomingChannel in this.IncomingChannel)
{
tasks.Add(this.HandleIncomingChannelAsync(incomingChannel, cancellationToken));
}
await Task.WhenAll(tasks).ConfigureAwait(false);
this.OutgoingChannel.Writer.Complete();
}
public ChannelWriter<T> AddIncomingChannel()
{
using (this.IsGoingLock.Lock())
{
if (this.IsGoing)
{
throw new System.InvalidOperationException("New incoming channels cannot be added while going!");
}
}
Channel<T> incomingChannel = Channel.CreateUnbounded<T>();
this.IncomingChannel.Add(incomingChannel);
return incomingChannel.Writer;
}
public ChannelReader<T> GetOutgoingChannel()
{
return this.OutgoingChannel.Reader;
}
private async Task HandleIncomingChannelAsync(Channel<T> incomingChannel, CancellationToken cancellationToken)
{
await foreach (var item in incomingChannel.Reader.ReadAllAsync(cancellationToken).ConfigureAwait(false))
{
await this.OutgoingChannel.Writer.WriteAsync(item, cancellationToken).ConfigureAwait(false);
}
}
}
}
Usage within the consumer(s) and producer(s) is completely unchanged from that shown in my original question.
The only thing I had to change outside of them is how the classes using them are constructed. Consumer construction was changed from...
private Channel<Thing> WantedThingsChannel { get; init; }
(...)
this.WantedThingsChannel = Channel.CreateUnbounded<Thing>();
this.WantedThingsHandler = new(this.WantedThingsChannel.Reader);
... to...
private ChannelCondenser<Thing> WantedThingsCondenser { get; init; }
(...)
this.WantedThingsCondenser = new();
this.WantedThingsHandler = new(this.WantedThingsCondenser.GetOutgoingChannel());
Similarly, the producer's construction was changed from...
this.WantedThingsRetriever = new(this.WantedThingsChannel.Writer);
... to...
this.WantedThingsRetriever = new(this.WantedThingsCondenser.AddIncomingChannel());
Oh, no, wait, I lied. One other change outside of them: My program's main Task.WhenAll was changed so that it additionally waits on the ChannelCondenser. So, from...
List<Task> tasks = new()
{
this.WantedThingsHandler.GoAsync(cancellationToken),
this.WantedThingsRetriever.GoAsync(cancellationToken),
};
... to...
List<Task> tasks = new()
{
this.WantedThingsCondenser.GoAsync(cancellationToken),
this.WantedThingsHandler.GoAsync(cancellationToken),
this.WantedThingsRetriever.GoAsync(cancellationToken),
};
I'm trying to understand async actions and I'm a bit confused.
Actions are just glorified Delegates. Given the Actions
Action act = null;
act += () => { Console.WriteLine("Sync"); };
act += async () => { await File.AppendAllLinesAsync("C:/Test.Txt",
new[] { "Async File Operation" });
};
How can we invoke this async seeing as one of the delegates is async and the other is not. I've seen some extension methods in other SO answers simplified for the example would look like so:
public static void InvokeAsync(this Action action, AsyncCallback ar, object userObject = null)
{
var listeners = action.GetInvocationList();
foreach (var t in listeners)
{
var handler = (Action)t;
handler.BeginInvoke(ar, userObject);
}
}
I'm concerned if this even works because it looks like it invokes your callback for each listener which doesn't make sense.
I've only been using async with the more friendly version async/await so I do not understand this syntax as much. (I'm assuming the callback would be everything after the await and the userObject is equivalent to the dreadful SyncronizationContext that causes deadlocks if when calling sync without ConfigureAwait(false), but that is just a guess)
This is syntax inconvenient so I would perfer to use async await syntax, since async/await is called using duck-typing. I've read a blog about using async with delegates which for the example
public static class DelegateExtensions
{
public static TaskAwaiter GetAwaiter(this Action action)
{
Task task = new Task(action);
task.Start();
return task.GetAwaiter();
}
}
This too concerns me for a few reason, this looks much like an anti pattern.
Isn't this just creating a task which will run my action synchronous on a seperate thread? I also don't see this run through the invocation list.
Are either of these methods proper for invoking run delegates asynchronously?
Is there a way I can invoke an async delegate with the await syntax while still fully leveraging async?
What is the proper way to invoke async delegates with multiple functions in the invocation list?
I think Eric Lippert's comment have clarified the situation more than I could ever.
Overall, if you need to act on the return type of a method, you shouldn't use multicast delegates. If you still have to, at least use a Func<Task> signature, then you can iterate on each individual delegate using GetInvocationList, as explained here.
But would it be really impossible to work your way out of a multicast delegate with async void method?
It turns out that you can be notified of beginning and end of async void methods by using a custom synchronization context and overriding the OperationStarted and OperationCompleted methods. We can also override the Post method to set the synchronization context of child operations, to capture subsequent async void calls.
Piecing it together, you could come with something like:
class Program
{
static async Task Main(string[] args)
{
Action act = null;
act += () => { Console.WriteLine("Sync"); };
act += async () =>
{
Callback();
await Task.Delay(1000);
Console.WriteLine("Async");
};
await AwaitAction(act);
Console.WriteLine("Done");
Console.ReadLine();
}
static async void Callback()
{
await Task.Delay(2000);
Console.WriteLine("Async2");
}
static Task AwaitAction(Action action)
{
var delegates = action.GetInvocationList();
var oldSynchronizationContext = SynchronizationContext.Current;
var asyncVoidSynchronizationContext = new AsyncVoidSynchronizationContext();
try
{
SynchronizationContext.SetSynchronizationContext(asyncVoidSynchronizationContext);
var tasks = new Task[delegates.Length];
for (int i = 0; i < delegates.Length; i++)
{
((Action)delegates[i]).Invoke();
tasks[i] = asyncVoidSynchronizationContext.GetTaskForLastOperation();
}
return Task.WhenAll(tasks);
}
finally
{
SynchronizationContext.SetSynchronizationContext(oldSynchronizationContext);
}
}
}
public class AsyncVoidSynchronizationContext : SynchronizationContext
{
private TaskCompletionSource<object> _tcs;
private Task _latestTask;
private int _operationCount;
public Task GetTaskForLastOperation()
{
if (_latestTask != null)
{
var task = _latestTask;
_latestTask = null;
return task;
}
return Task.CompletedTask;
}
public override void Post(SendOrPostCallback d, object state)
{
Task.Run(() =>
{
SynchronizationContext.SetSynchronizationContext(this);
d(state);
});
}
public override void OperationStarted()
{
if (Interlocked.Increment(ref _operationCount) == 1)
{
// First operation
_tcs = new TaskCompletionSource<object>();
_latestTask = _tcs.Task;
}
base.OperationStarted();
}
public override void OperationCompleted()
{
if (Interlocked.Decrement(ref _operationCount) == 0)
{
// Last operation
_tcs.TrySetResult(null);
}
base.OperationCompleted();
}
}
The output would be:
Sync
Async
Async2
Done
Of course, this code is provided just for recreational purpose. There's plenty of limitations, such as the fact the fact that it wouldn't work as-is if you're already using a synchronization context (such as the WPF one). I'm also certain that it has a few subtle bugs and concurrency issues here and there.
I'm working on a series of methods that execute many different database calls using entity framework. Many of these methods can run asynchronously as I really don't care about their output, and don't rely on them.
However, when I try implementing certain methods, I get a warning from the compiler saying: "Because this call is not awaited, the current method continues to run before the call is completed"
But to me, this seems like that's my desired behavior, as I don't care what the methods do.
Here's an example of the methods
public async Task SetupAccessControl(int objectTypeId, int objectId, int? organizationId)
{
using (var context = new SupportContext(CustomerId))
{
... // omitted for brevity
if (objectTypeId == (int) ObjectType.User)
{
AddToUserRoleBridge("Everyone", objectId);//Warning on this line
AddToUserRoleBridge("Default", objectId); //Warning on this line
}
... // omitted for brevity
}
}
public async Task AddToUserRoleBridge(string role, int userId)
{
using (var context = new SupportContext(CustomerId))
{
var defaultRole = context.Roles.FirstOrDefault(n => n.Name == role);
if (defaultRole != null)
{
var urb = new UserRoleBridge
{
RoleId = defaultRole.Id,
UserId = userId
};
context.UserRoleBridges.Add(urb);
await context.SaveChangesAsync();
}
}
}
edit
Essentially, when I run the main function, I want a series of method calls to all fire off somewhat simultaneously and handle everything in their own threads so that I don't have to worry about it. Here is a pseudo-code example.
public async void RunAllAsync() {
taskA(*some value*);
taskA(*some value*);
taskB(*some value*);
taskB(*some value*);
await AllTasksCompleted
}
public async Task taskA(int item){
//do something with item
}
public async Task taskB(int item) {
subTaskB(*some value*)
subTaskB(*some value*)
}
public async Task subTaskB(int item) {
// do something
}
In the above example, when #RunAllAsync is called, every function call it makes (and the function calls they make) are fired off simultaneously. When all of these calls are completed, whatever method called #RunAllAsync would continue to execute.
If you're not using await, the async keyword doesn't really do anything useful and you can leave it off. You can await a method returning a Task regardless of whether it's marked as async or not.
public Task DoSomethingAsync()
{
Task someTaskJustLikeANormalReturnValue = Task.Delay(1000);
return someTaskJustLikeANormalReturnValue;
}
// later...
public async Task SomeOtherFunction()
{
// You can await DoSomethingAsync just like any async method, because
// you're really awaiting the Task which got returned.
await DoSomethingAsync();
}
In your case I would probably collect the tasks and await them all together:
public async Task SetupAccessControl(int objectTypeId, int objectId, int? organizationId)
{
var tasks = new List<Task>();
using (var context = new SupportContext(CustomerId))
{
... // omitted for brevity
if (objectTypeId == (int) ObjectType.User)
{
tasks.Add(AddToUserRoleBridge("Everyone", objectId));
tasks.Add(AddToUserRoleBridge("Default", objectId));
}
... // omitted for brevity
}
await Task.WhenAll(tasks.ToArray());
}
This allows you to pass the decision up to the caller of whether to await on the subtasks or not. This will also allow the any caller to unwrap any exceptions if they happen in AddToUserRoleBridge.
You can do it two ways:
Change the signature of AddToUserRoleBridge to return void will
remove the warning (but nothing can await it then).
Store the result of the call. var ignoreme=AddToUserRoleBridge(...) will remove the warning as well.
I am trying to block RequestHandler.ParseAll() with await ConsumerTask;, but when i set a breakpoint there, i ALWAYS get the "Done..." output first... and then Parse2() fails with a NullReferenceException. (thats my guess: "the GC starts cleaning up because _handler got out of scope")
Anyway, I can't figure out why that happens.
class MainClass
{
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
/* fill mUrls here with values */
await Task.Run(() => _handler.ParseSpecific(mUrls));
Console.WriteLine("Done...");
}
}
static class Parser
{
public static async Task<IEnumerable<string>> QueryWebPage(string url) { /*Query the url*/ }
public static async Task Parse1(Query query)
{
Parallel.ForEach(/*Process data here*/);
}
public static async Task Parse2(Query query)
{
foreach(string line in query.WebPage)
/* Here i get a NullReference exception because query.WebPage == null */
}
}
sealed class RequestHandler
{
private BlockingCollection<Query> Queue;
private Task ConsumerTask = Task.Run(() => /* call consume() for each elem in the queue*/);
private async void Consume(Query obj)
{
await (obj.BoolField ? Parser.Parse1(obj) : Parser.Parse2(obj));
}
public async void ParseSpecific(string[] urls)
{
foreach(string v in urls)
Queue.Add(new Query(await QueryWebPage(v), BoolField: false));
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
private async Task ParseAll(bool onlySome)
{
ReInit();
Parallel.ForEach(mCollection, v => Queue.Add(new Query(url, BoolField:false)));
Queue.CompleteAdding();
await ConsumerTask;
/* Process stuff further */
}
}
struct Query
{
public readonly string[] WebPage;
public readonly bool BoolField;
public Query(uint e, IEnumerable<string> page, bool b) : this()
{
Webpage = page.ToArray();
BoolField = b;
}
}
CodesInChaos has spotted the problem in comments. It stems from having async methods returning void, which you should almost never do - it means you've got no way to track them.
Instead, if your async methods don't have any actual value to return, you should just make them return Task.
What's happening is that ParseSpecific is only running synchronously until the first await QueryWebPage(v) that doesn't complete immediately. It's then returning... so the task started here:
await Task.Run(() => _handler.ParseSpecific(mUrls));
... completes immediately, and "Done" gets printed.
Once you've made all your async methods return Task, you can await them. You also won't need Task.Run at all. So you'd have:
public async void DoWork()
{
RequestHandler _handler = new RequestHandler();
string[] mUrls;
await _handler.ParseSpecific(mUrls);
Console.WriteLine("Done...");
}
...
public async TaskParseSpecific(string[] urls)
{
foreach(string v in urls)
{
// Refactored for readability, although I'm not sure it really
// makes sense now that it's clearer! Are you sure this is what
// you want?
var page = await QueryWebPage(v);
Queue.Add(new Query(page, false);
}
Queue.CompleteAdding();
await ConsumerTask;
await ParseAll(true);
}
Your Reinit method also needs changing, as currently the ConsumerTask will basically complete almost immediately, as Consume will return immediately as it's another async method returning void.
To be honest, what you've got looks very complex, without a proper understanding of async/await. I would read up more on async/await and then probably start from scratch. I strongly suspect you can make this much, much simpler. You might also want to read up on TPL Dataflow which is designed to make producer/consumer scenarios simpler.