Related
I have a server which communicates with 50 or more devices over TCP LAN. There is a Task.Run for each socket reading message loop.
I buffer each message reach into a blocking queue, where each blocking queue has a Task.Run using a BlockingCollection.Take().
So something like (semi-pseudocode):
Socket Reading Task
Task.Run(() =>
{
while (notCancelled)
{
element = ReadXml();
switch (element)
{
case messageheader:
MessageBlockingQueue.Add(deserialze<messageType>());
...
}
}
});
Message Buffer Task
Task.Run(() =>
{
while (notCancelled)
{
Process(MessageQueue.Take());
}
});
So that would make 50+ reading tasks and 50+ tasks blocking on their own buffers.
I did it this way to avoid blocking the reading loop and allow the program to distribute processing time on messages more fairly, or so I believe.
Is this an inefficient way to handle it? what would be a better way?
You may be interested in the "channels" work, in particular: System.Threading.Channels. The aim of this is to provider asynchronous producer/consumer queues, covering both single and multiple producer and consumer scenarios, upper limits, etc. By using an asynchronous API, you aren't tying up lots of threads just waiting for something to do.
Your read loop would become:
while (notCancelled) {
var next = await queue.Reader.ReadAsync(optionalCancellationToken);
Process(next);
}
and the producer:
switch (element)
{
case messageheader:
queue.Writer.TryWrite(deserialze<messageType>());
...
}
so: minimal changes
Alternatively - or in combination - you could look into things like "pipelines" (https://www.nuget.org/packages/System.IO.Pipelines/) - since you're dealing with TCP data, this would be an ideal fit, and is something I've looked at for the custom web-socket server here on Stack Overflow (which deals with huge numbers of connections). Since the API is async throughout, it does a good job of balancing work - and the pipelines API is engineered with typical TCP scenarios in mind, for example partially consuming incoming data streams as you detect frame boundaries. I've written about this usage a lot, with code examples mostly here. Note that "pipelines" doesn't include a direct TCP layer, but the "kestrel" server includes one, or the third-party library https://www.nuget.org/packages/Pipelines.Sockets.Unofficial/ does (disclosure: I wrote it).
I actually do something similar in another project. What I learned or would do differently are the following:
First of all, better to use dedicated threads for the reading/writing loop (with new Thread(ParameterizedThreadStart)) because Task.Run uses a pool thread and as you use it in a (nearly) endless loop the thread is practically never returned to the pool.
var thread = new Thread(ReaderLoop) { Name = nameof(ReaderLoop) }; // priority, etc if needed
thread.Start(cancellationToken);
Your Process can be an event, which you can invoke asynchronously so your reader loop can be return immediately to process the new incoming packages as fast as possible:
private void ReaderLoop(object state)
{
var token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
try
{
var message = MessageQueue.Take(token);
OnMessageReceived(new MessageReceivedEventArgs(message));
}
catch (OperationCanceledException)
{
if (!disposed && IsRunning)
Stop();
break;
}
}
}
Please note that if a delegate has multiple targets it's async invocation is not trivial. I created this extension method for invoking a delegate on pool threads:
public static void InvokeAsync<TEventArgs>(this EventHandler<TEventArgs> eventHandler, object sender, TEventArgs args)
{
void Callback(IAsyncResult ar)
{
var method = (EventHandler<TEventArgs>)ar.AsyncState;
try
{
method.EndInvoke(ar);
}
catch (Exception e)
{
HandleError(e, method);
}
}
foreach (EventHandler<TEventArgs> handler in eventHandler.GetInvocationList())
handler.BeginInvoke(sender, args, Callback, handler);
}
So the OnMessageReceived implementation can be:
protected virtual void OnMessageReceived(MessageReceivedEventArgs e)
=> messageReceivedHandler.InvokeAsync(this, e);
Finally it was a big lesson that BlockingCollection<T> has some performance issues. It uses SpinWait internally, whose SpinOnce method waits longer and longer times if there is no incoming data for a long time. This is a tricky issue because even if you log every single step of the processing you will not notice that everything is started delayed unless you can mock also the server side. Here you can find a fast BlockingCollection implementation using an AutoResetEvent for triggering incoming data. I added a Take(CancellationToken) overload to it as follows:
/// <summary>
/// Takes an item from the <see cref="FastBlockingCollection{T}"/>
/// </summary>
public T Take(CancellationToken token)
{
T item;
while (!queue.TryDequeue(out item))
{
waitHandle.WaitOne(cancellationCheckTimeout); // can be 10-100 ms
token.ThrowIfCancellationRequested();
}
return item;
}
Basically that's it. Maybe not everything is applicable in your case, eg. if the nearly immediate response is not crucial the regular BlockingCollection also will do it.
Yes, this is a bit inefficient, because you block ThreadPool threads.
I already discussed this problem Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern
You can also look at examples with testing a producer -consumer pattern here:
https://github.com/BBGONE/TestThreadAffinity
You can use await Task.Yield in the loop to give other tasks access to this thread.
You can solve it also by using dedicated threads or better a custom ThreadScheduler which uses its own thread pool. But it is ineffective to create 50+ plain threads. Better to adjust the task, so it would be more cooperative.
If you use a BlockingCollection (because it can block the thread for long while waiting to write (if bounded) or to read or no items to read) then it is better to use System.Threading.Tasks.Channels https://github.com/stephentoub/corefxlab/blob/master/src/System.Threading.Tasks.Channels/README.md
They don't block the thread while waiting when the collection will be available to write or to read. There's an example how it is used https://github.com/BBGONE/TestThreadAffinity/tree/master/ThreadingChannelsCoreFX/ChannelsTest
Let's say I have a Windows Service which is doing some bit of work, then sleeping for a short amount of time, over and over forever (until the service is shut down). So in the service's OnStart, I could start up a thread whose entry point is something like:
private void WorkerThreadFunc()
{
while (!shuttingDown)
{
DoSomething();
Thread.Sleep(10);
}
}
And in the service's OnStop, I somehow set that shuttingDown flag and then join the thread. Actually there might be several such threads, and other threads too, all started in OnStart and shut down/joined in OnStop.
If I want to instead do this sort of thing in an async/await based Windows Service, it seems like I could have OnStart create cancelable tasks but not await (or wait) on them, and have OnStop cancel those tasks and then Task.WhenAll().Wait() on them. If I understand correctly, the equivalent of the "WorkerThreadFunc" shown above might be something like:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
DoSomething();
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #1: Uh... right? I am new to async/await and still trying to get my head around it.
Assuming that's right, now let's say that DoSomething() call is (or includes) a synchronous write I/O to some piece of hardware. If I'm understanding correctly:
Question #2: That is bad? I shouldn't be doing synchronous I/O within a Task in an async/await-based program? Because it ties up a thread from the thread pool while the I/O is happening, and threads from the thread pool are a highly limited resource? Please note that I might have dozens of such Workers going simultaneously to different pieces of hardware.
I am not sure I'm understanding that correctly - I am getting the idea that it's bad from articles like Stephen Cleary's "Task.Run Etiquette Examples: Don't Use Task.Run for the Wrong Thing", but that's specifically about it being bad to do blocking work within Task.Run. I'm not sure if it's also bad if I'm just doing it directly, as in the "private async Task Work()" example above?
Assuming that's bad too, then if I understand correctly I should instead utilize the nonblocking version of DoSomething (creating a nonblocking version of it if it doesn't already exist), and then:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
await DoSomethingAsync(cancel).ConfigureAwait(false);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #3: But... what if DoSomething is from a third party library, which I must use and cannot alter, and that library doesn't expose a nonblocking version of DoSomething? It's just a black box set in stone that at some point does a blocking write to a piece of hardware.
Maybe I wrap it and use TaskCompletionSource? Something like:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
cancel.ThrowIfCancellationRequested();
await WrappedDoSomething().ConfigureAwait(false);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
private Task WrappedDoSomething()
{
var tcs = new TaskCompletionSource<object>();
DoSomething();
tcs.SetResult(null);
return tcs.Task;
}
But that seems like it's just pushing the issue down a bit further rather than resolving it. WorkAsync() will still block when it calls WrappedDoSomething(), and only get to the "await" for that after WrappedDoSomething() has already completed the blocking work. Right?
Given that (if I understand correctly) in the general case async/await should be allowed to "spread" all the way up and down in a program, would this mean that if I need to use such a library, I essentially should not make the program async/await-based? I should go back to the Thread/WorkerThreadFunc/Thread.Sleep world?
What if an async/await-based program already exists, doing other things, but now additional functionality that uses such a library needs to be added to it? Does that mean that the async/await-based program should be rewritten as a Thread/etc.-based program?
Actually there might be several such threads, and other threads too, all started in OnStart and shut down/joined in OnStop.
On a side note, it's usually simpler to have a single "master" thread that will start/join all the others. Then OnStart/OnStop just deals with the master thread.
If I want to instead do this sort of thing in an async/await based Windows Service, it seems like I could have OnStart create cancelable tasks but not await (or wait) on them, and have OnStop cancel those tasks and then Task.WhenAll().Wait() on them.
That's a perfectly acceptable approach.
If I understand correctly, the equivalent of the "WorkerThreadFunc" shown above might be something like:
Probably want to pass the CancellationToken down; cancellation can be used by synchronous code, too:
private async Task WorkAsync(CancellationToken cancel)
{
while (true)
{
DoSomething(cancel);
await Task.Delay(10, cancel).ConfigureAwait(false);
}
}
Question #1: Uh... right? I am new to async/await and still trying to get my head around it.
It's not wrong, but it only saves you one thread on a Win32 service, which doesn't do much for you.
Question #2: That is bad? I shouldn't be doing synchronous I/O within a Task in an async/await-based program? Because it ties up a thread from the thread pool while the I/O is happening, and threads from the thread pool are a highly limited resource? Please note that I might have dozens of such Workers going simultaneously to different pieces of hardware.
Dozens of threads are not a lot. Generally, asynchronous I/O is better because it doesn't use any threads at all, but in this case you're on the desktop, so threads are not a highly limited resource. async is most beneficial on UI apps (where the UI thread is special and needs to be freed), and ASP.NET apps that need to scale (where the thread pool limits scalability).
Bottom line: calling a blocking method from an asynchronous method is not bad but it's not the best, either. If there is an asynchronous method, call that instead. But if there isn't, then just keep the blocking call and document it in the XML comments for that method (because an asynchronous method blocking is rather surprising behavior).
I am getting the idea that it's bad from articles like Stephen Cleary's "Task.Run Etiquette Examples: Don't Use Task.Run for the Wrong Thing", but that's specifically about it being bad to do blocking work within Task.Run.
Yes, that is specifically about using Task.Run to wrap synchronous methods and pretend they're asynchronous. It's a common mistake; all it does is trade one thread pool thread for another.
Assuming that's bad too, then if I understand correctly I should instead utilize the nonblocking version of DoSomething (creating a nonblocking version of it if it doesn't already exist)
Asynchronous is better (in terms of resource utilization - that is, fewer threads used), so if you want/need to reduce the number of threads, you should use async.
Question #3: But... what if DoSomething is from a third party library, which I must use and cannot alter, and that library doesn't expose a nonblocking version of DoSomething? It's just a black box set in stone that at some point does a blocking write to a piece of hardware.
Then just call it directly.
Maybe I wrap it and use TaskCompletionSource?
No, that doesn't do anything useful. That just calls it synchronously and then returns an already-completed task.
But that seems like it's just pushing the issue down a bit further rather than resolving it. WorkAsync() will still block when it calls WrappedDoSomething(), and only get to the "await" for that after WrappedDoSomething() has already completed the blocking work. Right?
Yup.
Given that (if I understand correctly) in the general case async/await should be allowed to "spread" all the way up and down in a program, would this mean that if I need to use such a library, I essentially should not make the program async/await-based? I should go back to the Thread/WorkerThreadFunc/Thread.Sleep world?
Assuming you already have a blocking Win32 service, it's probably fine to just keep it as it is. If you are writing a new one, personally I would make it async to reduce threads and allow asynchronous APIs, but you don't have to do it either way. I prefer Tasks over Threads in general, since it's much easier to get results from Tasks (including exceptions).
The "async all the way" rule only goes one way. That is, once you call an async method, then its caller should be async, and its caller should be async, etc. It does not mean that every method called by an async method must be async.
So, one good reason to have an async Win32 service would be if there's an async-only API you need to consume. That would cause your DoSomething method to become async DoSomethingAsync.
What if an async/await-based program already exists, doing other things, but now additional functionality that uses such a library needs to be added to it? Does that mean that the async/await-based program should be rewritten as a Thread/etc.-based program?
No. You can always just block from an async method. With proper documentation so when you are reusing/maintaining this code a year from now, you don't swear at your past self. :)
If you still spawn your threads, well, yes, it's bad. Because it will not give you any benefit as the thread is still allocated and consuming resources for the specific purpose of running your worker function. Running a few threads to be able to do work in parallel within a service has a minimal impact on your application.
If DoSomething() is synchronous, you could switch to the Timer class instead. It allows multiple timers to use a smaller amount of threads.
If it's important that the jobs can complete, you can modify your worker classes like this:
SemaphoreSlim _shutdownEvent = new SemaphoreSlim(0,1);
public async Task Stop()
{
return await _shutdownEvent.WaitAsync();
}
private void WorkerThreadFunc()
{
while (!shuttingDown)
{
DoSomething();
Thread.Sleep(10);
}
_shutdownEvent.Release();
}
.. which means that during shutdown you can do this:
var tasks = myServices.Select(x=> x.Stop());
Task.WaitAll(tasks);
A thread can only do one thing at a time. While it is working on your DoSomething it can't do anything else.
In an interview Eric Lippert described async-await in a restaurant metaphor. He suggests to use async-await only for functionality where your thread can do other things instead of waiting for a process to complete, like respond to operator input.
Alas, your thread is not waiting, it is doing hard work in DoSomething. And as long as DoSomething is not awaiting, your thread will not return from DoSomething to do the next thing.
So if your thread has something meaningful to do while procedure DoSomething is executing, it's wise to let another thread do the DoSomething, while your original thread is doing the meaningful stuff. Task.Run( () => DoSomething()) could do this for you. As long as the thread that called Task.Run doesn't await for this task, it is free to do other things.
You also want to cancel your process. DoSomething can't be cancelled. So even if cancellation is requested you'll have to wait until DoSomething is completed.
Below is your DoSomething in a form with a Start button and a Cancel button. While your thread is DoingSomething, one of the meaningful things your GUI thread may want to do is respond to pressing the cancel button:
void CancellableDoSomething(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
DoSomething()
}
}
async Task DoSomethingAsync(CancellationToken token)
{
var task = Task.Run(CancellableDoSomething(token), token);
// if you have something meaningful to do, do it now, otherwise:
return Task;
}
CancellationTokenSource cancellationTokenSource = null;
private async void OnButtonStartSomething_Clicked(object sender, ...)
{
if (cancellationTokenSource != null)
// already doing something
return
// else: not doing something: start doing something
cancellationTokenSource = new CancellationtokenSource()
var task = AwaitDoSomethingAsync(cancellationTokenSource.Token);
// if you have something meaningful to do, do it now, otherwise:
await task;
cancellationTokenSource.Dispose();
cancellationTokenSource = null;
}
private void OnButtonCancelSomething(object sender, ...)
{
if (cancellationTokenSource == null)
// not doing something, nothing to cancel
return;
// else: cancel doing something
cancellationTokenSource.Cancel();
}
I've been reading about Tasks after asking this question and seeing that I completely misunderstood the concept. Answers such as the top answers here and here explain the idea, but I still don't get it.
So I've made this a very specific question: What actually happens on the CPU when a Task is executed?
This is what I've understood after some reading: A Task will share CPU time with the caller (and let's assume the caller is the "UI") so that if it's CPU-intensive - it will slow down the UI. If the Task is not CPU-intensive - it will be running "in the background". Seems clear enough …… until tested. The following code should allow the user to click on the button, and then alternately show "Shown" and "Button". But in reality: the Form is completely busy (-no user input possible) until the "Shown"s are all shown.
public Form1()
{
InitializeComponent();
Shown += Form1_Shown;
}
private async void Form1_Shown(object sender, EventArgs e)
{
await Doit("Shown");
}
private async Task Doit(string s)
{
WebClient client = new WebClient();
for (int i = 0; i < 10; i++)
{
client.DownloadData(uri);//This is here in order to delay the Text writing without much CPU use.
textBox1.Text += s + "\r\n";
this.Update();//textBox1.
}
}
private async void button1_Click(object sender, EventArgs e)
{
await Doit("Button");
}
Can someone please tell me what is actually happening on the CPU when a Task is executed (e.g. "When the CPU is not used by the UI, the Task uses it, except for when… etc.")?
The key to understanding this is that there are two kinds of tasks - one that executes code (what I call Delegate Tasks), and one that represents a future event (what I call Promise Tasks). Those two tasks are completely different, even though they're both represented by an instance of Task in .NET. I have some pretty pictures on my blog that may help understand how these types of task are different.
Delegate Tasks are the ones created by Task.Run and friends. They execute code on the thread pool (or possibly another TaskScheduler if you're using a TaskFactory). Most of the "task parallel library" documentation deals with Delegate Tasks. These are used to spread CPU-bound algorithms across multiple CPUs, or to push CPU-bound work off a UI thread.
Promise Tasks are the ones created by TaskCompletionSource<T> and friends (including async). These are the ones used for asynchronous programming, and are a natural fit for I/O-bound code.
Note that your example code will cause a compiler warning to the effect that your "asynchronous" method Doit is not actually asynchronous but is instead synchronous. So as it stands right now, it will synchronously call DownloadData, blocking the UI thread until the download completes, and then it will update the text box and finally return an already-completed task.
To make it asynchronous, you have to use await:
private async Task Doit(string s)
{
WebClient client = new WebClient();
for (int i = 0; i < 10; i++)
{
await client.DownloadDataTaskAsync(uri);
textBox1.Text += s + "\r\n";
this.Update();//textBox1.
}
}
Now it's returning an incomplete task when it hits the await, which allows the UI thread to return to its message processing loop. When the download completes, the remainder of this method will be queued to the UI thread as a message, and it will resume executing that method when it gets around to it. When the Doit method completes, then the task it returned earlier will complete.
So, tasks returned by async methods logically represent that method. The task itself is a Promise Task, not a Delegate Task, and does not actually "execute". The method is split into multiple parts (at each await point) and executes in chunks, but the task itself does not execute anywhere.
For further reading, I have a blog post on how async and await actually work (and how they schedule the chunks of the method), and another blog post on why asynchronous I/O tasks do not need to block threads.
As per your linked answers, Tasks and Threads are totally different concepts, and you are also getting confused with async / await
A Task is just a representation of some work to be done. It says nothing about HOW that work should be done.
A Thread is a representation of some work that is running on the CPU, but is sharing the CPU time with other threads that it can know nothing about.
You can run a Task on a Thread using Task.Run(). Your Task will run asynchronously and independently of any other code providing a threadpool thread is available.
You can also run a Task asynchronously on the SAME thread using async / await. Anytime the thread hits an await, it can save the current stack state, then travel back up the stack and carry on with other work until the awaited task has finished. Your Doit() code never awaits anything, so will run synchronously on your GUI thread until complete.
Tasks use the ThreadPool you can read extensively about what it is and how it works here
But in a nutshell, when a task is executed, the Task Scheduler looks in the ThreadPool to see if there is a thread available to run the action of the task. If not, it's going to be queued until one becomes available.
A ThreadPool is just a collection of already-instantiated threads made available so that multithreaded code can safely use concurrent programming without overwhelming the CPU with context-switching all the time.
Now, the problem with your code is that even though you return an object of type Task, you are not running anything concurrently - No separate thread is ever started!
In order to do that, you have two options, either you start yourDoit method as a Task, with
Option1
Task.Run(() => DoIt(s));
This will run the whole DoIt method on another thread from the Thread Pool, but it will lead to more problems, because in this method, you're trying to access UI-controls. therefore, you will need either to marshal those calls to the UI thread, or re-think your code so that the UI access is done directly on the UI thread after the asynchronous tasks completes.
Option 2 (preferred, if you can)
You use .net APIs which are already asynchronous, such as client.DownloadDataTaskAsync(); instead of client.DownloadData();
now, in your case, the problem is that you will need to have 10 calls, which are going to return 10 different objects of type Task<byte[]> and you want to await on the completion of all of them, not just one.
In order to do this, you will need to create a List<Task<byte[]>> returnedTasks and you will add to it all returned value from DownloadDataTaskAsync(). then, once this is done, you can use the following return value for your DoIt method.
return Task.WhenAll(returnedTasks);
I have 2 services servicing WCF calls. From a client I send the same asynchronous WCF BeginXXX call to both services and then start waiting for replies with a WaitHandle.WaitAny(waitHandles) where waitHandles is an array of WaitHandles from the IAsyncResults returned by the 2 BeginXXX calls.
I want to use only the reply from the service that answers faster, i.e. when WaitHandle.WaitAny returns with an index I only call EndXXX with the corresponding IAsyncResult to get the faster result. I don't ever call the other EndXXX.
My reason for doing this is that sometimes a service uses several seconds in garbage collection and is not able to answer fast. According to my experiences the 2 services do garbage collections usually in different times so one of them is almost always capable of returning a fast answer. My client application is very time critical, I need an answer within a few milliseconds.
My questions are:
Can I safely ignore calling EndXXX method for the other service that was slower in answering? I am not interested in the slower result but want to use the faster result ASAP. According to my experiments nothing bad seems to happen even if I don't call the EndXXX method for the corresponding slower BeginXXX async result.
Would somebody mind explaining to me what exactly happens when I don't make an EndXXX call for a corresponding BeginXXX? Under debugger in Visual Studio I seem to able to see that another answer is processed in the .NET framework via an I/O completion port and this processing does not originate from my client calling EndXXX. And I don't seem to have any memory leaks because of not making the EndXXX call. I presume all objects involved are garbage collected.
Does it make any difference whether the server side method XXX implementation is a single synchronous XXX or an explicit asynchronous BeginXXX/EndXXX pair?
IMHO a synchronous XXX method implementation will always return an answer that
needs to be handled somewhere. Does it happen on client or server
side in my case when I fail to call EndXXX?
Is using the WaitHandles a good and most efficient way of waiting for the fastest result?
If I have to call EndXXX for each BeginXXX I have sent out makes things quite awkward. I would have to delegate the uninteresting EndXXX call into another thread that would just ignore the results. Calling all EndXXX calls in my original thread would defeat the purpose of getting hold of and using the faster answer in a synchronous manner.
The documentation says that you have to call the end method. If you violate what the docs demand you are in undefined behavior land. Resources can leak. Maybe they just do so under load, who knows?
I don't know, sorry. I'm giving a partial answer. My suggestion: Implement a service method that does nothing and invoke it 10M times in a loop. Do resources leak? If yes, you have your answer.
No, Server and client are independent. The server can be sync, the client async or vice versa. Both cannot even tell the difference of what the other does. The two services are separated by TCP and a well-defined protocol. It is impossible for a client to even know what the server does. The server does not even have to use .NET.
I'm not sure what you're asking. Under the hood, WCF clients use TCP. Incoming data will be handled "somewhere" (in practice on the thread-pool).
If your code is fundamentally synchronous, this is the best you can do. You'll burn one thread waiting for N asynchronous service calls. That's ok.
Why don't you just specify a callback in BeginXXX that does nothing else but call EndXXX? That way you always call EndXXX and conform to how the framework is meant to be used. You can still use wait handles.
Depends on the object you call the begin/end pattern on. some are known to leak. from CLR via C# by Jeffrey Richter:
You must call Endxxx or you will leak resources. CLR allocates some
internal resources when you initiate asynchronous operation. If Endxxx
is never called, these resources will be reclaimed only when the
process terminates.
AFAIK the Task-based pattern uses the thread pool to handle its work.
My client makes thousands of calls per second and would completely
trash the thread pool.
It would be so if you used Task.Run or Task.Factory.StartNew. By itself, Task.Factory.FromAsync doesn't create or switch threads explicitly.
Back to your scenatio:
I want to use only the reply from the service that answers faster,
i.e. when WaitHandle.WaitAny returns with an index I only call EndXXX
with the corresponding IAsyncResult to get the faster result. I don't
ever call the other EndXXX.
Let's create the Task wrapper for BeginXXX/EndXXX asynchronous service call:
public static class WcfExt
{
public static Task<object> WorkAsync(this IService service, object arg)
{
return Task.Factory.FromAsync(
(asyncCallback, asyncState) =>
service.BeginWork(arg, asyncCallback, asyncState),
(asyncResult) =>
service.EndWork(asyncResult), null);
}
}
And implement the whatever-service-answers-faster logic:
static async Task<object> CallBothServicesAsync(
IService service1, IService service2, object arg)
{
var task1 = service1.WorkAsync(arg);
var task2 = service2.WorkAsync(arg);
var resultTask = await Task.WhenAny(task1, task2).ConfigureAwait(false);
return resultTask.Result;
}
So far, there has been no blocking code and we still don't create new threads explicitly. The WorkAsync wrapper passes a continuation callback to BeginWork. This callback will be called by the service when the operation started by BeginWork has finished.
It will be called on whatever thread happened to serve the completion of such operation. Most often, this is a random IOCP (input/output completion port) thread from the thread pool. For more details, check Stephen Cleary's "There Is No Thread". The completion callback will automatically call EndWork to finalize the operation and retrieve its result, so the service won't leak resources, and store the result inside the Task<object> instance (returned by WorkAsync).
Then, your code after await Task.WhenAny will continue executing on that particular thread. So, there may be a thread switch after await, but it naturally uses the IOCP thread where the asynchronous operation has completed.
You almost never need to use low-level synchronization primitives like manual reset events with Task Parallel Library. E.g., if you need to wait on the result of CallBothServicesAsync, you'd simple do:
var result = CallBothServicesAsync(service1, service2).Result;
Console.WriteLine("Result: " + result);
Which is the same as:
var task = CallBothServicesAsync(service1, service2);
task.Wait();
Console.WriteLine("Result: " + task.result);
This code would block the current thread, similarly to what WaitHandle.WaitAny does in your original scenario.
Now, blocking like this is not recommended either, as you'd loose the advantage of the asynchronous programming model and hurt the scalability of your app. The blocked thread could be doing some other useful work rather than waiting, e.g., in case with a web app, it could be serving another incoming client-side request.
Ideally, your logic should be "async all the way", up to some root entry point. E.g., with a console app:
static async Task CoreLoopAsync(CancellationToken token)
{
using(var service1 = CreateWcfClientProxy())
using(var service2 = CreateWcfClientProxy())
{
while (true)
{
token.ThrowIfCancellationRequested();
var result = await CallBothServicesAsync("data");
Console.WriteLine("Result: " + result);
}
}
}
static void Main()
{
var cts = CancellationTokenSource(10000); // cancel in 10s
try
{
// block at the "root" level, i.e. inside Main
CoreLoopAsync(cts.Token).Wait();
}
catch (Exception ex)
{
while (ex is AggregatedException)
ex = ex.InnerException;
// report the error
Console.WriteLine(ex.Message);
}
}
Microsoft just announced the new C# Async feature. Every example I've seen so far is about asynchronously downloading something from HTTP. Surely there are other important async things?
Suppose I'm not writing a new RSS client or Twitter app. What's interesting about C# Async for me?
Edit I had an Aha! moment while watching Anders' PDC session. In the past I have worked on programs that used "watcher" threads. These threads sit waiting for something to happen, like watching for a file to change. They aren't doing work, they're just idle, and notify the main thread when something happens. These threads could be replaced with await/async code in the new model.
Ooh, this sounds interesting. I'm not playing with the CTP just yet, just reviewing the whitepaper. After seeing Anders Hejlsberg's talk about it, I think I can see how it could prove useful.
As I understand, async makes writing asynchronous calls easier to read and implement. Very much in the same way writing iterators is easier right now (as opposed to writing out the functionality by hand). This is essential blocking processes since no useful work can be done, until it is unblocked. If you were downloading a file, you cannot do anything useful until you get that file letting the thread go to waste. Consider how one would call a function which you know will block for an undetermined length and returns some result, then process it (e.g., store the results in a file). How would you write that? Here's a simple example:
static object DoSomeBlockingOperation(object args)
{
// block for 5 minutes
Thread.Sleep(5 * 60 * 1000);
return args;
}
static void ProcessTheResult(object result)
{
Console.WriteLine(result);
}
static void CalculateAndProcess(object args)
{
// let's calculate! (synchronously)
object result = DoSomeBlockingOperation(args);
// let's process!
ProcessTheResult(result);
}
Ok good, we have it implemented. But wait, the calculation takes minutes to complete. What if we wanted to have an interactive application and do other things while the calculation took place (such as rendering the UI)? This is no good, since we called the function synchronously and we have to wait for it to finish effectively freezing the application since the thread is waiting to be unblocked.
Answer, call the function expensive function asynchronously. That way we're not bound to waiting for the blocking operation to complete. But how do we do that? We'd call the function asynchronously and register a callback function to be called when unblocked so we may process the result.
static void CalculateAndProcessAsyncOld(object args)
{
// obtain a delegate to call asynchronously
Func<object, object> calculate = DoSomeBlockingOperation;
// define the callback when the call completes so we can process afterwards
AsyncCallback cb = ar =>
{
Func<object, object> calc = (Func<object, object>)ar.AsyncState;
object result = calc.EndInvoke(ar);
// let's process!
ProcessTheResult(result);
};
// let's calculate! (asynchronously)
calculate.BeginInvoke(args, cb, calculate);
}
Note: Sure we could start another thread to do this but that would mean we're spawning a thread that just sits there waiting to be unblocked, then do some useful work. That would be a waste.
Now the call is asynchronous and we don't have to worry about waiting for the calculation to finish and process, it's done asynchronously. It will finish when it can. An alternative to calling code asynchronously directly, you could use a Task:
static void CalculateAndProcessAsyncTask(object args)
{
// create a task
Task<object> task = new Task<object>(DoSomeBlockingOperation, args);
// define the callback when the call completes so we can process afterwards
task.ContinueWith(t =>
{
// let's process!
ProcessTheResult(t.Result);
});
// let's calculate! (asynchronously)
task.Start();
}
Now we called our function asynchronously. But what did it take to get it that way? First of all, we needed the delegate/task to be able to call it asynchronously, we needed a callback function to be able to process the results, then call the function. We've turned a two line function call to much more just to call something asynchronously. Not only that, the logic in the code has gotten more complex then it was or could be. Although using a task helped simplify the process, we still needed to do stuff to make it happen. We just want to run asynchronously then process the result. Why can't we just do that? Well now we can:
// need to have an asynchronous version
static async Task<object> DoSomeBlockingOperationAsync(object args)
{
//it is my understanding that async will take this method and convert it to a task automatically
return DoSomeBlockingOperation(args);
}
static async void CalculateAndProcessAsyncNew(object args)
{
// let's calculate! (asynchronously)
object result = await DoSomeBlockingOperationAsync(args);
// let's process!
ProcessTheResult(result);
}
Now this was a very simplified example with simple operations (calculate, process). Imagine if each operation couldn't conveniently be put into a separate function but instead have hundreds of lines of code. That's a lot of added complexity just to gain the benefit of asynchronous calling.
Another practical example used in the whitepaper is using it on UI apps. Modified to use the above example:
private async void doCalculation_Click(object sender, RoutedEventArgs e) {
doCalculation.IsEnabled = false;
await DoSomeBlockingOperationAsync(GetArgs());
doCalculation.IsEnabled = true;
}
If you've done any UI programming (be it WinForms or WPF) and attempted to call an expensive function within a handler, you'll know this is handy. Using a background worker for this wouldn't be that much helpful since the background thread will be sitting there waiting until it can work.
Suppose you had a way to control some external device, let's say a printer. And you wanted to restart the device after a failure. Naturally it will take some time for the printer to start up and be ready for operation. You might have to account for the restart not helping and attempt to restart again. You have no choice but to wait for it. Not if you did it asynchronously.
static async void RestartPrinter()
{
Printer printer = GetPrinter();
do
{
printer.Restart();
printer = await printer.WaitUntilReadyAsync();
} while (printer.HasFailed);
}
Imagine writing the loop without async.
One last example I have. Imagine if you had to do multiple blocking operations in a function and wanted to call asynchronously. What would you prefer?
static void DoOperationsAsyncOld()
{
Task op1 = new Task(DoOperation1Async);
op1.ContinueWith(t1 =>
{
Task op2 = new Task(DoOperation2Async);
op2.ContinueWith(t2 =>
{
Task op3 = new Task(DoOperation3Async);
op3.ContinueWith(t3 =>
{
DoQuickOperation();
}
op3.Start();
}
op2.Start();
}
op1.Start();
}
static async void DoOperationsAsyncNew()
{
await DoOperation1Async();
await DoOperation2Async();
await DoOperation3Async();
DoQuickOperation();
}
Read the whitepaper, it actually has a lot of practical examples like writing parallel tasks and others.
I can't wait to start playing with this either in the CTP or when .NET 5.0 finally makes it out.
The main scenarios are any scenario that involves high latency. That is, lots of time between "ask for a result" and "obtain a result". Network requests are the most obvious example of high latency scenarios, followed closely by I/O in general, and then by lengthy computations that are CPU bound on another core.
However, there are potentially other scenarios that this technology will mesh nicely with. For example, consider scripting the logic of a FPS game. Suppose you have a button click event handler. When the player clicks the button you want to play a siren for two seconds to alert the enemies, and then open the door for ten seconds. Wouldn't it be nice to say something like:
button.Disable();
await siren.Activate();
await Delay(2000);
await siren.Deactivate();
await door.Open();
await Delay(10000);
await door.Close();
await Delay(1000);
button.Enable();
Each task gets queued up on the UI thread, so nothing blocks, and each one resumes the click handler at the right point after its job is finished.
I've found another nice use-case for this today: you can await user interaction.
For example, if one form has a button that opens another form:
Form toolWindow;
async void button_Click(object sender, EventArgs e) {
if (toolWindow != null) {
toolWindow.Focus();
} else {
toolWindow = new Form();
toolWindow.Show();
await toolWindow.OnClosed();
toolWindow = null;
}
}
Granted, this isn't really any simpler than
toolWindow.Closed += delegate { toolWindow = null; }
But I think it nicely demonstrates what await can do. And once the code in the event handler is non-trivial, await make programming much easier. Think about the user having to click a sequence of buttons:
async void ButtonSeries()
{
for (int i = 0; i < 10; i++) {
Button b = new Button();
b.Text = i.ToString();
this.Controls.Add(b);
await b.OnClick();
this.Controls.Remove(b);
}
}
Sure, you could do this with normal event handlers, but it would require you to take apart the loop and convert it into something much harder to understand.
Remember that await can be used with anything that gets completed at some point in the future. Here's the extension method Button.OnClick() to make the above work:
public static AwaitableEvent OnClick(this Button button)
{
return new AwaitableEvent(h => button.Click += h, h => button.Click -= h);
}
sealed class AwaitableEvent
{
Action<EventHandler> register, deregister;
public AwaitableEvent(Action<EventHandler> register, Action<EventHandler> deregister)
{
this.register = register;
this.deregister = deregister;
}
public EventAwaiter GetAwaiter()
{
return new EventAwaiter(this);
}
}
sealed class EventAwaiter
{
AwaitableEvent e;
public EventAwaiter(AwaitableEvent e) { this.e = e; }
Action callback;
public bool BeginAwait(Action callback)
{
this.callback = callback;
e.register(Handler);
return true;
}
public void Handler(object sender, EventArgs e)
{
callback();
}
public void EndAwait()
{
e.deregister(Handler);
}
}
Unfortunately it doesn't seem possible to add the GetAwaiter() method directly to EventHandler (allowing await button.Click;) because then the method wouldn't know how to register/deregister that event.
It's a bit of boilerplate, but the AwaitableEvent class can be re-used for all events (not just UI). And with a minor modification and adding some generics, you could allow retrieving the EventArgs:
MouseEventArgs e = await button.OnMouseDown();
I could see this being useful with some more complex UI gestures (drag'n'drop, mouse gestures, ...) - though you'd have to add support for cancelling the current gesture.
There are some samples and demos in the CTP that don't use the Net, and even some that don't do any I/O.
And it does apply to all multithreaded / parallel problem areas (that already exist).
Async and Await are a new (easier) way of structuring all parallel code, be it CPU-bound or I/O bound. The biggest improvement is in areas where before C#5 you had to use the APM (IAsyncResult) model, or the event model (BackgroundWorker, WebClient). I think that is why those examples lead the parade now.
A GUI clock is a good example; say you want to draw a clock, that updates the time shown every second. Conceptually, you want to write
while true do
sleep for 1 second
display the new time on the clock
and with await (or with F# async) to asynchronously sleep, you can write this code to run on the UI thread in a non-blocking fashion.
http://lorgonblog.wordpress.com/2010/03/27/f-async-on-the-client-side/
The async extensions are useful in some cases when you have an asynchronous operation. An asynchronous operation has a definite start and completion. When asynchronous operations complete, they may have a result or an error. (Cancellation is treated as a special kind of error).
Asynchronous operations are useful in three situations (broadly speaking):
Keeping your UI responsive. Any time you have a long-running operation (whether CPU-bound or I/O-bound), make it asynchronous.
Scaling your servers. Using asynchronous operations judiciously on the server side may help your severs to scale. e.g., asynchronous ASP.NET pages may make use of async operations. However, this is not always a win; you need to evaluate your scalability bottlenecks first.
Providing a clean asynchronous API in a library or shared code. async is excellent for reusability.
As you begin to adopt the async way of doing things, you'll find the third situation becoming more common. async code works best with other async code, so asynchronous code kind of "grows" through the codebase.
There are a couple of types of concurrency where async is not the best tool:
Parallelization. A parallel algorithm may use many cores (CPUs, GPUs, computers) to solve a problem more quickly.
Asynchronous events. Asynchronous events happen all the time, independent of your program. They often do not have a "completion." Normally, your program will subscribe to an asynchronous event stream, receive some number of updates, and then unsubscribe. Your program can treat the subscribe and unsubscribe as a "start" and "completion", but the actual event stream never really stops.
Parallel operations are best expressed using PLINQ or Parallel, since they have a lot of built-in support for partitioning, limited concurrency, etc. A parallel operation may easily be wrapped in an awaitable by running it from a ThreadPool thread (Task.Factory.StartNew).
Asynchronous events do not map well to asynchronous operations. One problem is that an asynchronous operation has a single result at its point of completion. Asynchronous events may have any number of updates. Rx is the natural language for dealing with asynchronous events.
There are some mappings from an Rx event stream to an asynchronous operation, but none of them are ideal for all situations. It's more natural to consume asynchronous operations by Rx, rather than the other way around. IMO, the best way of approaching this is to use asynchronous operations in your libraries and lower-level code as much as possible, and if you need Rx at some point, then use Rx from there on up.
Here is probably a good example of how not to use the new async feature (that's not writing a new RSS client or Twitter app), mid-method overload points in a virtual method call. To be honest, i am not sure there is any way to create more than a single overload point per method.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
namespace AsyncText
{
class Program
{
static void Main(string[] args)
{
Derived d = new Derived();
TaskEx.Run(() => d.DoStuff()).Wait();
System.Console.Read();
}
public class Base
{
protected string SomeData { get; set; }
protected async Task DeferProcessing()
{
await TaskEx.Run(() => Thread.Sleep(1) );
return;
}
public async virtual Task DoStuff() {
Console.WriteLine("Begin Base");
Console.WriteLine(SomeData);
await DeferProcessing();
Console.WriteLine("End Base");
Console.WriteLine(SomeData);
}
}
public class Derived : Base
{
public async override Task DoStuff()
{
Console.WriteLine("Begin Derived");
SomeData = "Hello";
var x = base.DoStuff();
SomeData = "World";
Console.WriteLine("Mid 1 Derived");
await x;
Console.WriteLine("EndDerived");
}
}
}
}
Output Is:
Begin Derived
Begin Base
Hello
Mid 1 Derived
End Base
World
EndDerived
With certain inheritance hierarchies (namely using command pattern) i find myself wanting to do stuff like this occasionally.
here is an article about showing how to use the 'async' syntax in a non-networked scenario that involves UI and multiple actions.