We Keep Coding

Does TaskCompletionSource guarantee awaiting code will resume on the thread which called TCS.TrySetResult?

I have a method which wraps a callback using TaskCompletionSource like this:
public Task<int> TestMethod(int argument)
{
var tcs = new TaskCompletionSource<int>();
var task = tcs.Task;
LegacyMethodWithCallback(argument, (returnValue) => tcs.TrySetResult(returnValue));
return task;
}
And then I await on that method:
async int CallingMethod()
{
var returnValue = await TestMethod(5);
Console.WriteLine(returnValue);
return returnValue;
}
EDIT: LegacyMethodWithCallback is a method communicating with a server using the network. The callback runs on a thread belonging to our proprietary thread pool (which is a reason to implement a custom SynchronizationContext). The custom SynchronizationContext is basically blank except its Post method enqueues the delegate into out thread pool. Here is implementation of the SC that should demonstrate when Post is called:
internal class ServiceSynchronizationContext : SynchronizationContext
{
public override void Post(SendOrPostCallback d, object state)
{
Console.WriteLine("Post " + d + ": " + state + " for " + this.serviceLogic.ServiceLogicId);
// Enqueue the delegate into our thread pool
}
}
I have implemented my own synchronization context, but I was suprised to see that CallingMethod appears to resume immediately on the same thread which called the TrySetResult (and therefore the same thread which invoked the callback). I originally expected the continuation to be Post-ed via the SynchronizationContext.
Is this behavior guaranteed or are there any cases in which the SynchronizationContext (or any other thread) is invoked? If this behavior is not fully deterministic, can I control it somehow?
I thoroughly inspected the call stack at the moment when CallingMethod resumes and it appears that the SynchronizationContext is not even considered (but there is still some code that cannot be easily looked into), so it appears that this behavior is inded guaranteed. Am I correct?

Is this behavior guaranteed or are there any cases in which the SynchronizationContext (or any other thread) is invoked?
SynchronizationContext, if present, is always used by the awaiting code (unless specified otherwise by ConfigureAwait(false)). That means you can't assume the awaiting code will run on your thread if you can't control the awaiting code.
There are a few reasons for you continuations not to be inlined on the same thread. One is whether TaskCreationOptions.RunContinuationsAsynchronously is used. Others like the current SynchronizationContext and TaskScheduler are in IsValidLocationForInlining:
internal static bool IsValidLocationForInlining
{
get
{
// If there's a SynchronizationContext, we'll be conservative and say
// this is a bad location to inline.
var ctx = SynchronizationContext.CurrentNoFlow;
if (ctx != null && ctx.GetType() != typeof(SynchronizationContext)) return false;
// Similarly, if there's a non-default TaskScheduler, we'll be conservative
// and say this is a bad location to inline.
var sched = TaskScheduler.InternalCurrent;
return sched == null || sched == TaskScheduler.Default;
}
}

Await stops execution of thread and never continues

I have the following:
public async Task<bool> SearchForUpdatesAsync()
{
return await TaskEx.Run(() =>
{
if (!ConnectionChecker.IsConnectionAvailable())
return false;
// Check for SSL and ignore it
ServicePointManager.ServerCertificateValidationCallback += delegate { return (true); };
var configurations = UpdateConfiguration.Download(UpdateConfigurationFileUri, Proxy);
var result = new UpdateResult(configurations, CurrentVersion,
IncludeAlpha, IncludeBeta);
if (!result.UpdatesFound)
return false;
_updateConfigurations = result.NewestConfigurations;
double updatePackageSize = 0;
foreach (var updateConfiguration in _updateConfigurations)
{
var newPackageSize = GetUpdatePackageSize(updateConfiguration.UpdatePackageUri);
if (newPackageSize == null)
throw new SizeCalculationException(_lp.PackageSizeCalculationExceptionText);
updatePackageSize += newPackageSize.Value;
_packageOperations.Add(new UpdateVersion(updateConfiguration.LiteralVersion),
updateConfiguration.Operations);
}
TotalSize = updatePackageSize;
return true;
});
}
As you can see I'm using Microsoft.Bcl.
Now in my other class I wrote this code in a normal void:
TaskEx.Run(async delegate
{
// ...
_updateAvailable = await _updateManager.SearchForUpdatesAsync();
MessageBox.Show("Test");
});
The problem I have is that it executes _updateAvailable = await _updateManager.SearchForUpdatesAsync(); and then it doesn't continue the thread, it just stops as if there is nothing after that call. Visual Studio also tells me this after a while: Thread ... exited with code 259, so something seems to be still alive.
I debugged through it to search for any exceptions that could maybe be swallowed, but nothing, everything works fine and it executes the return-statement.
And that is what I don't understand, I never see the MessageBox and/or no code beyond this line's being executed.
After I talked to some friends, they confirmed that this shouldn't be. Did I make a horrible mistake when implementing async-await?
Thanks in advance, that's actually all I can say about that, I got no more information, I appreciate any tips and help as far as it's possible.

The main issue that you're having is that you're unnecessarily wrapping your method in TaskEx.Run() and you are probably experiencing deadlock somewhere.
Your method signature is currently:
public async Task<bool> SearchForUpdatesAsync()
This means the following:
async --> Doesn't actually do anything, but provides a "heads up" that await might be called within this method and that this method can be used as a runnable Task.
Task --> This method returns a runnable task that can be run asynchronously on the threadpool
<bool> --> This method actually returns bool when awaited.
The await TaskEx.Run() is unnecessarily since this says run this method and then don't return until after a value is available. This is most likely causing a synchronization problem. Removing this construct will make your method work properly, however you'll notice that now you have no reason to even include the async operator or the Task<T> portion since the method is actually synchronous anyway. Usually you're only going to use async identifier on the method signature if you have methods that you are going to call await on them.
Instead you have two options.
Whenever you want to call SearchForUpdates() you can wrap this in a Task<bool>.Run() to run it asynchronously (or the Bcl equivalent)
Since you are using WinForms you might be better off using a BackgroundWorker and just calling this method within it.
Regarding using the async-await pattern I think that this is a great article to use to make sure you're following best practices: https://msdn.microsoft.com/en-us/magazine/jj991977.aspx

The best practice is to have async all the way through your layers, and then call await or less desirably .Wait() / .Result at the final use site.
Also, try to keep your UI calls separate from the backend work, since you can run into synchronicity/thread-context issue.
public class WinFormsCode
{
private async Task WinsFormCodeBehindMethodAsync()
{
var updatesAvailable = await _updateManager.SearchForUpdatesAsync();
MessageBox.Show("Updates Available: " + updatesAvailable.ToString());
}
private void WinsFormCodeBehindMethodSync()
{
var updatesAvailable = _updateManager.SearchForUpdatesAsync().Result;
MessageBox.Show("Updates Available: " + updatesAvailable.ToString());
}
}
public class UpdateManager
{
public async Task<bool> SearchForUpdatesAsync()
{
return true;
}
}

Async API call inside an actor and exceptions

I know about PipeTo, but some stuff, like synchronous waiting on nested continuation, seems to go against the async & await way.
So, my first question [1] would be: is there any 'magic' here, so that we can just synchronously wait for nested tasks in a continuation and it's still async in the end?
While we're at async & await differences, how are failures handled?
Let's create a simple example:
public static class AsyncOperations
{
public async static Task<int> CalculateAnswerAsync()
{
await Task.Delay(1000).ConfigureAwait(false);
throw new InvalidOperationException("Testing!");
//return 42;
}
public async static Task<string> ConvertAsync(int number)
{
await Task.Delay(600).ConfigureAwait(false);
return number + " :)";
}
}
In a 'regular', async & await way:
var answer = await AsyncOperations.CalculateAnswerAsync();
var converted = await AsyncOperations.ConvertAsync(answer);
the exception will bubble up from the first operation, just as you'd expect.
Now, let's create an actor that's going to work with those async operations. For the sake of an argument, let's say that CalculateAnswerAsync and ConvertAsync should be used one after another as one, full operation (similar to, for example, StreamWriter.WriteLineAsync and StreamWriter.FlushAsync if you just want to write one line to a stream).
public sealed class AsyncTestActor : ReceiveActor
{
public sealed class Start
{
}
public sealed class OperationResult
{
private readonly string message;
public OperationResult(string message)
{
this.message = message;
}
public string Message
{
get { return message; }
}
}
public AsyncTestActor()
{
Receive<Start>(msg =>
{
AsyncOperations.CalculateAnswerAsync()
.ContinueWith(result =>
{
var number = result.Result;
var conversionTask = AsyncOperations.ConvertAsync(number);
conversionTask.Wait(1500);
return new OperationResult(conversionTask.Result);
})
.PipeTo(Self);
});
Receive<OperationResult>(msg => Console.WriteLine("Got " + msg.Message));
}
}
If there are no exceptions, I still get Got 42 :) without any issues, which brings me back to 'magic' point above [1].
Also, are the AttachedToParent and ExecuteSynchronously flags provided in an example optional, or are they pretty much required to have everything working as intended? They don't seem to have any effect on exception handling...
Now, if the CalculateAnswerAsync throws an exception, which means that result.Result throws AggregateException, it's pretty much swallowed without a trace.
What should I do here, if it's even possible, to make the exception inside an asynchronous operation crash the actor as a 'regular' exception would?

The joys of error-handling in the TPL :)
Once a Task starts running on its own thread, everything that happens inside it is already asynchronous from the caller - including error-handling
When you kick off your first Task inside of an actor, that task runs independently on the ThreadPool from your actor. This means that anything you do inside that Task will already be asynchronous from your actor - because it's running on a different thread. This is why I made a Task.Wait call inside the PipeTo sample you linked to at the top of your post. Makes no difference to the actor - it just looks like a long-running task.
Exceptions - if your inner task failed, the conversionTask.Result property will throw the exception captured during its run, so you'll want to add some error-handling inside your Task to ensure that your actor gets notified that something went wrong. Notice I did just that here: https://github.com/petabridge/akkadotnet-code-samples/blob/master/PipeTo/src/PipeTo.App/Actors/HttpDownloaderActor.cs#L117 - if you turn your Exceptions into messages your actor can handle: birds start singing, rainbows shine, and TPL errors stop being a source of pain and agony.
As for what happens when an exception gets thrown...
Now, if the CalculateAnswerAsync throws an exception, which means that
result.Result throws AggregateException, it's pretty much swallowed
without a trace.
The AggregateException will contain the list of inner exceptions wrapped inside of it - the reason the TPL has this concept of aggregate errors is in the event that (a) you have one task that is the continuation of multiple tasks in aggregate, i.e. Task.WhenAll or (b) you have errors propagated up the ContinueWith chain back to the parent. You can also call the AggregateException.Flatten() call to make it a little easier to manage nested exceptions.
Best Practices for TPL + Akka.NET
Dealing with Exceptions from the TPL is a nuisance, that's true - but the best way to deal with it is to try..catch.. exceptions inside your Task and turn them into message classes your actor can handle.
Also, are the AttachedToParent and ExecuteSynchronously flags provided in an example optional, or are they pretty much required to have everything working as intended?
This is mostly an issue for when you have continuations on continuations - PipeTo automatically uses these flags on itself. It has zero impact on error handling, but ensures that your continuations are executed immediately on the same thread as the original Task.
I recommend using these flags only when you're doing a lot of nested continuations - the TPL starts to take some liberties with how it schedules your tasks once you go deeper than 1 continuation (and in fact, flags like OnlyOnCompleted stop being accepted after more than 1 continuation.)

Just to add to what Aaron said.
As of yesterday, we do support safe async await inside actors when using the Task dispatcher.
public class AsyncAwaitActor : ReceiveActor
{
public AsyncAwaitActor()
{
Receive<string>(async m =>
{
await Task.Delay(TimeSpan.FromSeconds(1));
Sender.Tell("done");
});
}
}
public class AskerActor : ReceiveActor
{
public AskerActor(ActorRef other)
{
Receive<string>(async m =>
{
var res = await other.Ask(m);
Sender.Tell(res);
});
}
}
public class ActorAsyncAwaitSpec : AkkaSpec
{
[Fact]
public async Task Actors_should_be_able_to_async_await_ask_message_loop()
{
var actor = Sys.ActorOf(Props.Create<AsyncAwaitActor>()
.WithDispatcher("akka.actor.task-dispatcher"),
"Worker");
//IMPORTANT: you must use the akka.actor.task-dispatcher
//otherwise async await is not safe
var asker = Sys.ActorOf(Props.Create(() => new AskerActor(actor))
.WithDispatcher("akka.actor.task-dispatcher"),
"Asker");
var res = await asker.Ask("something");
Assert.Equal("done", res);
}
}
This is not our default dispatcher since it does come with a price in performance/throughput.
There is also a risk of deadlocks if you trigger tasks that block(e.g. using task.Wait() or task.Result)
So the PipeTo pattern is still the preferred approach since it is more true to the actor model.
But the async await support is there for you as an extra tool if you really need to do some TPL integration.
This feature actually uses PipeTo under the covers.
It will take every task continuation and wrap that up in a special message and pass that message back to the actor and execute that task inside the actors own concurrency context.

Calling async methods from a synchronous context

I'm calling a service over HTTP (ultimately using the HttpClient.SendAsync method) from within my code. This code is then called into from a WebAPI controller action. Mostly, it works fine (tests pass) but then when I deploy on say IIS, I experience a deadlock because caller of the async method call has been blocked and the continuation cannot proceed on that thread until it finishes (which it won't).
While I could make most of my methods async I don't feel as if I have a basic understanding of when I'd must do this.
For example, let's say I did make most of my methods async (since they ultimately call other async service methods) how would I then invoke the first async method of my program if I built say a message loop where I want some control of the degree of parallelism?
Since the HttpClient doesn't have any synchronous methods, what can I safely presume to do if I have an abstraction that isn't async aware? I've read about the ConfigureAwait(false) but I don't really understand what it does. It's strange to me that it's set after the async invocation. To me that feels as if a race waiting to happen... however unlikely...
WebAPI example:
public HttpResponseMessage Get()
{
var userContext = contextService.GetUserContext(); // <-- synchronous
return ...
}
// Some IUserContextService implementation
public IUserContext GetUserContext()
{
var httpClient = new HttpClient();
var result = httpClient.GetAsync(...).Result; // <-- I really don't care if this is asynchronous or not
return new HttpUserContext(result);
}
Message loop example:
var mq = new MessageQueue();
// we then run say 8 tasks that do this
for (;;)
{
var m = mq.Get();
var c = GetCommand(m);
c.InvokeAsync().Wait();
m.Delete();
}
When you have a message loop that allow things to happen in parallel and you have asynchronous methods, there's a opportunity to minimize latency. Basically, what I want to accomplish in this instance is to minimize latency and idle time. Though I'm actually unsure as to how to invoke into the command that's associated with the message that arrives off the queue.
To be more specific, if the command invocation needs to do service requests there's going to be latency in the invocation that could be used to get the next message. Stuff like that. I can totally do this simply by wrapping up things in queues and coordinating this myself but I'd like to see this work with just some async/await stuff.

While I could make most of my methods async I don't feel as if I have a basic understanding of when I'd must do this.
Start at the lowest level. It sounds like you've already got a start, but if you're looking for more at the lowest level, then the rule of thumb is anything I/O-based should be made async (e.g., HttpClient).
Then it's a matter of repeating the async infection. You want to use async methods, so you call them with await. So that method must be async. So all of its callers must use await, so they must also be async, etc.
how would I then invoke the first async method of my program if I built say a message loop where I want some control of the degree of parallelism?
It's easiest to put the framework in charge of this. E.g., you can just return a Task<T> from a WebAPI action, and the framework understands that. Similarly, UI applications have a message loop built-in that async will work naturally with.
If you have a situation where the framework doesn't understand Task or have a built-in message loop (usually a Console application or a Win32 service), you can use the AsyncContext type in my AsyncEx library. AsyncContext just installs a "main loop" (that is compatible with async) onto the current thread.
Since the HttpClient doesn't have any synchronous methods, what can I safely presume to do if I have an abstraction that isn't async aware?
The correct approach is to change the abstraction. Do not attempt to block on asynchronous code; I describe that common deadlock scenario in detail on my blog.
You change the abstraction by making it async-friendly. For example, change IUserContext IUserContextService.GetUserContext() to Task<IUserContext> IUserContextService.GetUserContextAsync().
I've read about the ConfigureAwait(false) but I don't really understand what it does. It's strange to me that it's set after the async invocation.
You may find my async intro helpful. I won't say much more about ConfigureAwait in this answer because I think it's not directly applicable to a good solution for this question (but I'm not saying it's bad; it actually should be used unless you can't use it).
Just bear in mind that async is an operator with precedence rules and all that. It feels magical at first, but it's really not so much. This code:
var result = await httpClient.GetAsync(url).ConfigureAwait(false);
is exactly the same as this code:
var asyncOperation = httpClient.GetAsync(url).ConfigureAwait(false);
var result = await asyncOperation;
There are usually no race conditions in async code because - even though the method is asynchronous - it is also sequential. The method can be paused at an await, and it will not be resumed until that await completes.
When you have a message loop that allow things to happen in parallel and you have asynchronous methods, there's a opportunity to minimize latency.
This is the second time you've mentioned a "message loop" "in parallel", but I think what you actually want is to have multiple (asynchronous) consumers working off the same queue, correct? That's easy enough to do with async (note that there is just a single message loop on a single thread in this example; when everything is async, that's usually all you need):
await tasks.WhenAll(ConsumerAsync(), ConsumerAsync(), ConsumerAsync());
async Task ConsumerAsync()
{
for (;;) // TODO: consider a CancellationToken for orderly shutdown
{
var m = await mq.ReceiveAsync();
var c = GetCommand(m);
await c.InvokeAsync();
m.Delete();
}
}
// Extension method
public static Task<Message> ReceiveAsync(this MessageQueue mq)
{
return Task<Message>.Factory.FromAsync(mq.BeginReceive, mq.EndReceive, null);
}
You'd probably also be interested in TPL Dataflow. Dataflow is a library that understands and works well with async code, and has nice parallel options built-in.

While I appreciate the insight from community members it's always difficult to express the intent of what I'm trying to do but tremendously helpful to get advice about circumstances surrounding the problem. With that, I eventually arrived that the following code.
public class AsyncOperatingContext
{
struct Continuation
{
private readonly SendOrPostCallback d;
private readonly object state;
public Continuation(SendOrPostCallback d, object state)
{
this.d = d;
this.state = state;
}
public void Run()
{
d(state);
}
}
class BlockingSynchronizationContext : SynchronizationContext
{
readonly BlockingCollection<Continuation> _workQueue;
public BlockingSynchronizationContext(BlockingCollection<Continuation> workQueue)
{
_workQueue = workQueue;
}
public override void Post(SendOrPostCallback d, object state)
{
_workQueue.TryAdd(new Continuation(d, state));
}
}
/// <summary>
/// Gets the recommended max degree of parallelism. (Your main program message loop could use this value.)
/// </summary>
public static int MaxDegreeOfParallelism { get { return Environment.ProcessorCount; } }
#region Helper methods
/// <summary>
/// Run an async task. This method will block execution (and use the calling thread as a worker thread) until the async task has completed.
/// </summary>
public static T Run<T>(Func<Task<T>> main, int degreeOfParallelism = 1)
{
var asyncOperatingContext = new AsyncOperatingContext();
asyncOperatingContext.DegreeOfParallelism = degreeOfParallelism;
return asyncOperatingContext.RunMain(main);
}
/// <summary>
/// Run an async task. This method will block execution (and use the calling thread as a worker thread) until the async task has completed.
/// </summary>
public static void Run(Func<Task> main, int degreeOfParallelism = 1)
{
var asyncOperatingContext = new AsyncOperatingContext();
asyncOperatingContext.DegreeOfParallelism = degreeOfParallelism;
asyncOperatingContext.RunMain(main);
}
#endregion
private readonly BlockingCollection<Continuation> _workQueue;
public int DegreeOfParallelism { get; set; }
public AsyncOperatingContext()
{
_workQueue = new BlockingCollection<Continuation>();
}
/// <summary>
/// Initialize the current thread's SynchronizationContext so that work is scheduled to run through this AsyncOperatingContext.
/// </summary>
protected void InitializeSynchronizationContext()
{
SynchronizationContext.SetSynchronizationContext(new BlockingSynchronizationContext(_workQueue));
}
protected void RunMessageLoop()
{
while (!_workQueue.IsCompleted)
{
Continuation continuation;
if (_workQueue.TryTake(out continuation, Timeout.Infinite))
{
continuation.Run();
}
}
}
protected T RunMain<T>(Func<Task<T>> main)
{
var degreeOfParallelism = DegreeOfParallelism;
if (!((1 <= degreeOfParallelism) & (degreeOfParallelism <= 5000))) // sanity check
{
throw new ArgumentOutOfRangeException("DegreeOfParallelism must be between 1 and 5000.", "DegreeOfParallelism");
}
var currentSynchronizationContext = SynchronizationContext.Current;
InitializeSynchronizationContext(); // must set SynchronizationContext before main() task is scheduled
var mainTask = main(); // schedule "main" task
mainTask.ContinueWith(task => _workQueue.CompleteAdding());
// for single threading we don't need worker threads so we don't use any
// otherwise (for increased parallelism) we simply launch X worker threads
if (degreeOfParallelism > 1)
{
for (int i = 1; i < degreeOfParallelism; i++)
{
ThreadPool.QueueUserWorkItem(_ => {
// do we really need to restore the SynchronizationContext here as well?
InitializeSynchronizationContext();
RunMessageLoop();
});
}
}
RunMessageLoop();
SynchronizationContext.SetSynchronizationContext(currentSynchronizationContext); // restore
return mainTask.Result;
}
protected void RunMain(Func<Task> main)
{
// The return value doesn't matter here
RunMain(async () => { await main(); return 0; });
}
}
This class is complete and it does a couple of things that I found difficult to grasp.
As general advice you should allow the TAP (task-based asynchronous) pattern to propagate through your code. This may imply quite a bit of refactoring (or redesign). Ideally you should be allowed to break this up into pieces and make progress as you work towards to overall goal of making your program more asynchronous.
Something that's inherently dangerous to do is to call asynchronous code callously in an synchronous fashion. By this we mean invoking the Wait or Result methods. These can lead to deadlocks. One way to work around something like that is to use the AsyncOperatingContext.Run method. It will use the current thread to run a message loop until the asynchronous call is complete. It will swap out whatever SynchronizationContext is associated with the current thread temporarily to do so.
Note: I don't know if this is enough, or if you are allowed to swap back the SynchronizationContext this way, assuming that you can, this should work. I've already been bitten by the ASP.NET deadlock issue and this could possibly function as a workaround.
Lastly, I found myself asking the question, what is the corresponding equivalent of Main(string[]) in an async context? Turns out that's the message loop.
What I've found is that there are two things that make out this async machinery.
SynchronizationContext.Post and the message loop. In my AsyncOperatingContext I provide a very simple message loop:
protected void RunMessageLoop()
{
while (!_workQueue.IsCompleted)
{
Continuation continuation;
if (_workQueue.TryTake(out continuation, Timeout.Infinite))
{
continuation.Run();
}
}
}
My SynchronizationContext.Post thus becomes:
public override void Post(SendOrPostCallback d, object state)
{
_workQueue.TryAdd(new Continuation(d, state));
}
And our entry point, basically the equivalent of an async main from synchronous context (simplified version from original source):
SynchronizationContext.SetSynchronizationContext(new BlockingSynchronizationContext(_workQueue));
var mainTask = main(); // schedule "main" task
mainTask.ContinueWith(task => _workQueue.CompleteAdding());
RunMessageLoop();
return mainTask.Result;
All of this is costly and we can't just go replace calls to async methods with this but it does allow us to rather quickly create the facilities required to keep writing async code where needed without having to deal with the whole program. It's also very clear from this implementation where the worker threads go and how the impact concurrency of your program.
I look at this and think to myself, yeap, that's how Node.js does it. Though JavaScript does not have this nice async/await language support that C# currently does.
As an added bonus, I have complete control of the degree of parallelism, and if I want, I can run my async tasks completely single threaded. Though, If I do so and call Wait or Result on any task, it will deadlock the program because it will block the only message loop available.

Where is the return statment for the Task object?

The following code compiles and runs well. But where is the return statement for the Consumer() and Producer() methods?
class Program
{
static BufferBlock<Int32> m_buffer = new BufferBlock<int>(
new DataflowBlockOptions { BoundedCapacity = 10 });
public static async Task Producer() <----- How is a Task object returned?
{
while (true)
{
await m_buffer.SendAsync<Int32>(DateTime.Now.Second);
Thread.Sleep(1000);
}
}
public static async Task Consumer() <----- How is a Task object returned?
{
while (true)
{
Int32 n = await m_buffer.ReceiveAsync<Int32>();
Console.WriteLine(n);
}
}
static void Main(string[] args)
{
Task.WaitAll(Consumer(), Producer());
}
}

While your question states the obvious - the code compiles - and the other answers try to explain-by-example, I think the answer is best described in the following two articles:
"Above-the-surface" answer - full article is here: http://msdn.microsoft.com/en-us/magazine/hh456401.aspx
[...] C# and Visual Basic [...] giving enough hints to the compilers
to build the necessary mechanisms for you behind the scenes. The
solution has two parts: one in the type system, and one in the
language.
The CLR 4 release defined the type Task [...] to represent the
concept of “some work that’s going to produce a result of type T in
the future.” The concept of “work that will complete in the future but
returns no result” is represented by the non-generic Task type.
Precisely how the result of type T is going to be produced in the
future is an implementation detail of a particular task; [...]
The language half of the solution is the new await keyword. A regular
method call means “remember what you’re doing, run this method until
it’s completely finished, and then pick up where you left off, now
knowing the result of the method.” An await expression, in contrast,
means “evaluate this expression to obtain an object representing work
that will in the future produce a result. Sign up the remainder of the
current method as the callback associated with the continuation of
that task. Once the task is produced and the callback is signed up,
immediately return control to my caller.”
2.The under-the-hood explanation is found here: http://msdn.microsoft.com/en-us/magazine/hh456403.aspx
[...] Visual Basic and C# [...] let you express discontinuous
sequential code. [...] When the Visual Basic or C# compiler gets hold
of an asynchronous method, it mangles it quite a bit during
compilation: the discontinuity of the method is not directly supported
by the underlying runtime and must be emulated by the compiler. So
instead of you having to pull the method apart into bits, the compiler
does it for you. [...]
The compiler turns your asynchronous method into a statemachine. The
state machine keeps track of where you are in the execution and what
your local state is. [...]
Asynchronous methods produce Tasks. More specifically, an asynchronous
method returns an instance of one of the types Task or Task from
System.Threading.Tasks, and that instance is automatically generated.
It doesn’t have to be (and can’t be) supplied by the user code. [...]
From the compiler’s point of view, producing Tasks is the easy part.
It relies on a framework-supplied notion of a Task builder, found in
System.Runtime.CompilerServices [...] The builder lets the compiler
obtain a Task, and then lets it complete the Task with a result or an
Exception. [...] Task builders are special helper types meant only for
compiler consumption. [...]
[...] build up a state machine around the production and consumption
of the Tasks. Essentially, all the user logic from the original method
is put into the resumption delegate, but the declarations of locals
are lifted out so they can survive multiple invocations. Furthermore,
a state variable is introduced to track how far things have gotten,
and the user logic in the resumption delegate is wrapped in a big
switch that looks at the state and jumps to a corresponding label. So
whenever resumption is called, it will jump right back to where it
left off the last time.

when the method has no return statement its return type is Task. Have a look at the MSDN page of Async Await
Async methods have three possible return types: Task<TResult>, Task, void
if you need a Task you must specify the return statement. Taken from the MSDN page:
// TASK<T> EXAMPLE
async Task<int> TaskOfT_MethodAsync()
{
// The body of the method is expected to contain an awaited asynchronous
// call.
// Task.FromResult is a placeholder for actual work that returns a string.
var today = await Task.FromResult<string>(DateTime.Now.DayOfWeek.ToString());
// The method then can process the result in some way.
int leisureHours;
if (today.First() == 'S')
leisureHours = 16;
else
leisureHours = 5;
// Because the return statement specifies an operand of type int, the
// method must have a return type of Task<int>.
return leisureHours;
}

async keyword kind of tells compiler that the method body should be used as a Task body. In simple way we can say that these are equivalents to your examples:
public static Task Producer() <----- How is a Task object returned?
{
return Task.Run(() =>
{
while (true)
{
m_buffer.SendAsync<Int32>(DateTime.Now.Second).Wait();
Thread.Sleep(1000);
}
});
}
public static Task Consumer() <----- How is a Task object returned?
{
return Task.Run(() =>
{
while (true)
{
Int32 n = m_buffer.ReceiveAsync<Int32>().Wait();
Console.WriteLine(n);
}
});
}
Of course the compiled result of your methods will be completely different from my examples, because compiler smart enough and it can generate code in such way, so some of the lines in your method will be invoked on the context (thread), which calls the method and some of them on the background context. And this is actually why Thread.Sleep(1000); is not recommended in the body of async methods, because this code can be invoked on the thread, which calls this method. Task has equivalent which can replace Thread.Sleep(1000); the equivalent await Task.Delay(1000), which will be invoked on background thread. As you can see await keyword guaranties you that this call will be invoked on the background context and will not block the caller context.
Let's take a look on one more example:
async Task Test1()
{
int a = 0; // Will be called on the called thread.
int b = await this.GetValueAsync(); // Will be called on background thread
int c = GetC(); // Method execution will come back to the called thread again on this line.
int d = await this.GetValueAsync(); // Going again to background thread
}
So we can say that this will be generated code:
Task Test1()
{
int a = 0; // Will be called on the called thread.
vat syncContext = Task.GetCurrentSynchronizationContext(); // This will help us go back to called thread
return Task.Run(() =>
{
// We already on background task, so we safe here to wait tasks
var bTask = this.GetValueAsync();
bTask.Wait();
int b = bTask.Result;
// syncContext helps us to invoke something on main thread
// because 'int c = 1;' probably was expected to be called on
// the caller thread
var cTask = Task.Run(() => return GetC(), syncContext);
cTask.Wait();
int c = cTask.Result;
// This one was with 'await' - calling without syncContext,
// not on the thread, which calls our method.
var dTask = this.GetValueAsync();
dTask.Wait();
int d = dTask.Result;
});
}
Again, this is not the same code which you will get from compiler, but it should just give you some idea how this works. If you really want to take a look on what will be in the produced library, use for example IlSpy to take a look on generated code.
Also I really recommend to read this article Best Practices in Asynchronous Programming

That's exactly what the async keyword means. It takes a method and (usually) converts it to a Task returning method. If the normal method would have the return type of void, the async method will have Task. If the return type would be some other T, the new return type will be Task<T>.
For example, to download an process some data synchronously, you could write something like:
Foo GetFoo()
{
string s = DownloadFoo();
Foo foo = ParseFoo(s);
return foo;
}
The asynchronous version would then look like this:
Task<Foo> GetFoo()
{
string s = await DownloadFoo();
Foo foo = ParseFoo(s);
return foo;
}
Notice that the return type changed from Foo to Task<Foo>, but you're still returning just Foo. And similarly with void-returning methods: since they don't have to contain a return statement, async Task (without any <T>) methods also don't.
The Task object is constructed by the compiler-generated code, so you don't have to do that.