We Keep Coding

Calling async method from sync method is ASP .NET web api - issue

I have a ASP .NET web api 2 application, and I'm trying to call asnyc method from sync method and meanwhile I encountered some issues. If I just call the method as a regular method the body after delay doesn't get executed, if I call it with Task.Run() it gets executed
public void CallingMethod(MethodExecutionArgs args)
{
//do something with the args
System.Diagnostics.Debug.WriteLine("BEFORE ");
WriteFileToDiskAsync(args); // If I run only this, I never see "WriteFile() - AFTER delay" in the output
Task.Run(async () => await WriteFileToDiskAsync(args)); // this executes the entire async method
System.Diagnostics.Debug.WriteLine($"Finally written from sync method");
}
private async Task<bool> WriteFileToDiskAsync(dynamic file)
{
System.Diagnostics.Debug.WriteLine("Before delay inside async");
await Task.Delay(3000);
System.Diagnostics.Debug.WriteLine("WriteFile() - AFTER delay");
}
Result: I always see the lines written from CallingMethod, but when I call the async method like a regular method, I only see the "Before delay inside async", If I call it with Task.Run() it see both lines from the async method.
Question1: Can someone explain this behavior, why doesn't the async method execute fully?
Does it have something to do with what Stephen Cleary says: If you lose your AppDomain for any reason, that in-progress work is lost.
I read these articles but can't figure out why is this happening:
How to call asynchronous method from synchronous method in C#?
Fire and forget async method in asp.net mvc
C# async/await with/without awaiting (fire and forget)
https://blog.stephencleary.com/2012/12/returning-early-from-aspnet-requests.html According to most articles what I'm trying to do is not recommended (among other things the exceptions from the async method would get swallowed) but in my case I already have the information that I need to generate the response and what is done in the Async method doesn't concern the client at all...
Context: What I'm trying to achieve is, on a existing API route that creates a X Resource (saves it in a database), after the X resource is created, I want to asynchronously call a method that will Create a json file with some information from that X Resource and save it in a filesystem. But even if the writing of the file fails, I want to return a successful response to the client (because his request was to actually save the X Resource - which succeeded) he doesn't care about Creating external file in XYZ filesystem.
Question2: What would you say is the best practice to achieve what I described above, considering all the existing methods chained in the CreateResource route are sync?

Question1: Can someone explain this behavior, why doesn't the async method execute fully? Does it have something to do with what Stephen Cleary says: If you lose your AppDomain for any reason, that in-progress work is lost.
No. The AppDomain is still there in this case. In ASP.NET (pre-Core), there is an ASP.NET SynchronizationContext that represents the request. await by default captures the current context and uses that to resume executing its async method.
So, when the await completes, it attempts to resume executing on the SynchronizationContext for that request. However, that request has been completed, and so it doesn't make sense to resume on that context. The resulting behavior is undefined.
Question2: What would you say is the best practice to achieve what I described above, considering all the existing methods chained in the CreateResource route are sync?
If you are sure you want to return early, then you should have a background service handle creating the file, not the ASP.NET service. Since you have a database already, you could use the Outbox Pattern, where you place messages in an "outbox" table as part of the same transaction as your resource creation. Then a separate background service reads the "outbox" table and writes the JSON file.

Deep understanding of async / await on ASP.NET MVC

I don't understand exactly what is going on behind the scenes when I have an async action on an MVC controller especially when dealing with I/O operations. Let's say I have an upload action:
public async Task<ActionResult> Upload (HttpPostedFileBase file) {
....
await ReadFile(file);
...
}
From what I know these are the basic steps that happen:
A new thread is peeked from threadpool and assigned to handle incomming request.
When await gets hit, if the call is an I/O operation then the original thread gets back into pool and the control is transfered to a so-called IOCP (Input output completion port). What I do not understand is why the request is still alive and waits for an answer because in the end the calling client will wait for our request to complete.
My question is: Who / when / how does this wait for complete blocking occurs?
Note: I saw the blog post There Is No Thread, and it makes sense for GUI applications, but for this server side scenario I don't get it. Really.

There's some good resources on the 'net that do describe this in detail. I wrote an MSDN article that describes this at a high level.
What i do not understand is why the request is still alive and waits for an answer because in the end the calling client will wait for our request to complete.
It's still alive because the ASP.NET runtime has not yet completed it. Completing the request (by sending the response) is an explicit action; it's not like the request will complete on its own. When ASP.NET sees that the controller action returns a Task/Task<T>, it will not complete the request until that task completes.
My question is: Who / when / how does this wait for complete blocking occurs ?
Nothing is waiting.
Think of it this way: ASP.NET has a collection of current requests that it's processing. For a given request, as soon as it's complete, the response is sent out and then that request is removed from the collection.
The key is that it's a collection of requests, not threads. Each of those requests may or may not have a thread working on it at any point in time. Synchronous requests always have a single thread (the same thread). Asynchronous requests may have periods when they don't have threads.
Note: i saw this thread: http://blog.stephencleary.com/2013/11/there-is-no-thread.html and it makes sense for GUI applications but for this server side scenario I don't get it.
The threadless approach to I/O works exactly the same for ASP.NET apps as it does for GUI apps.
Eventually, the file write will complete, which (eventually) completes the task returned from ReadFile. This "completing of the task" work is normally done with a thread pool thread. Since the task is now complete, the Upload action will continue executing, causing that thread to enter the request context (that is, there is now a thread executing that request again). When the Upload method is complete, then the task returned from Upload is complete, and ASP.NET writes out the response and removes the request from its collection.

Under the hood the compiler performs a sleight of hand and transforms your async \ await code into a Task-based code with a callback. In the most simple case:
public async Task X()
{
A();
await B();
C();
}
Gets changed into something like:
public Task X()
{
A();
return B().ContinueWith(()=>{ C(); })
}
So there's no magic - just a lot of Tasks and callbacks. For more complex code the transformations will be more complex too, but in the end the resulting code will be logically equivalent to what you wrote. If you want, you can take one of ILSpy/Reflector/JustDecompile and see for yourself what is compiled "under the hood".
ASP.NET MVC infrastructure in turn is intelligent enough to recognize if your action method is a normal one, or a Task based one, and alter its behavior in turn. Therefore the request doesn't "disappear".
One common misconception is that everything with async spawns another thread. In fact, it's mostly the opposite. At the end of the long chain of the async Task methods there is normally a method which performs some asynchronous IO operation (such as reading from disk or communicating via network), which is a magical thing performed by Windows itself. For the duration of this operation, there is no thread associated with the code at all - it's effectively halted. After the operation completes however, Windows calls back and then a thread from the thread pool is assigned to continue the execution. There's a bit of framework code involved to preserve the HttpContext of the request, but that's all.

The ASP.NET runtime understands what tasks are and delays sending the HTTP response until the task is done. In fact the Task.Result value is needed in order to even generate a response.
The runtime basically does this:
var t = Upload(...);
t.ContinueWith(_ => SendResponse(t));
So when your await is hit both your code and the runtimes code gets off the stack and "there is no thread" at that point. The ContinueWith callback revives the request and sends the response.

Asynchronous DB-Query to trigger Stored Procedure

I want to performa an asynchronous DB Query in C# that calls a stored procedure for a Backup. Since we use Azure this takes about 2 minutes and we don't want the user to wait that long.
So the idea is to make it asynchronous, so that the task continues to run, after the request.
[HttpPost]
public ActionResult Create(Snapshot snapshot)
{
db.Database.CommandTimeout = 7200;
Task.Run(() => db.Database.ExecuteSqlCommandAsync("EXEC PerformSnapshot #User = '" + CurrentUser.AccountName + "', #Comment = '" + snapshot.Comment + "';"));
this.ShowUserMessage("Your snapshot has been created.");
return this.RedirectToActionImpl("Index", "Snapshots", new System.Web.Routing.RouteValueDictionary());
}
I'm afraid that I haven't understood the concept of asynchronous taks. The query will not be executed (or aborted?), if I don't use the wait statement. But actually "waiting" is the one thing I espacially don't want to do here.
So... why am I forced to use wait here?
Or will the method be started, but killed if the requst is finished?

We don't want the user to wait that long.
async-await won't help you with that. Odd as it may sound, the basic async-await pattern is about implementing synchronous behavior in a non-blocking fashion. It doesn't re-arrange your logical flow; in fact, it goes to great lengths to preserve it. The only thing you've changed by going async here is that you're no longer tying up a thread during that 2-minute database operation, which is a huge win your app's scalability if you have lots of concurrent users, but doesn't speed up an individual request one bit.
I think what you really want is to run the operation as a background job so you can respond to the user immediately. But be careful - there are bad ways to do that in ASP.NET (i.e. Task.Run) and there are good ways.

Dave, you're not forced to use await here. And you're right - from user perspective it still will take 2 minutes. The only difference is that the thread which processes your request can now process other requests meanwhile database does its job. And when database finishes, the thread will continue process your request.
Say you have limited number of threads capable to process HTTP request. This async code will help you to process more requests per time period, but it won't help user to get the job done faster.

This seems to be down to a misunderstanding as to what async and await do.
async does not mean run this on a new thread, in essence it acts as a signal to the compiler to build a state machine, so a method like this:
Task<int> GetMeAnInt()
{
return await myWebService.GetMeAnInt();
}
sort of (cannot stress this enough), gets turned into this:
Task<int> GetMeAnInt()
{
var awaiter = myWebService.GetMeAnInt().GetAwaiter();
awaiter.OnCompletion(() => goto done);
return Task.InProgress;
done:
return awaiter.Result;
}
MSDN has way more information about this, and there's even some code out there explaining how to build your own awaiters.
async and await at their very core just enable you to write code that uses callbacks under the hood, but in a nice way that tells the compiler to do the heavy lifting for you.
If you really want to run something in the background, then you need to use Task:
Task<int> GetMeAnInt()
{
return Task.Run(() => myWebService.GetMeAnInt());
}
OR
Task<int> GetMeAnInt()
{
return Task.Run(async () => await myWebService.GetMeAnInt());
}
The second example uses async and await in the lambda because in this scenario GetMeAnInt on the web service also happens to return Task<int>.
To recap:
async and await just instruct the compiler to do some jiggerypokery
This uses labels and callbacks with goto
Fun fact, this is valid IL but the C# compiler doesn't allow it for your own code, hence why the compiler can get away with the magic but you can't.
async does not mean "run on a background thread"
Task.Run() can be used to queue a threadpool thread to run an arbitrary function
Task.Factory.Start() can be used to grab a brand new thread to run an arbitrary function
await instructs the compiler that this is the point at which the result of the awaiter for the awaitable (e.g. Task) being awaited is required - this is how it knows how to structure the state machine.

As I describe in my MSDN article on async ASP.NET, async is not a silver bullet; it doesn't change the HTTP protocol:
When some developers learn about async and await, they believe it’s a way for the server code to “yield” to the client (for example, the browser). However, async and await on ASP.NET only “yield” to the ASP.NET runtime; the HTTP protocol remains unchanged, and you still have only one response per request.
In your case, you're trying to use a web request to kick off a backend operation and then return to the browser. ASP.NET was not designed to execute backend operations like this; it is only a web tier framework. Having ASP.NET execute work is dangerous because ASP.NET is only aware of work coming in from its requests.
I have an overview of various solutions on my blog. Note that using a plain Task.Run, Task.Factory.StartNew, or ThreadPool.QueueUserWorkItem is extremely dangerous because ASP.NET doesn't know anything about that work. At the very least you should use HostingEnvironment.QueueBackgroundWorkItem so ASP.NET at least knows about the work. But that doesn't guarantee that the work will actually ever complete.
A proper solution is to place the work in a persistent queue and have an independent background worker process that queue. See the Asynchronous Messaging Primer (specifically, your scenario is "Decoupling workloads").

How can I verify a deadlock in my async to sync method and HttpClient.PostAsync calls?

I'm having a deadlock in my program but I can't figure out why and how to fix it.
Short version:
An async to sync method locks waiting for an async call, which calls another, and so on until a HttpClient.PostAsync call. The http post call never happens, at least it doesn't arrives in the server (which I own too). No errors in the client. My main questions are at the end of this SO question. Thanks!
Longer version:
I know that it is very easy to come up with deadlocks and we must be careful. I'm trying to adhere to the advices of 1. Propagate the async down to all the methods and 2. Use ConfigureAwait(false) whenever possible/make sense. (I think the main advocate of this is Stephen Cleary as his blog and answers in many SO which I have read many of them).
Well, my program...
I cannot post a full sample code here unfortunately, it's rather long; I describe it as a sync method A that calls an async method B, which calls another async, which calls another async, etc.
Eventually the last async call is HttpClient.PostAsync.
Program stop's at the await HttpClient.PostAsync. Also, the Http post never fires (I own the target server and in this case it never receives a request). (I double checked the uri etc.).
As a result, the synchronous method locks waiting for B's async task to complete which never happens. A looks like this:
public Byte[] A()
{
var task = BAsync();
return task.GetAwaiter().GetResult();
}
I use synchronization this way as he method does not need the context back, and I saw the method suggested by Stephen in Nito is implemented using this. (I also tried task.Result instead with same results).
The code of the http post call is:
private static async Task<HttpResponseMessage> PostAsync(Uri serverUri, HttpContent httpContent)
{
var client = new HttpClient();
return await client.PostAsync(serverUri, httpContent).ConfigureAwait(false);
}
The app's UI freezes. It's a WP8 app. In this aspect, the buttons do not react to Click events but ScrollViews still work. This is topic for another question but I prefer to include this detail in the question.
I'm not even sure I have a deadlock, or maybe is a HttpClient bug? (I don't think so, the same method is working in other workflows in the same app).
I tried to verify for deadlock and used VS's threads debugger window but only the main thread seems to be active. I also tried with all other sort of debugger windows but nothing gave me a hint.
My main questions are:
Am I having a deadlock?
How can I verify, maybe using VS, that I do have a deadlock?
Is it at Client.PostAsync? If yes why and how to fix it?
If not, any ideas please.
Edit: Method A gets called using reflection with MethodInfo.Invoke. I believe now this is part of the problem.

I believe you are seeing a deadlock, though I'm not sure how to prove it.
HttpClient on most platforms properly uses ConfigureAwait(false) internally, but on a few platforms does not. I believe that WP8 is one of those that does not. This means that you can use ConfigureAwait(false) all you want correctly, but HttpClient can still cause a deadlock.
The best solution is to go async all the way. But if that's not possible, then two alternatives come to mind:
Push the HttpClient work off to a background thread.
public Byte[] A()
{
var task = Task.Run(() => BAsync());
return task.GetAwaiter().GetResult();
}
Establish a nested loop on the foreground thread to execute any asynchronous work.
public Byte[] A()
{
return AsyncContext.Run(() => BAsync());
}
Note that AsyncContext is from my AsyncEx library. I believe AsyncContext should work on WP8 but I haven't actually used it on that platform.

MVC4 + async/await + return response before action completes

In my MVC4 app I need to add a controller for uploading and processing large files. Immediately after the file is uploaded I need to start async processing of that file and return response to the browser without waiting for the processing to complete.
Obviously I could start a new thread for processing the file manually, but I'm wondering if I can implement this scenario using async/await mechanism introduced with .net 4.5
To test the concept I've tried something like this:
public async Task<ActionResult> Test()
{
TestAsync();
return View("Test");
}
public async void TestAsync()
{
await LongRunning();
}
private Task<int> LongRunning()
{
return Task<int>.Factory.StartNew(() => Pause());
}
private int Pause()
{
Thread.Sleep(10000);
return 3;
}
The async mechanism seems to work in general: when I debug the code I hit the "return View("Test");" line before the line "return 3". However, the browser receives the response only after the Pause method completes.
This seems to behave like regular async controllers (the ones with Async and Completed methods). Is there a way to use async/await in controllers for my scenario?

Obviously I could start a new thread for processing the file manually, but I'm wondering if I can implement this scenario using async/await mechanism introduced with .net 4.5
No, you cannot, because async doesn't change the HTTP protocol.
Svick and James have already posted the correct answers as comments, which I duplicate below for convenience:
IIS can recycle your application pretty much at any time.
If you have long-running things to do async to the request, do them elsewhere. The 'typical' is a persistent queue (MSMQ, Azure, RabbitMQ, etc) with something else (windows service, exe run by task scheduler, app using Quartz.net, etc) processing them.
To summarize, HTTP gives you one request and one response (async - and anything else - won't change this).
ASP.NET is designed around HTTP requests (and makes assumptions like "if there are no outstanding requests, then it's safe to stop this web site"). You can kick off a new thread and keep your upload in memory (which is the easiest way to do it), but it's strongly not recommended.
For your situation, I recommend you follow James' suggestion:
When your upload completes, save the upload to persistent storage (e.g., an Azure queue), and return a "ticket" to the browser.
Have some other process (e.g., an Azure worker role) process the queue, eventually marking it as complete.
Have the browser poll the server with its "ticket". The server uses the "ticket" to look up the file in persistent storage and return whether or not it's complete.
There are some variations to this (e.g., using SignalR to notify the browser when processing is complete), but the general architecture is the same.
This is complex, but it's the right way to do it.

You have an error in your code.
You must await the call to TestAsync in your action.
If you don't, it simply returns a "Task" object without running anything.
public async Task<ActionResult> Test()
{
await TestAsync();
return View("Test");
}
and the correct way to sleep in an async method is to call
await Task.Delay(10000);
You have to change the signature of TestAsync: you should not mark a method async if it returns void. It must return Task instead. Returning void is reserved for compatibility with .net events.
public async Task TestAsync()
{
await LongRunning();
}
The method body is the same.

Your LongRunning method is synchronously sleeping 10 seconds. Change it so the sleep happens in the task instead.