We Keep Coding

REST-Call with async server implementation

In my application I have a server which provides a REST-Api where my UI can communicate with.
Now I have to start a long running Process on the Server but I want the client not to wait for the response of the server.
I know I could just fire and forget the Post-Call and not await the response but I need to know at the client that the Process on the server was startet correctly.
So I thought about the following:
[HttpPost]
[Route("startscan")]
public HttpResponseMessage StartScan()
{
Task.Factory.StartNew( () =>
{
//Do long running things here
});
return Request.CreateResponse(HttpStatusCode.OK);
}
So my question now is: Will the task I started be executed to it's end? Background of the question is, that as far as I knwo a controller-instance is created for each call to it. So will the instance of the controller be terminated when the requested finished or will the Task I started run to it's end. The task can take up to 10 minutes or longer sometimes.

A simple approach would be to just use an asynchronous method without awaiting the Task result like so:
[HttpPost]
[Route("startscan")]
public async HttpResponseMessage StartScan()
{
DoLongRunningThings();
return Request.CreateResponse(HttpStatusCode.OK);
}
public async Task DoLongRunningThings()
{
// Do Stuff here
}
However, if you Processing is more complex and requires more resilience you should look into how to use background jobs. Here is a good collection of what you can use: https://stackoverflow.com/a/36098218/16154479

Will the task I started be executed to it's end?
Probably yes, but also possibly no. Sometimes it won't be executed completely. This is the result of misusing an ASP.NET app (which handles HTTP requests) as a background service (which runs outside of a request context).
There is a best practice that avoids possibly-partial execution: a distributed architecture (as I describe on my blog). The idea is that your controller enqueues a message describing the work to be done into a durable queue and then returns. Then there's a separate background service that reads from the queue and does the actual work.

Deadlock when calling an async method inside a non async action

We have a web application which consists of a Web API consumed by our UI. The web application is installed on different servers (tests before installing it by our clients).
On one server, we get a deadlock by the following endpoint:
[HttpGet, Route(Order = 1)]
public IHttpActionResult GetUserDetails(long userKey)
{
var userService = ResolveService<IUserService>();
var nameService = ResolveService<INameService >();
LoggInfo("Start userService.Get()"); //logged in logs
var result = userService.Get(userKey);
this.LoggInfo($"End userService.Get() with id = "{result.Id}); // logged in logs
try
{
LoggInfo($"Start nameService.Get()"); // logged in logs
result.Name = nameService.Get(result.Namekey).Result?.Name;
LoggInfo($"End nameService.Get()"); // not logged in logs
}
catch(Exception ex)
{
LogError("An error occured in NameService"); // not logged in logs
}
return Ok(result);
}
nameService.Get(id) is an async method :
public async Task<NameDTO> GetAsync(long key)
{
LogInfo("start NameService.Get()"); // not logged in logs
var query = GetQuery();
var name = await query.Select(MapName())
.FirstOrDefaultAsync(n => n.Key == key);
return name;
}
When I remove the async signature everything is working as expected, according to this article it is normal to have a deadlock.
Could you please explain me why this works in the other servers ?
Thanks in advance for your help

Could you please explain me why this works in the other servers?
In order for the deadlock to occur, there are three necessary parts:
A one-thread-at-a-time context, which all servers would have (the pre-Core ASP.NET context).
An await (without ConfigureAwait(false)) that acts asynchronously.
Code elsewhere that blocks a thread inside that context, waiting for the async method to finish. All servers also have this.
Most likely, the difference in behavior is due to the second part: an await that acts asynchronously. await first examines its awaitable (e.g., the Task passed to it), and if that awaitable is complete, then it continues executing the async method synchronously.
Specifically, this would happen if the Task<NameDTO> returned from GetAsync completed before the calling code hit the .Result. This would have to be extremely fast, but it's possible depending on caching, how fast your network hop is, how much the code is slowed down by load/neighbors/anti-virus, etc.

Wait for a third-party API callback

I need to create an REST API that connect to a third party SOAP API. The third party API events are sent by callback to an URL I provide.
The typical steps my API go through is it starts a session with the third party by providing an ID and an callback URL. The third party can now send new events to my API through this URL when, for example, a new participant connects. Now sometimes i need to request specific info, like the list of participants for a given session(ID), and wait for the event containing the info.
Note that there may be multiple open sessions at the same time.
An example of what I need:
private string url = "http://myapi/callback";
[HttpGet]
[Route("createSession")]
public async Task<string> CreateSession()
{
var id = Guid.NewGuid().ToString();
var result = await ExternAPI.CreateSession(id, this.url);
return result; //contains the id
}
[HttpGet]
[Route("endSession")]
public async Task<string> EndSession([FromUri] string id)
{
var result = await ExternAPI.EndSession(id);
return result;
}
[HttpGet]
[Route("partipants")]
public async Task<string> Partipants([FromUri] string id)
{
ExternAPI.participants(id); // The results of this method will be sent to the callback function
results = // Wait for the results for this id
return results;
}
[HttpPost]
[Route("callback")]
public void Callback(body)
{
// notify waiting function and pass body
}
I came up with a solution using ReactiveX but I'm not really sure about its reliability in production. What I have in mind is to create a subject that never terminate and handle all the events but it is not a usual lifetime for a subject, what happens on error ? And I don't think I did it the "RX-way" (state concerns).
Here it is (you will need System.Reactive to run this code):
class Data
{
public int id;
public string value;
}
class Program
{
private static Subject<Data> sub;
static void Main(string[] args)
{
sub = new Subject<Data>();
Task.Run(async () => {
int id = 1;
ExternAPI(CallBackHook, id);
Data result = await sub.Where(data => data.id == id).FirstAsync();
Console.WriteLine("{0}", result.value);
});
Console.ReadLine();
}
static void CallBackHook(Data data)
{
sub.OnNext(data);
}
static String ExternAPI(Action<Data> callback, int id)
{
// Third-party API, access via SOAP. callback is normally an url (string)
Task.Run(() =>
{
Thread.Sleep(1000);
callback(new Data { id = id, value = "test" });
});
return "success";
}
}
An other way will be a dictionary of subjects, one for each session, so I could manage their lifetimes.

it is not a usual lifetime for a subject
what happens on error?
And I don't think I did it the "RX-way"
Yes, these are all perfectly valid concerns with this kind of approach. Personally, I don't much mind the last one, because even though Subjects are frowned-upon, many times they're just plain easier to use than the proper Rx way. With the learning curve of Rx what it is, I tend to optimize for developer maintainability, so I do "cheat" and use Subjects unless the alternative is equally understandable.
Regarding lifetime and errors, the solutions there depend on how you want your application to behave.
For lifetime, it looks like currently you have a WebAPI resource (the SOAP connection) requiring an explicit disconnect call from your client; this raises some red flags. At the very least, you'd want some kind of timeout there where that resource is disposed even if endSession is never called. Otherwise, it'll be all too easy to end up with dangling resources.
Also for errors, you'll need to decide the appropriate approach. You could "cache" the error and report it to each call that tries to use that resource, and "clear" the error when endSession is called. Or, if it's more appropriate, you could let an error take down your ASP.NET process. (ASP.NET will restart a new one for you).
To delay an API until you get some other event, use TaskCompletionSource<T>. When starting the SOAP call (e.g., ExternAPI.participants), you should create a new TCS<T>. The API call should then await the TaskCompletionSource<T>.Task. When the SOAP service responds with an event, it should take that TaskCompletionSource<T> and complete it. Points of note:
If you have multiple SOAP calls that are expecting responses over the same event, you'll need a collection of TaskCompletionSource<T> instances, along with some kind of message-identifier to match up which events are for which calls.
Be sure to watch your thread safety. Incoming SOAP events are most likely arriving on the thread pool, with (possibly multiple) API requests on other thread pool threads. TaskCompletionSource<T> itself is threadsafe, but you'd need to make your collection threadsafe as well.
You may want to write a Task-based wrapper for your SOAP service first (handling all the TaskCompletionSource<T> stuff), and then consume that from your WebAPI.
As a very broad alternative, instead of bridging SOAP with WebAPI, I would consider bridging SOAP with SignalR. You may find that this is a more natural translation. Among other things, SignalR will give you client-connect and client-disconnect events (complete with built-in timeouts for clients). So that may solve your lifetime issues more naturally. You can use the same Task-based wrapper for your SOAP service as well, or just expose the SOAP events directly as SignalR messages.

Async does not work in asynchronous controller mvc 4.0

I have MVC 4.0 application targated at targetFramework="4.5".
I have to basically convert the existing functionality of file processing from synchronous to asynchronous (so that for large file user don't have to wait for other task).
My code is
[HttpPost]
public async Task<ActionResult> FileUpload(HttpPostedFileBase fileUpload)
{
Coreservice objVDS = new Coreservice ();
//validate the contents of the file
model =objVDS. ValidateFileContents(fileUpload);
// if file is valid start processing asynchronously
await Task.Factory.StartNew(() => { objVDS.ProcessValidFile(model); }, CancellationToken.None,
TaskCreationOptions.DenyChildAttach,
TaskScheduler.FromCurrentSynchronizationContext());
return view();
}
Basically I want to call a asynchronous method which is in services which does database operations( diffrent project).
I want asynchronous process to have access to the context in services methods. Thats why I am using
TaskScheduler.FromCurrentSynchronizationContext() in Task.Factory.StartNew().
The service method is like following in which, based on file type, a second service is getting called for data operations
public async task ProcessValidFile(fileProcessDataModel model)
{
employeeWorkedDataservice service =new employeeWorkedDataservice()
await Task.Factory.StartNew(() =>
{
service .ProcessEmployeeDataFile(model.DataSetToProcess, OriginalFileName, this, model.Source);
},
CancellationToken.None,
TaskCreationOptions.DenyChildAttach,
TaskScheduler.FromCurrentSynchronizationContext());
}
ProcessEmployeeDataFile returns void and its not asynchronous method.
When the code above is executed it does not return to controller untill it completes the data processing. I think that I am missing something here.
Please guide me to solution.
Thanks,
Amol

Looks like you've misunderstood how await works.
Read this https://msdn.microsoft.com/en-us/library/hh191443.aspx#BKMK_WhatHappensUnderstandinganAsyncMethod
Setting something running in a task will allow it to run asynchronously so you can do something else while it's running.
When you need the result to continue, you use the await keyword.
By creating your task an immediately awaiting it, you're instantly blocking until the task resolves; making it effectively synchronous.
If you're happy to return to your view without waiting for processing to complete, I don't think you need await at all, since at no point do you want to wait for the result of the operation.
public task ProcessValidFile(fileProcessDataModel model)
{
employeeWorkedDataservice service =new employeeWorkedDataservice()
return Task.Factory.StartNew(() =>
{
service.ProcessEmployeeDataFile(model.DataSetToProcess, OriginalFileName, this, model.Source);
},
CancellationToken.None,
TaskCreationOptions.DenyChildAttach,
TaskScheduler.FromCurrentSynchronizationContext());
}
[HttpPost]
public ActionResult FileUpload(HttpPostedFileBase fileUpload)
{
Coreservice objVDS = new Coreservice ();
//validate the contents of the file
model =objVDS. ValidateFileContents(fileUpload);
// if file is valid start processing asynchronously
// This returns a task, but if we're not interested in waiting
// for its results, we can ignore it.
objVDS.ProcessValidFile(model);
return view();
}
Regarding your comments:
I would seriously consider not passing your controller to your service, or having your service rely on the session and context since you're tightly coupling your business logic to your API controller.
Get the bits you need from the controller while you're in it and pass them to your service.

I have to basically convert the existing functionality of file processing from synchronous to asynchronous (so that for large file user don't have to wait for other task).
That's not what async does; as I describe on my blog, async does not change the HTTP protocol.
What you want is some form of "fire and forget" on ASP.NET. I have another blog post that covers a few solutions. Note that using Task.Factory.StartNew is the most dangerous of all these solutions.
The best (read: most reliable) solution is to use a proper distributed architecture: your ASP.NET app should create a description of the work to be done and place that in a reliable queue (e.g., MSMQ); then have an independent backend (e.g., Win32 service) that processes the queue. This is complex, but much less error-prone than attempting to force ASP.NET to do something it was never meant to do.

WCF suspended call

I have a WCF service that implements long-polling. However, I see no way to have each service call spawn a new thread upon being called.
As it stands, the long-polled contract is waiting for an event to occur and is blocking any other contracts from being called.
What is the recommended way to have one contract run asynchronously from another contract in WCF?
I thought of keeping a static thread pool but I'm not quite sure if that solution scales.
Thanks!

In the context of your question, I assume long-polling is some kind of an operation which is periodically issuing an HTTP request to a 3rd party resource, until a desired response has been returned, or until timed out.
To implement it efficiently, you can use .NET 4.5, TAP pattern and async/await.
Example (untested):
// contract
[ServiceContract]
public interface IService
{
//task-based asynchronous pattern
[OperationContract]
Task<bool> DoLongPollingAsync(string url, int delay, int timeout);
}
// implementation
public class Service : IService
{
public async Task<bool> DoLongPollingAsync(
string url, int delay, int timeout)
{
// handle timeout via CancellationTokenSource
using (var cts = new CancellationTokenSource(timeout))
using (var httpClient = new System.Net.Http.HttpClient())
using (cts.Token.Register(() => httpClient.CancelPendingRequests()))
{
try
{
while (true)
{
// do the polling iteration
var data = await httpClient.GetStringAsync(url).ConfigureAwait(false);
if (data == "END POLLING") // should we end polling?
return true;
// pause before the next polling iteration
await Task.Delay(delay, cts.Token);
}
}
catch (OperationCanceledException)
{
// is cancelled due to timeout?
if (!cts.IsCancellationRequested)
throw;
}
// timed out
throw new TimeoutException();
}
}
}
This scales well, because most of the time the DoLongPolling is in the pending state, asynchronously awaiting the result of HttpClient.GetStringAsync or Task.Delay calls. Which means it doesn't block a thread from ThreadPool, so the WCF service can serve more DoLongPolling requests concurrently. Check "There Is No Thread" by Stephen Cleary for more details on how this works behind the scene.
On the client side, you can call your WCF service asynchronously, too. Tick "Allow generation of asynchronous operations" when you create the WCF service proxy and select "Generate task-based operations".
If you need to target .NET 4.0, you can find some alternative options in "Asynchronous Operations in WCF" by Jaliya Udagedara.