So I've been digging up on the implementation of HttpClient.SendAsync via Reflector. What I intentionally wanted to find out was the flow of execution of these methods, and to determine which API gets called to execute the asynchronous IO work.
After exploring the various classes inside HttpClient, I saw that internally it uses HttpClientHandler which derives from HttpMessageHandler and implements its SendAsync method.
This is the implementation of HttpClientHandler.SendAsync:
protected internal override Task<HttpResponseMessage> SendAsync(HttpRequestMessage request, CancellationToken cancellationToken)
{
if (request == null)
{
throw new ArgumentNullException("request", SR.net_http_handler_norequest);
}
this.CheckDisposed();
this.SetOperationStarted();
TaskCompletionSource<HttpResponseMessage> source = new TaskCompletionSource<HttpResponseMessage>();
RequestState state = new RequestState
{
tcs = source,
cancellationToken = cancellationToken,
requestMessage = request
};
try
{
HttpWebRequest request2 = this.CreateAndPrepareWebRequest(request);
state.webRequest = request2;
cancellationToken.Register(onCancel, request2);
if (ExecutionContext.IsFlowSuppressed())
{
IWebProxy proxy = null;
if (this.useProxy)
{
proxy = this.proxy ?? WebRequest.DefaultWebProxy;
}
if ((this.UseDefaultCredentials || (this.Credentials != null)) || ((proxy != null) && (proxy.Credentials != null)))
{
this.SafeCaptureIdenity(state);
}
}
Task.Factory.StartNew(this.startRequest, state);
}
catch (Exception exception)
{
this.HandleAsyncException(state, exception);
}
return source.Task;
}
What I found weird is that the above uses Task.Factory.StartNew to execute the request while generating a TaskCompletionSource<HttpResponseMessage> and returning the Task created by it.
Why do I find this weird? well, we go on alot about how I/O bound async operations have no need for extra threads behind the scenes, and how its all about overlapped IO.
Why is this using Task.Factory.StartNew to fire an async I/O operation? this means that SendAsync isn't only using pure async control flow to execute this method, but spinning a ThreadPool thread "behind our back" to execute its work.
this.startRequest is a delegate that points to StartRequest which in turn uses HttpWebRequest.BeginGetResponse to start async IO. HttpClient is using async IO under the covers, just wrapped in a thread-pool Task.
That said, note the following comment in SendAsync
// BeginGetResponse/BeginGetRequestStream have a lot of setup work to do before becoming async
// (proxy, dns, connection pooling, etc). Run these on a separate thread.
// Do not provide a cancellation token; if this helper task could be canceled before starting then
// nobody would complete the tcs.
Task.Factory.StartNew(startRequest, state);
This works around a well-known problem with HttpWebRequest: Some of its processing stages are synchronous. That is a flaw in that API. HttpClient is avoiding blocking by moving that DNS work to the thread-pool.
Is that good or bad? It is good because it makes HttpClient non-blocking and suitable for use in a UI. It is bad because we are now using a thread for long-running blocking work although we expected to not use threads at all. This reduces the benefits of using async IO.
Actually, this is a nice example of mixing sync and async IO. There is nothing inherently wrong with using both. HttpClient and HttpWebRequest are using async IO for long-running blocking work (the HTTP request). They are using threads for short-running work (DNS, ...). That's not a bad pattern in general. We are avoiding most blocking and we only have to make a small part of the code async. A typical 80-20 trade-off. It is not good to find such things in the BCL (a library) but in application level code that can be a very smart trade-off.
It seems it would have been preferable to fix HttpWebRequest. Maybe that is not possible for compatibility reasons.
Related
We have a third-party method Foo which sometimes runs in a deadlock for unknown reasons.
We are executing an single-threaded tcp-server and call this method every 30 seconds to check that the external system is available.
To mitigate the problem with the deadlock in the third party code we put the ping-call in a Task.Run to so that the server does not deadlock.
Like
async Task<bool> WrappedFoo()
{
var timeout = 10000;
var task = Task.Run(() => ThirdPartyCode.Foo());
var delay = Task.Delay(timeout);
if (delay == await Task.WhenAny(delay, task ))
{
return false;
}
else
{
return await task ;
}
}
But this (in our opinion) has the potential to starve the application of free threads. Since if one call to ThirdPartyCode.Foo deadlock the thread will never recover from this deadlock and if this happens often enough we might run out of resources.
Is there a general approach how one should handle deadlocking third-party code?
A CancellationToken won't work because the third-party-api does not provide any cancellation options.
Update:
The method at hand is from the SAPNCO.dll provided by SAP to establish and test rfc-connections to a sap-system, therefore the method is not a simple network-ping. I renamed the method in the question to avoid further misunderstandings
Is there a general approach how one should handle deadlocking third-party code?
Yes, but it's not easy or simple.
The problem with misbehaving code is that it can not only leak resources (e.g., threads), but it can also indefinitely hold onto important resources (e.g., some internal "handle" or "lock").
The only way to forcefully reclaim threads and other resources is to end the process. The OS is used to cleaning up misbehaving processes and is very good at it. So, the solution here is to start a child process to do the API call. Your main application can communicate with its child process by redirected stdin/stdout, and if the child process ever times out, the main application can terminate it and restart it.
This is, unfortunately, the only reliable way to cancel uncancelable code.
Cancelling a task is a collaborative operation in that you pass a CancellationToken to the desired method and externally you use CancellationTokenSource.Cancel:
public void Caller()
{
try
{
CancellationTokenSource cts=new CancellationTokenSource();
Task longRunning= Task.Run(()=>CancellableThirdParty(cts.Token),cts.Token);
Thread.Sleep(3000); //or condition /signal
cts.Cancel();
}catch(OperationCancelledException ex)
{
//treat somehow
}
}
public void CancellableThirdParty(CancellationToken token)
{
while(true)
{
// token.ThrowIfCancellationRequested() -- if you don't treat the cancellation here
if(token.IsCancellationRequested)
{
// code to treat the cancellation signal
//throw new OperationCancelledException($"[Reason]");
}
}
}
As you can see in the code above , in order to cancel an ongoing task , the method running inside it must be structured around the CancellationToken.IsCancellationRequested flag or simply CancellationToken.ThrowIfCancellationRequested method ,
so that the caller just issues the CancellationTokenSource.Cancel.
Unfortunately if the third party code is not designed around CancellationToken ( it does not accept a CancellationToken parameter ), then there is not much you can do.
Your code isn't cancelling the blocked operation. Use a CancellationTokenSource and pass a cancellation token to Task.Run instead :
var cts=new CancellationTokenSource(timeout);
try
{
await Task.Run(() => ThirdPartyCode.Ping(),cts.Token);
return true;
}
catch(TaskCancelledException)
{
return false;
}
It's quite possible that blocking is caused due to networking or DNS issues, not actual deadlock.
That still wastes a thread waiting for a network operation to complete. You could use .NET's own Ping.SendPingAsync to ping asynchronously and specify a timeout:
var ping=new Ping();
var reply=await ping.SendPingAsync(ip,timeout);
return reply.Status==IPStatus.Success;
The PingReply class contains far more detailed information than a simple success/failure. The Status property alone differentiates between routing problems, unreachable destinations, time outs etc
I'm trying to understand how and when to use async programming and got to I/O bound operations, but I don't understand them. I want to implement them from scratch. How can I do that?
Consider the example below which is synchronous:
private void DownloadBigImage() {
var url = "https://cosmos-magazine.imgix.net/file/spina/photo/14402/180322-Steve-Full.jpg";
new WebClient().DownloadFile(url, "image.jpg");
}
How do I implement the async version by only having the normal synchronous method DownloadBigImage without using Task.Run since that will use a thread from the thread pool only for waiting - that's just being wasteful!
Also do not use the special method that's already async! This is the purpose of this question: how do I make it myself without relying on methods which are already async? So, NO things like:
await new WebClient().DownloadFileTaskAsync(url, "image.jpg");
Examples and documentation available are very lacking in this regard. I found only this:
https://learn.microsoft.com/en-us/dotnet/standard/async-in-depth
which says:
The call to GetStringAsync() calls through lower-level .NET libraries (perhaps calling other async methods) until it reaches a P/Invoke interop call into a native networking library. The native library may subsequently call into a System API call (such as write() to a socket on Linux). A task object will be created at the native/managed boundary, possibly using TaskCompletionSource. The task object will be passed up through the layers, possibly operated on or directly returned, eventually returned to the initial caller.
Basically I have to use a "P/Invoke interop call into a native networking library"... but how?
This is a great question which really isn't explained well in most texts about C# and async.
I searched for this for ages thinking I could and should maybe be implementing my own async I/O methods. If a method/library I was using didn't have async methods I thought I should somehow wrap these functions in code that made them asynchronous. It turns out that this isn't really feasible for most programmers. Yes, you can spawn a new thread using Thread.Start(() => {...}) and that does make your code asynchronous, but it also creates a new thread which is an expensive overhead for asynchronous operations. It can certainly free up your UI thread to ensure your app stays responsive, but it doesn't create a truly async operation the way that HttpClient.GetAsync() is a truly asynchronous operation.
This is because async methods in the .net libraries use something called "standard P/Invoke asynchronous I/O system in .NET" to call low level OS code that doesn't require a dedicated CPU thread while doing outbound IO (networking or storage). It actually doesn't dedicate a thread to its work and signals the .net runtime when it's done doing its stuff.
I'm not familiar with the details but this knowledge is enough to free me from trying to implement async I/O and make me focus on using the async methods already present in the .net libraries (such as HttpClient.GetAsync()). More interesting info can be found here (Microsoft async deep dive) and a nice description by Stephen Cleary here
I think this is a very interesting question and a fun learning exercise.
Fundamentally, you cannot use any existing API that is synchronous. Once it's synchronous there is no way to turn it truly asynchronous. You correctly identified that Task.Run and it's equivalents are not a solution.
If you refuse to call any async .NET API then you need to use PInvoke to call native APIs. This means that you need to call the WinHTTP API or use sockets directly. This is possible but I don't have the experience to guide you.
Rather, you can use async managed sockets to implement an async HTTP download.
Start with the synchronous code (this is a raw sketch):
using (var s = new Socket(...))
{
s.Connect(...);
s.Send(GetHttpRequestBytes());
var response = new StreamReader(new NetworkStream(s)).ReadToEnd();
}
This very roughly gets you an HTTP response as a string.
You can easily make this truly async by using await.
using (var s = new Socket(...))
{
await s.ConnectAsync(...);
await s.SendAsync(GetHttpRequestBytes());
var response = await new StreamReader(new NetworkStream(s)).ReadToEndAsync();
}
If you consider await cheating with respect to your exercise goals you would need to write this using callbacks. This is awful so I'm just going to write the connect part:
var s = new Socket(...)
s.BeginConnect(..., ar => {
//perform next steps here
}, null);
Again, this code is very raw but it shows the principle. Instead of waiting for an IO to complete (which happens implicitly inside of Connect) you register a callback that is called when the IO is done. That way your main thread continues to run. This turns your code into spaghetti.
You need to write safe disposal with callbacks. This is a problem because exception handling cannot span callbacks. Also, you likely need to write a read loop if you don't want to rely on the framework to do that. Async loops can be mind bending.
TLDR: Generally you can by using TaskCompletionSource.
If you only have blocking calls available then you cannot do this. But usually there are "old" asynchronous methods that do not use async nor Task, but rely instead on callbacks. In that case you can use a TaskCompletionSource to both create a Task that can be returned, and use it to set the Task to completed when the callback returns.
Example using the old .Net Framework 3.0 methods in WebClient (but programmed in later .Net that has Task):
public Task DownloadCallbackToAsync(string url, string filename)
{
using (var client = new WebClient())
{
TaskCompletionSource taskCreator = new TaskCompletionSource();
client.DownloadFileCompleted += (sender, args) => taskCreator.SetResult();
client.DownloadFileAsync(url, filename);
return taskCreator.Task;
}
}
Here you will imidiately initiate the call and return a Task. If you await the Task in the calling method you will not continue until the callback (DownloadFileCompleted) has occurred.
Note that this method itself is not async as it does not need to await a Task.
Create a new task that executes the synchronous code. The task will be executed by a thread of the threadpool.
private async Task DownloadBigImage()
{
await Task.Run(()=>
{
var url = "https://cosmos-magazine.imgix.net/file/spina/photo/14402/180322-Steve-Full.jpg";
new WebClient().DownloadFile(url, "image.jpg");
});
}
The MSDN documentation appears to state that async and await are suitable for IO-bound tasks whereas Task.Run should be used for CPU-bound tasks.
I'm working on an application that performs HTTP requests to retrieve HTML documents, which it then parses. I have a method that looks like this:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync())
return await Task.Run(() => LoadHtmlDocument(contentStream)); //CPU-bound
}
Is this good and suitable use of async and await, or am I over-using it?
There are two good answers already, but to add my 0.02...
If you're talking about consuming asynchronous operations, async/await works excellently for both I/O-bound and CPU-bound.
I think the MSDN docs do have a slight slant towards producing asynchronous operations, in which case you do want to use TaskCompletionSource (or similar) for I/O-bound and Task.Run (or similar) for CPU-bound. Once you've created the initial Task wrapper, it's best consumed by async and await.
For your particular example, it really comes down to how much time LoadHtmlDocument will take. If you remove the Task.Run, you will execute it within the same context that calls LoadPage (possibly on a UI thread). The Windows 8 guidelines specify that any operation taking more than 50ms should be made async... keeping in mind that 50ms on your developer machine may be longer on a client's machine...
So if you can guarantee that LoadHtmlDocument will run for less than 50ms, you can just execute it directly:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync()) //IO-bound
return LoadHtmlDocument(contentStream); //CPU-bound
}
However, I would recommend ConfigureAwait as #svick mentioned:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)
.ConfigureAwait(continueOnCapturedContext: false)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync()
.ConfigureAwait(continueOnCapturedContext: false)) //IO-bound
return LoadHtmlDocument(contentStream); //CPU-bound
}
With ConfigureAwait, if the HTTP request doesn't complete immediately (synchronously), then this will (in this case) cause LoadHtmlDocument to be executed on a thread pool thread without an explicit call to Task.Run.
If you're interested in async performance at this level, you should check out Stephen Toub's video and MSDN article on the subject. He has tons of useful information.
It is appropriate to await any operation that is asynchronous (i.e. is represented by a Task).
The key point is that for IO operations, whenever possible, you want to use a provided method that is, at it's very core, asynchronous, rather than using Task.Run on a blocking synchronous method. If you're blocking a thread (even a thread pool thread) while performing IO, you're not leveraging the real power of the await model.
Once you have created a Task that represents your operation you no longer care if it's CPU or IO bound. To the caller it's just some async operation that needs to be await-ed.
There are several things to consider:
In a GUI application, you want as little code as possible to execute on the UI thread. In that case, offloading CPU-bound operation to another thread using Task.Run() is probably a good idea. Though the users of your code can do that themselves, if they want.
In something like ASP.NET application, there is no UI thread and all you care about is performance. In that case, there is some overhead in using Task.Run() instead of running the code directly, but it shouldn't be significant if the operation actually takes some time. (Also, there is some overhead in returning to the synchronization context, which is one more reason why you should use ConfigureAwait(false) for most awaits in your library code.)
If your method is async (which BTW should be also reflected in the name of the method, not just its return type), people will expect that it won't block the synchronization context thread, even for CPU-bound work.
Weighting that, I think using await Task.Run() is the right choice here. It does have some overhead, but also some advantages, which can be significant.
The implementation of the ExecuteNonQueryAsync() method in System.Data.SqlClient.SqlCommand is as follows:
public override Task<int> ExecuteNonQueryAsync(CancellationToken cancellationToken) {
Bid.CorrelationTrace("<sc.SqlCommand.ExecuteNonQueryAsync|API|Correlation> ObjectID%d#, ActivityID %ls\n", ObjectID);
SqlConnection.ExecutePermission.Demand();
TaskCompletionSource<int> source = new TaskCompletionSource<int>();
CancellationTokenRegistration registration = new CancellationTokenRegistration();
if (cancellationToken.CanBeCanceled) {
if (cancellationToken.IsCancellationRequested) {
source.SetCanceled();
return source.Task;
}
registration = cancellationToken.Register(CancelIgnoreFailure);
}
Task<int> returnedTask = source.Task;
try {
RegisterForConnectionCloseNotification(ref returnedTask);
Task<int>.Factory.FromAsync(BeginExecuteNonQueryAsync, EndExecuteNonQueryAsync, null).ContinueWith((t) => {
registration.Dispose();
if (t.IsFaulted) {
Exception e = t.Exception.InnerException;
source.SetException(e);
}
else {
if (t.IsCanceled) {
source.SetCanceled();
}
else {
source.SetResult(t.Result);
}
}
}, TaskScheduler.Default);
}
catch (Exception e) {
source.SetException(e);
}
return returnedTask;
}
Which I would summarize as:
Create TaskCompletionSource<int> source = new TaskCompletionSource<int>();
Create a new task using Task<int>.Factory.FromAsync, using the APM "Begin/End" API
Invoke source.SetResult() when the task finishes.
Return source.Task
What is the point of using TaskCompletionSource here and why not to return the task created by Task<int>.Factory.FromAsync() directly? This task also has the result and exception (if any) wrapped.
In C# in a Nutshell book, in the Asynchronous Programming and Continuations section, it states:
In writing Delay, we
used TaskCompletionSource, which is a standard way to implement “bottom-level”
I/O-bound asynchronous methods.
For compute-bound methods, we use Task.Run to initiate thread-bound concurrency.
Simply by returning the task to the caller, we create an asynchronous method.
Why is it that the compute-bound methods can be implemented using Task.Run(), but not the I/O bound methods?
Note that for a definitive answer, you would have to ask the author of the code. Barring that, we can only speculate. However, I think it's reasonable to make some inferences with reasonable accuracy…
What is the point of using TaskCompletionSource here and why not to return the task created by Task.Factory.FromAsync() directly?
In this case, it appears to me that the main reason is to allow the implementation to deregister the registered callback CancelIgnoreFailure() before the task is actually completed. This ensures that by the time the client code receives completion notification, the API itself has completely cleaned up from the operation.
A secondary reason might be simply to provide a complete abstraction. I.e. to not allow any of the underlying implementation to "leak" from the method, in the form of a Task object that a caller might inspect or (worse) manipulate in a way that interferes with the correct and reliable operation of the task.
Why is it that the compute-bound methods can be implemented using Task.Run(), but not the I/O bound methods?
You can implement I/O bound operations using Task.Run(), but why would you? Doing so commits a thread to the operation which, for an operation that would not otherwise require a thread, is wasteful.
I/O bound operations generally have support from an I/O completion port and the IOCP thread pool (the threads of which handle completions of an arbitrarily large number of IOCPs) and so it is more efficient to simply use the existing asynchronous I/O API, rather than to use Task.Run() to call a synchronous I/O method.
The MSDN documentation appears to state that async and await are suitable for IO-bound tasks whereas Task.Run should be used for CPU-bound tasks.
I'm working on an application that performs HTTP requests to retrieve HTML documents, which it then parses. I have a method that looks like this:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync())
return await Task.Run(() => LoadHtmlDocument(contentStream)); //CPU-bound
}
Is this good and suitable use of async and await, or am I over-using it?
There are two good answers already, but to add my 0.02...
If you're talking about consuming asynchronous operations, async/await works excellently for both I/O-bound and CPU-bound.
I think the MSDN docs do have a slight slant towards producing asynchronous operations, in which case you do want to use TaskCompletionSource (or similar) for I/O-bound and Task.Run (or similar) for CPU-bound. Once you've created the initial Task wrapper, it's best consumed by async and await.
For your particular example, it really comes down to how much time LoadHtmlDocument will take. If you remove the Task.Run, you will execute it within the same context that calls LoadPage (possibly on a UI thread). The Windows 8 guidelines specify that any operation taking more than 50ms should be made async... keeping in mind that 50ms on your developer machine may be longer on a client's machine...
So if you can guarantee that LoadHtmlDocument will run for less than 50ms, you can just execute it directly:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync()) //IO-bound
return LoadHtmlDocument(contentStream); //CPU-bound
}
However, I would recommend ConfigureAwait as #svick mentioned:
public async Task<HtmlDocument> LoadPage(Uri address)
{
using (var httpResponse = await new HttpClient().GetAsync(address)
.ConfigureAwait(continueOnCapturedContext: false)) //IO-bound
using (var responseContent = httpResponse.Content)
using (var contentStream = await responseContent.ReadAsStreamAsync()
.ConfigureAwait(continueOnCapturedContext: false)) //IO-bound
return LoadHtmlDocument(contentStream); //CPU-bound
}
With ConfigureAwait, if the HTTP request doesn't complete immediately (synchronously), then this will (in this case) cause LoadHtmlDocument to be executed on a thread pool thread without an explicit call to Task.Run.
If you're interested in async performance at this level, you should check out Stephen Toub's video and MSDN article on the subject. He has tons of useful information.
It is appropriate to await any operation that is asynchronous (i.e. is represented by a Task).
The key point is that for IO operations, whenever possible, you want to use a provided method that is, at it's very core, asynchronous, rather than using Task.Run on a blocking synchronous method. If you're blocking a thread (even a thread pool thread) while performing IO, you're not leveraging the real power of the await model.
Once you have created a Task that represents your operation you no longer care if it's CPU or IO bound. To the caller it's just some async operation that needs to be await-ed.
There are several things to consider:
In a GUI application, you want as little code as possible to execute on the UI thread. In that case, offloading CPU-bound operation to another thread using Task.Run() is probably a good idea. Though the users of your code can do that themselves, if they want.
In something like ASP.NET application, there is no UI thread and all you care about is performance. In that case, there is some overhead in using Task.Run() instead of running the code directly, but it shouldn't be significant if the operation actually takes some time. (Also, there is some overhead in returning to the synchronization context, which is one more reason why you should use ConfigureAwait(false) for most awaits in your library code.)
If your method is async (which BTW should be also reflected in the name of the method, not just its return type), people will expect that it won't block the synchronization context thread, even for CPU-bound work.
Weighting that, I think using await Task.Run() is the right choice here. It does have some overhead, but also some advantages, which can be significant.