i am after some advice/strategy on limiting http requests when consuming multiple web services. I feel i could do this if the requests were happening synchronously, but they are asynchronous and think i should try to perform the limit logic in a way that i wont block.
Due to the web app consuming multiple web services there will be different limits for different requests. I was thinking something like this, but aren't sure how to proceed in a no blocking manner:
request method:
public static Task<string> AsyncRequest(string url, enum webService)
{
using(LimitingClass limiter = new LimitingClass(webService))
{
//Perform async request
}
}
In the LimitingClass it will have logic like checking the last request for the given webservice, if it violates the limit then it will wait a certain amount of time. But in the mean time if another request comes in to a different webservice then i dont want that request to be blocked while the LimitingClass is waiting. Is there anything fundamentally wrong with the above approach? Should i open up a new thread with each LimitingClass instance?
Some pseudo code would be great if possible.
Many Thanks
UPDATE:
This is a simplified version of my current request method:
public static Task<string> MakeAsyncRequest(string url, string contentType)
{
HttpWebRequest request = //set up my request
Task<WebResponse> task = Task.Factory.FromAsync(
request.BeginGetResponse,
asyncResult => request.EndGetResponse(asyncResult),
(object)null);
return task.ContinueWith(t => ReadCallback(t.Result));
}
I just want to wrap this in a using to check the limits and which doesnt block other requests.
So how are you limiting an access to resource without blocking ?
I think you need to look at Semaphores - they allow to limits the number of threads that can access a resource or pool of resources concurrently.
Related
(This discussion might not be specific to C#...)
I have a C# method SendMultipleRequests that sends HTTP POST request 10 times sequentially.
Is it possible for the server to receive requests out of order?
If my understanding is correct if the requests are sent concurrently (without await), the server could receive requests out of order, but in the example below it needs to wait for the response to be received at the client before sending next request, so the server will receive requests in order.
public async Task SendRequest(int i)
{
// definition of endpoint is omitted in this example
var content = new StringContent($"I am {i}-th request");
await HttpClient.PostAsync(endpoint, content);
}
public async Task SendMultipleRequests()
{
for (int i = 0; i < 10; i++)
{
await SendRequest(i);
}
}
with await your app will wait for the task returned by PostAsync to finish before it issues the next request - see the docs for postasync where it says “This operation will not block. The returned Task<TResult> object will complete after the whole response (including content) is read.” - using await will mean that you will only issue the next request after you I’ve read the content of the previous response
If you remove the await then your code will queue up ten tasks and start working on them all in some undefined order. The server will see requests in an unspecified order. This may be further exacerbated by the fact that the requests may take different routes through the internet, some slower. If you remove the await then you should capture the returned task into a list, then you can use something like await Task.WhenAll(list) to wait for them all to complete (unless you really want to fire and forget in which case you can assign them to the discard _ = client.PostAsync... but keeping the task allows you to discover and deal with exceptions that arose)
I have to make multiple Http post requests (few hundreds or can be more) and not wait for any of the responses as have an SLA. Without waiting for those response I need to send back response from my Web API (before performing mentioned multiple HTTP requests, I fetch data from another API and need to return back).
I have looked around and found "fire and forget" implementation which does not wait for response. I am not sure if this right way to do and since im returning without waiting for parallel fire and forget requests, how will HttpClient get disposed?
HttpClient client = new HttpClient();
var CompositeResponse = client.GetAsync(_SOMEURL);
List<MEDLogresp> sortedmeds = MEDLogresps.OrderBy(x => x.rxId).ThenBy(y => y.recordActionType);
Task.Run(() => Parallel.ForEach(sortedmeds, ele => clientMED.PostAsync(URL , new StringContent(JsonConvert.SerializeObject(ele), Encoding.UTF8, "application/json"))));
return ResponseMessage(Request.CreateResponse<CompositeResponse>(HttpStatusCode.OK, compositeResponse));
since im returning without waiting for parallel fire and forget
requests, how will httpclient get disposed?
You can use a single, shared static instance of HttpClient for the lifetime of your application and never Dispose() it. It is safe to use the same HttpClient from multiple threads concurrently.
Consider the simple MVC5 controller:
public class DocumentsController {
// ctor code is omitted
[HttpPost, Route("api/documents/request/stamp={stamp}")]
public ActionResult RequestDocuments(string stamp) {
var documents = this.DocumentsRequestService.RequestByStamp(stamp);
return new JsonResult(documents);
}
}
The DocumentsRequestService does these things internally:
it sends a request to a dedicated MSMQ-queue (let's call it M) AND synchronously waits for an incoming message at the M's response queue:
using(var requestMessage = new Message()) {
requestMessage.Body = documentStamp;
requestMessage.Recoverable = true;
requestMessage.Label = "request";
requestMessage.ResponseQueue = this.requestedDocumentsResponseQueue;
requestMessage.Formatter = new XmlMessageFormatter(new Type[] { typeof(String) });
// send request
this.requestedDocumentsQueue.Send(requestMessage);
// synchronously wait for response
var responseMessage = this.requestedDocumentsResponseQueue.Receive();
if(responseMessage.Label.EndsWith("success")) {
return new DocumentsRequestResult(
success: true,
matches: parseMatchesList(responseMessage)
);
}
return new DocumentsRequestResult(
success: false,
matches: Enumerable.Empty<DocumentsRequestMatch>()
);
}
the consumer (Windows Service) of that message makes a specific api call. By saying 'specific' I mean that we use a third-party means to do that. This call is synchronous and quite long. When the processing ends the consumer sends a response message to the requesting message's response queue.
when response arrives at the M's response queue it's a time to parse and return the results to the controller.
From the end user's perspective this task should be blocking, or at least it should look like blocking.
As far as I understand running a Task makes use of parallelization. Whereas using the async-await pair makes the running task asynchronous. It could be helpful if several tasks would run in parallel.
Is it reasonable/possible to incorporate with Tasking/Asynchrony in my case? If yes, then where do I start?
The "asynchrony" of a network call is transparent to the caller. It doesn't matter to the caller whether the implementation is synchronous or asynchronous. Put another way, from a client's perspective, it's always asynchronous.
For example, the HTTP client couldn't care less if RequestDocuments is synchronous or asynchronous; either way, the HTTP client will send a request and receive a response some time later (i.e., asynchronously).
Similarly, the HTTP web server doesn't care whether the Win32 service is implemented synchronously or asynchronously. It just knows that it puts a message on a queue and some time later (i.e., asynchronously) it gets a response message from the queue.
As far as I understand running a Task makes use of parallelization. Whereas using the async-await pair makes the running task asynchronous.
Sort of. Task can be used for either asynchronous or parallel code, a fact that has caused much confusion. However, Task Parallel Library constructs such as Parallel and PLINQ are firmly in the parallel (non-asynchronous) world.
It could be helpful if several tasks would run in parallel.
I believe "concurrently" is the appropriate term here.
First, note that ASP.NET gives you a considerable amount of concurrency for free. If you want to make each request internally concurrent, then you can do so fairly easily via Task.WhenAll. For example, you can change your DocumentsRequestService call to be asynchronous (assuming your message queue API supports async calls):
using(var requestMessage = new Message()) {
...
// send request
await this.requestedDocumentsQueue.SendAsync(requestMessage);
// asynchronously wait for response
var responseMessage = await this.requestedDocumentsResponseQueue.ReceiveAsync();
...
}
Then you can call it multiple times simultaneously from a single controller action as such:
public async Task<ActionResult> RequestDocuments(string stamp1, string stamp2) {
var task1 = this.DocumentsRequestService.RequestByStampAsync(stamp1);
var task2 = this.DocumentsRequestService.RequestByStampAsync(stamp2);
var documents = await Task.WhenAll(task1, task2);
return new JsonResult(documents);
}
I'm with some performance problems working with WEB Api. On my real/production code, I'll do a SOAP WS call, on this sample, I'll just sleep. I have 400+ clients sending request's to the Web API.
I guess it's a problem with web api, because if I open 5 process, I can handle more requests than when I'm only with one process.
My test async version of the controller looks like this
[HttpPost]
public Task<HttpResponseMessage> SampleRequest()
{
return Request.Content.ReadAsStringAsync()
.ContinueWith(content =>
{
Thread.Sleep(Timeout);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(content.Result, Encoding.UTF8, "text/plain")
};
});
}
The sync version looks like this
[HttpPost]
public HttpResponseMessage SampleRequest()
{
var content = Request.Content.ReadAsStringAsync().Result;
Thread.Sleep(Timeout);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(content, Encoding.UTF8, "text/plain")
};
}
My client code to this test, looks like this (it is configured to time out after 30 seconds)
for (int i = 0; i < numberOfRequests; i++)
{
tasks.Add(new Task(() =>
{
MakeHttpPostRequest();
}));
}
foreach (var task in tasks)
{
task.Start();
}
I was not able to put it here in a nice way, but the table with the results are available at github
The CPU, memory and disk IO is low. There's always at least 800 available threads (both worker and io threads)
public static void AvailableThreads()
{
int workerThreads;
int ioThreads;
ThreadPool.GetAvailableThreads(out workerThreads, out ioThreads);
Console.WriteLine("Available threads {0} ioThreads {1}", workerThreads, ioThreads);
}
I've configured the DefaultConnectionLimit
System.Net.ServicePointManager.DefaultConnectionLimit = Int32.MaxValue;
My question is why there's a queue to answer those request?
In every test, I began with a response time almost exactly like the server Thread.Sleep() time, but the responses get slower as new request arrive.
Any tip on how I can discover where's the bootleneck?
It is a .net 4.0 solution, using self host option.
Edit: I've also tested with .net 4.5 and Web API 2.0, and got the same behaviour.
First requests got the answer almost as soon as sleep expires, later it takes up to 4x the sleep time to get an answer.
Edit2: Gist of the web api1 implementation and gist of the web api2 implementation
Edit3: The MakeHttpPost method creates a new WebApiClient
Edit4:
If I change the
Thread.Sleep()
to
await Task.Delay(10000);
in the .net 4.5 version, it can handle all requests, as expected. So I don't think something related to any network issue.
Since Thread.Sleep() blocks the thread and Task.Delay don't, looks like there's an issue with webapi to consume more threads? But there's available threads in the threadpool...
Edit 5: If I open 5 servers and double the number of clients, the server can respond to all requests. So looks like it's not a problem with number of request to a server, because I can 'scale' this solution running a lot of process in different ports. It's more a problem with the number of request to the same process.
How to Check the TCP/IP stack for overloading
Run Netstat on the server having the issue, look for any Time-Waits, Fin-Wait-1, Fin-Wait-2 and RST-Wait, RST-Wait2. These are half-baked sessions whereby the stack is waiting for the other side to clean up.....OR.....the other side did send a packet in but the local machine could not process them yet depending on the Stacks' availability to do the job.
The kicker is that even sessions showing Established could be in trouble in that the time-out hasn't fired yet.
The symptoms described above are reminiscent of network or TCP/IP stack overload. Very similar behavior is seen when routers get overloaded.
I am just looking at the default MVC5 project and how it uses async in the controllers.
I would like to know what benefit async provides here over simply using synchronous calls:
[HttpPost]
[ValidateAntiForgeryToken]
public async Task<ActionResult> Disassociate(string loginProvider, string providerKey)
{
ManageMessageId? message = null;
//why use an async database call here with await instead of just using a synchronous one?
IdentityResult result = await UserManager.RemoveLoginAsync(User.Identity.GetUserId(), new UserLoginInfo(loginProvider, providerKey));
if (result.Succeeded)
{
message = ManageMessageId.RemoveLoginSuccess;
}
else
{
message = ManageMessageId.Error;
}
return RedirectToAction("Manage", new { Message = message });
}
What am I missing?
Does this provide some kind of performance benefit in the type of wait that will occur here?
On the server side (e.g., ASP.NET MVC), any I/O you do (e.g., databases) should be done asynchronously. This frees up the request thread for the time that the I/O is in flight.
So, when the RemoveLoginAsync sends its SQL to the database, it returns an incomplete task, and when the request hits the await, it returns the request thread to the thread pool. Later, when the DB operation completes, a request thread is taken from the thread pool and used to continue the request.
The end result is scalability, because (in the 99.9% case at least) tasks scale better than threads. However, there isn't a big advantage if your database is just a single server and all requests hit the db, because your scalability bottleneck in that scenario is the db server, not the web server.