The scenario
I need to show N reports on a web page. The reports need to be requested to an external service. The time for the service to generate a report can vary from 2 seconds to 50 seconds, depending on the requested content.
To call the service I use HttpClient in an async action. To generate 1 report I call the service once. To generate 5 reports I call it 5 times and so on.
The Problem
Let's suppose we request 3 reports BigReport, MediumReport and SmallReport with a known relative generation time of 1 minute, 30 seconds and 2 seconds, and we call the service in the following order:
BigReport, MediumReport, SmallReport
The result of the HttpCalls will be as following:
HttpCall response for BigReport returns SmallReport (which is the quickest to be generated)
MediumReport will be correct
SmallReport response will contain the BigReport (which is the longest and the last)
Basicly, although the HttpCalls are different, for the fact they are made over a very short period of time, and they are still "active", the server will repond based on first arrived, first served, instead of serving each call with its exact response.
The Code
I have a Request controller with an async action like this:
public async Task<string> GenerateReport(string blockContent)
{
var formDataContent = new MultipartFormDataContent
{
AddStringContent(userid, "userid"),
AddStringContent(passcode, "passcode"),
AddStringContent(outputtype, "outputtype"),
AddStringContent(submit, "submit")
};
var blockStream = new StreamContent(new MemoryStream(Encoding.Default.GetBytes(blockContent)));
blockStream.Headers.Add("Content-Disposition", "form-data; name=\"file\"; filename=\"" + filename + "\"");
formDataContent.Add(blockStream);
using (var client = new HttpClient())
{
using(var message = await client.PostAsync(Url, formDataContent))
{
var report = await message.Content.ReadAsStringAsync();
return report;
}
}
}
The action is being called from a view via Ajax, like this
//FOREACH BLOCK, CALL THE REPORT SERVICE
$('.block').each(function(index, block) {
var reportActionUrl = "Report/GenerateReport/"+block.Content;
//AJAX CALL GetReportAction
$(block).load(reportActionUrl);
});
Everything works fine if I covert the action from async to sync, by removing async Task and instead of "awaiting" for the response, I just get result as
var result = client.PostAsync(Url, formDataContent).Result.
This will make everything run synchronously and working fine, but the waiting time for the user, will be much longer. I would really like to avoid this by making parallel calls or similar.
Conclusions and questions
The problem itself make sense, after inspecting it also with Fiddler, as we have multiple opened HttpRequests pending almost simultaneously.
I suppose I need a sort of handler or something to identify and match request/response, but I don't know what's the name of the "domain" I need to look for. So far, my questions are:
What is the technical name of "making multiple http calls in parallel"?
If the problem is understandable, what is name of the problem? (concurrency, parallel requests queuing, etc..?)
And of course, is there any solution?
Many thanks.
With a "bit" of delay, I post the solution.
The problem was that the filename parameter was incorrectly called filename instead of blockname. This was causing the very weird behaviour, as a file could have had many blocks.
The lesson learned was that in case of very weird behaviour, in this case with a HttpClient call, analyse all the possible parameters and test it with different values, even if it doesn't make too much sense. At worst it can throw an error.
Related
It seems that HttpClient is the recommended way to do HTTP communication. Downloading HTML of a URL seemed easy like
var html = httpClient.GetStringAsync(url);
But I need to change some header values like Referer each time. At first, I tried
httpClient.DefaultRequestHeaders.Add("Referer", referrer);
, but this caused an error at the second time. It seems that it is once set, cannot be changed.
I searched for a solution and found one ( https://stackoverflow.com/a/12023307/455796 ), but this seems very complicated than GetStringAsync. I need to create a HttpRequestMessage, call SendAsync, continue to call response.Content.ReadAsAsync, call Wait(), and then read the result. Also, the comment says I need to dispose the HttpRequestMessage. If this is the only way to change headers, I will do so, but is this the best way? Can't I use GetStringAsync and change a header value?
You could do it this way:
using(var msg = new HttpRequestMessage(HttpMethod.Get, new Uri("https://www.test.com")))
{
msg.Headers.Referrer = new Uri("https://www.example.com");
using (var req = await _httpClient.SendAsync(msg))
{
var str1 = await req.Content.ReadAsStringAsync();
}
}
or, if you want to override the default request headers, do it this way:
_httpClient.DefaultRequestHeaders.Referrer = new Uri("https://www.example.com");
var str2 = await _httpClient.GetStringAsync("https://www.test.com/");
The first way is if you want to reuse your HttpClient throughout the lifetime of your application. Meaning, if you share it with every object or method that needs to make an HTTP request. Doing it this will not cause conflicts if multiple threads are trying to use it at the same time when each thread is modifying headers. DeafaultRequestHeaders is not thread-safe... so you could get exceptions thrown if more than one thread is modifying the DeafaultRequestHeaders property at the same time.
We are having an issue with our Application where we are never getting a response back to the client (Chrome in this case) from a pretty simple asynchronous file upload call. It also bogs down our server for up to 2 minutes. Below is our Controller method:
public async Task<HttpResponseMessage> Post(string id, string fileName)
{
string[] allowedAttachmentFileTypes = ConfigurationManager.AppSettings["AttachmentsSetting"].Split(',');
string extension = Path.GetExtension(fileName);
bool extensionAllowed = allowedAttachmentFileTypes.Any(allowedAttachmentFileType => allowedAttachmentFileType.ToLower().Trim() == extension.ToLower().Trim());
if (extensionAllowed)
{
var fileResult = await Request.Content.ReadAsStreamAsync();
//...do async database stuff with fileResult
return Request.CreateResponse(HttpStatusCode.OK, result);
}
else
{
//this never makes it back to client
var response = new HttpResponseMessage(HttpStatusCode.UnsupportedMediaType)
{
Content = new StringContent(ConfigurationManager.AppSettings["AttachmentsSetting"])
};
return response;
}
}
What I am mainly concerned with at the moment is that we are testing with what would in our system be an invalid file extension so that it would go to the else clause and just return the bad response. When we put a breakpoint here, it hits our controller and eventually hits our return in the else clause, and works as expected, but in Chrome, it still shows "Pending".
Another thing is it seems to be dependent on the size of the file we send to the Controller, even though we don't really do anything with the file unless the extension is valid. An invalid file that is 26,939KB never gives us a server response. While one that is 17,432KB gave us one, although it still took a minute.
One other thing I should add: This is more inconsistent, but at times if we do have a valid file extension on a bigger file, like say the 26,939KB one, we would get "There is no longer an HttpContext available." when trying to copy the file to the file system
This ended up being a firewall issue. Nothing code related.
In my c# wpf application when a user presses a button I need to send to a server 10-20 requests. They can be sent in an arbitrary order but there has to be at least 10 of them because the server returns the results paginated.
Each client (my c# is a client) has an apy key and server can only handle 1 request per second per a certain client, otherwise the server returns an error.
How can send those requests properly? Should I necessarily use async and await? And can I send them in parallel and how? Doesn't async in this case means that they'll be sent in parallel?
And, how can I ensure that only 1 request per second is sent? I gathered it's not good to mix the threads, which is Thread.Sleep(1000) for my case, and async/await.
So, you could create a bunch of tasks that stagger the job by a second each time.
Something like:
List<Uri> uris=new List<Uri>(); //fill with uris
var tasks = uris.Select(async (u, i)=>{
await Task.Delay(TimeSpan.FromSeconds(i));
using(var wc = new WebClient())
{
return await wc.DownloadStringTaskAsync(u);
}
});
var results = await Task.WhenAll(tasks);
I'm with some performance problems working with WEB Api. On my real/production code, I'll do a SOAP WS call, on this sample, I'll just sleep. I have 400+ clients sending request's to the Web API.
I guess it's a problem with web api, because if I open 5 process, I can handle more requests than when I'm only with one process.
My test async version of the controller looks like this
[HttpPost]
public Task<HttpResponseMessage> SampleRequest()
{
return Request.Content.ReadAsStringAsync()
.ContinueWith(content =>
{
Thread.Sleep(Timeout);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(content.Result, Encoding.UTF8, "text/plain")
};
});
}
The sync version looks like this
[HttpPost]
public HttpResponseMessage SampleRequest()
{
var content = Request.Content.ReadAsStringAsync().Result;
Thread.Sleep(Timeout);
return new HttpResponseMessage(HttpStatusCode.OK)
{
Content = new StringContent(content, Encoding.UTF8, "text/plain")
};
}
My client code to this test, looks like this (it is configured to time out after 30 seconds)
for (int i = 0; i < numberOfRequests; i++)
{
tasks.Add(new Task(() =>
{
MakeHttpPostRequest();
}));
}
foreach (var task in tasks)
{
task.Start();
}
I was not able to put it here in a nice way, but the table with the results are available at github
The CPU, memory and disk IO is low. There's always at least 800 available threads (both worker and io threads)
public static void AvailableThreads()
{
int workerThreads;
int ioThreads;
ThreadPool.GetAvailableThreads(out workerThreads, out ioThreads);
Console.WriteLine("Available threads {0} ioThreads {1}", workerThreads, ioThreads);
}
I've configured the DefaultConnectionLimit
System.Net.ServicePointManager.DefaultConnectionLimit = Int32.MaxValue;
My question is why there's a queue to answer those request?
In every test, I began with a response time almost exactly like the server Thread.Sleep() time, but the responses get slower as new request arrive.
Any tip on how I can discover where's the bootleneck?
It is a .net 4.0 solution, using self host option.
Edit: I've also tested with .net 4.5 and Web API 2.0, and got the same behaviour.
First requests got the answer almost as soon as sleep expires, later it takes up to 4x the sleep time to get an answer.
Edit2: Gist of the web api1 implementation and gist of the web api2 implementation
Edit3: The MakeHttpPost method creates a new WebApiClient
Edit4:
If I change the
Thread.Sleep()
to
await Task.Delay(10000);
in the .net 4.5 version, it can handle all requests, as expected. So I don't think something related to any network issue.
Since Thread.Sleep() blocks the thread and Task.Delay don't, looks like there's an issue with webapi to consume more threads? But there's available threads in the threadpool...
Edit 5: If I open 5 servers and double the number of clients, the server can respond to all requests. So looks like it's not a problem with number of request to a server, because I can 'scale' this solution running a lot of process in different ports. It's more a problem with the number of request to the same process.
How to Check the TCP/IP stack for overloading
Run Netstat on the server having the issue, look for any Time-Waits, Fin-Wait-1, Fin-Wait-2 and RST-Wait, RST-Wait2. These are half-baked sessions whereby the stack is waiting for the other side to clean up.....OR.....the other side did send a packet in but the local machine could not process them yet depending on the Stacks' availability to do the job.
The kicker is that even sessions showing Established could be in trouble in that the time-out hasn't fired yet.
The symptoms described above are reminiscent of network or TCP/IP stack overload. Very similar behavior is seen when routers get overloaded.
I am using HttpClient to send a GET request to a server inside of a while loop
while (cycle < maxcycle)
{
var searchParameters = new ASearchParameters
{
Page = cycle++,
id = getid
};
var searchResponse = await Client.SearchAsync(searchParameters);
}
and the SearchAsync contains
public async Task<AuctionResponse> SearchAsync()
{
var uriString = "Contains a https url with parameters"
var searchResponseMessage = await HttpClient.GetAsync(uriString);
return await Deserialize<AuctionResponse>(searchResponseMessage);
}
The thing is after every request there is a delay before the next request is started.
you can see this in fiddler timeline and also in fiddler there is "Tunnel To" example.com:443 before every request
Question : Why is there a delay and how to remove it ?
I see two things that are happening here. First, depending on the deserializer, it may take a while to translate your response back into an object. You might want to time that step and see if that's not the majority of your time spent. Second, the SSL handshake (the origin of your "tunnel to") does require a round trip to establish the SSL channel. I thought HttpClient sent a Keep-Alive header by default, but you may want to see if it is A) not being sent or B) being rejected. If you are re-establishing an SSL channel for each request, that could easily take on the order of a hundred ms all by itself (depending upon the server/network load).
If you're using Fiddler, you can enable the ability to inspect SSL traffic to see what the actual request/response headers are.
I believe you see this delay for a couple of reasons. Based on the code you provided, all other actions besides the request itself take up some fraction of the time between requests. So deserializing the response will add to a delay.
Also, the delay might be tied to the amount of data that is being returned and processed further down the stack. I tried to recreate the scenario you describe in your question with the following code:
const int MaxNumberOfCycles = 10;
static void Main()
{
Start().Wait();
}
async Task Start()
{
var client = new Client();
var cycle = 0;
while (cycle < MaxNumberOfCycles)
{
var response = await client.SearchAsync(cycle++);
}
}
class Client
{
public async Task<HttpResponseMessage> SearchAsync(int n)
{
// parameter 'n' used to vary web service response data
var url = ... // url removed for privacy
using (var client = new HttpClient())
using (var response = await client.GetAsync(url))
{
return response;
}
}
}
With small response sizes I saw no delay between requests. As response sizes increased I began to see slightly longer delays. Here's a screenshot for a series of requests returning 1MB responses:
One thing I noticed about your scenario is that your transfer activity graph shows a solid black line at the end of each request. This line indicates the "time to first byte", meaning that response processing did not even start until the very end of your request.
Another issue you might consider is that Fiddler is that causing these delays. I noticed that your responses aren't being streamed by Fiddler, which probably impacts the results. You can read more about response streaming in Fiddler.
I hope some of this information helps...