We Keep Coding

WebRequest - GetRequestStream timing out with exception

I'm writing this short module where I have to modify an addressed resource with a PUT method. I'm using the WebRequest class to make this URI request and the GetRequestStream() to get the stream to write to.
However, it seems that after a couple of successful method calls (and using the PUT to modify resources) via this method below, my application hangs and then throws a WebException: The request timed out. error. Here's what the code looks like:
public void SendOffMessageToResource(int res_ID){
var httpWebRequest = WebRequest.Create ("http://192.168.x.x/api/sample_user/resources/1/state");
httpWebRequest.ContentType = "application/json";
httpWebRequest.Method = "PUT";
using (var streamWriter = new StreamWriter(httpWebRequest.GetRequestStream())) //here's where the VS seems to take a long long time to step over to the next line when the error happens.
{
string json = "{\"on\":false}";
streamWriter.Write(json);
streamWriter.Close();
}
}
I am already disposing the StreamWriter. I'm not even using the GetResponse() method, because all I need to do on this URI is actually modify the addressed resource with PUT method. I am not sure why it still throws an error and hangs the application. The search of previous threads only revealed that people should be using using statements to dispose resources, but I'm already doing that I think or perhaps I'm missing something? Do I always need to use GetResponse() to complete the request and dispose that always in addition to this?

Do I always need to use GetResponse() to complete the request and dispose that always in addition to this?
Yes, just that.

Web API call works but gives exception The view 'xxx' or its master was not found or no view engine supports

Strange one here, code calls the method, and the method grabacat is executed on the server (I debug it and step through right to the end). The code returns to the client, but the response it received was 500 Internal Server Error with the above message. So it's saying it couldn't find the web API method that it just called successfully.
using (var response = await client.PostAsXmlAsync("cats/grabacat", mycatprefs))
{
if (response.IsSuccessStatusCode) // 500 cats/grabacat not found
Controller code:
[Route("~/api/cats/grabacat")]
[HttpPost]
public async Task GrabACat()
{
}
After debugging, if I change it to public async Task<SomeObject> GrabACat() then it works OK. This lets me return an object back to the client. However I don't want to return anything back; I want it to be equivelent to calling a void method. It will examine the status code to determine if it was successful.
I have got it working by changing GrabACat to Task and returning new object(); but I am not sure why this is required. It seems crude to return an empty object just to get it to work.
Any ideas?

The WebAPI method has a Route attribute like this:
[Route("~/api/cats/grabacat")]
Which means the URL is wrong in the POST request - you are missing the /api prefix:
using (var response = await client.PostAsXmlAsync("api/cats/grabacat", mycatprefs))
//snip

An unexpected error occurred on a receive. WebRequest already overriden

I never thought I have to ask a question here, but I am at my wit's end.
We try contact a server and receive XML Data via SOAP from there. Regulary it's running fine, but sometimes and without any hint why, we've got the the following error:
Translated from german:
The underlying connection was closed: An unexpected error occurred on a
receive.
So I looked for possible reasons. And I overrode the WebRequest Method.
public class MyHttpClientProtocol : SoapHttpClientProtocol
{
protected override WebRequest GetWebRequest(Uri uri)
{
HttpWebRequest webRequest = (HttpWebRequest) base.GetWebRequest(uri);
//Setting KeepAlive to false
webRequest.KeepAlive = false;
return webRequest;
}
}
First of all? - Is this correct or did I chose the wrong base class?
I am not sure at all, but I use SoapHttpClientProtocoll in my service, but in the Request Header (used Fiddler) the "Connection" stills "keep-alive".
The error is rare but it occours and if it does, it does it for a longer time.
So I tried to debug the problem. And if I do and I've got the error he repeats to try the delegate:
ServicePointManager.ServerCertificateValidationCallback += delegate { return true; };
I can't jump in. VS 2010 does not support it.
I am working with X509Certificate2.
Please give me an idea.. I can't see the solution.

Web Api + HttpClient: An asynchronous module or handler completed while an asynchronous operation was still pending

I'm writing an application that proxies some HTTP requests using the ASP.NET Web API and I am struggling to identify the source of an intermittent error.
It seems like a race condition... but I'm not entirely sure.
Before I go into detail here is the general communication flow of the application:
Client makes a HTTP request to Proxy 1.
Proxy 1 relays the contents of the HTTP request to Proxy 2
Proxy 2 relays the contents of the HTTP request to the Target Web Application
Target Web App responds to the HTTP request and the response is streamed (chunked transfer) to Proxy 2
Proxy 2 returns the response to Proxy 1 which in turn responds to the original calling Client.
The Proxy applications are written in ASP.NET Web API RTM using .NET 4.5.
The code to perform the relay looks like so:
//Controller entry point.
public HttpResponseMessage Post()
{
using (var client = new HttpClient())
{
var request = BuildRelayHttpRequest(this.Request);
//HttpCompletionOption.ResponseHeadersRead - so that I can start streaming the response as soon
//As it begins to filter in.
var relayResult = client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead).Result;
var returnMessage = BuildResponse(relayResult);
return returnMessage;
}
}
private static HttpRequestMessage BuildRelayHttpRequest(HttpRequestMessage incomingRequest)
{
var requestUri = BuildRequestUri();
var relayRequest = new HttpRequestMessage(incomingRequest.Method, requestUri);
if (incomingRequest.Method != HttpMethod.Get && incomingRequest.Content != null)
{
relayRequest.Content = incomingRequest.Content;
}
//Copies all safe HTTP headers (mainly content) to the relay request
CopyHeaders(relayRequest, incomingRequest);
return relayRequest;
}
private static HttpRequestMessage BuildResponse(HttpResponseMessage responseMessage)
{
var returnMessage = Request.CreateResponse(responseMessage.StatusCode);
returnMessage.ReasonPhrase = responseMessage.ReasonPhrase;
returnMessage.Content = CopyContentStream(responseMessage);
//Copies all safe HTTP headers (mainly content) to the response
CopyHeaders(returnMessage, responseMessage);
}
private static PushStreamContent CopyContentStream(HttpResponseMessage sourceContent)
{
var content = new PushStreamContent(async (stream, context, transport) =>
await sourceContent.Content.ReadAsStreamAsync()
.ContinueWith(t1 => t1.Result.CopyToAsync(stream)
.ContinueWith(t2 => stream.Dispose())));
return content;
}
The error that occurs intermittently is:
An asynchronous module or handler completed while an asynchronous operation was still pending.
This error usually occurs on the first few requests to the proxy applications after which the error is not seen again.
Visual Studio never catches the Exception when thrown.
But the error can be caught in the Global.asax Application_Error event.
Unfortunately the Exception has no Stack Trace.
The proxy applications are hosted in Azure Web Roles.
Any help identifying the culprit would be appreciated.

Your problem is a subtle one: the async lambda you're passing to PushStreamContent is being interpreted as an async void (because the PushStreamContent constructor only takes Actions as parameters). So there's a race condition between your module/handler completing and the completion of that async void lambda.
PostStreamContent detects the stream closing and treats that as the end of its Task (completing the module/handler), so you just need to be sure there's no async void methods that could still run after the stream is closed. async Task methods are OK, so this should fix it:
private static PushStreamContent CopyContentStream(HttpResponseMessage sourceContent)
{
Func<Stream, Task> copyStreamAsync = async stream =>
{
using (stream)
using (var sourceStream = await sourceContent.Content.ReadAsStreamAsync())
{
await sourceStream.CopyToAsync(stream);
}
};
var content = new PushStreamContent(stream => { var _ = copyStreamAsync(stream); });
return content;
}
If you want your proxies to scale a bit better, I also recommend getting rid of all the Result calls:
//Controller entry point.
public async Task<HttpResponseMessage> PostAsync()
{
using (var client = new HttpClient())
{
var request = BuildRelayHttpRequest(this.Request);
//HttpCompletionOption.ResponseHeadersRead - so that I can start streaming the response as soon
//As it begins to filter in.
var relayResult = await client.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
var returnMessage = BuildResponse(relayResult);
return returnMessage;
}
}
Your former code would block one thread for each request (until the headers are received); by using async all the way up to your controller level, you won't block a thread during that time.

I would like to add some wisdom for anyone else who landed here with the same error, but all of your code seems fine. Look for any lambda expressions passed into functions across the call-tree from where this occurs.
I was getting this error on a JavaScript JSON call to an MVC 5.x controller action. Everything I was doing up and down the stack was defined async Task and called using await.
However, using Visual Studio's "Set next statement" feature I systematically skipped over lines to determine which one caused it. I kept drilling down into local methods until I got to a call into an external NuGet package. The called method took an Action as a parameter and the lambda expression passed in for this Action was preceded by the async keyword. As Stephen Cleary points out above in his answer, this is treated as an async void, which MVC does not like. Luckily said package had *Async versions of the same methods. Switching to using those, along with some downstream calls to the same package fixed the problem.
I realize this is not a novel solution to the problem, but I passed over this thread a few times in my searches trying to resolve the issue because I thought I didn't have any async void or async <Action> calls, and I wanted to help someone else avoid that.

A slightly simpler model is that you can actually just use the HttpContents directly and pass them around inside the relay. I just uploaded a sample illustrating how you can rely both requests and responses asynchronously and without buffering the content in a relatively simple manner:
http://aspnet.codeplex.com/SourceControl/changeset/view/7ce67a547fd0#Samples/WebApi/RelaySample/ReadMe.txt
It is also beneficial to reuse the same HttpClient instance as this allows you to reuse connections where appropriate.

How to perform a fast web request in C#

I have a HTTP based API which I potentially need to call many times. The problem is that I can't get the request to take less than about 20 seconds, though the same request made through a browser is near instantaneous. The following code illustrates how I have implemented it so far.
WebRequest r = HttpWebRequest.Create("https://example.com/http/command?param=blabla");
var response = r.GetResponse();
One solution would be to make an asynchronous request but I would like to know why it takes so long and if I can avoid it. I have also tried using the WebClient class but I suspect it uses a WebRequest internally.
Update:
Running the following code took about 40 seconds in Release Mode (measured with Stopwatch):
WebRequest g = HttpWebRequest.Create("http://www.google.com");
var response = g.GetResponse();
I'm working at a university where there might be different things in the network configuration affecting the performance, but the direct use of the browser illustrates that it should be near instant.
Update 2:
I uploaded the code to a remote machine and it worked fine so the conclusion must be that the .NET code does something extra compared to the browser or it has problems resolving the address through the university network (proxy issues or something?!).

This problem is similar to another post on StackOverflow:
Stackoverflow-2519655(HttpWebrequest is extremely slow)
Most of the time the problem is the Proxy server property. You should set this property to null, otherwise the object will attempt to search for an appropriate proxy server to use before going directly to the source. Note: this property is turn on by default, so you have to explicitly tell the object not to perform this proxy search.
request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}

I was having the 30 second delay on 'first' attempt - JamesR's reference to the other post mentioning setting proxy to null solved it instantly!
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_site.url);
request.Proxy = null; // <-- this is the good stuff
...
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Does your site have an invalid SSL cert? Try adding this
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AlwaysAccept);
//... somewhere AlwaysAccept is defined as:
using System.Security.Cryptography.X509Certificates;
using System.Net.Security;
public bool AlwaysAccept(object sender, X509Certificate certification, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
return true;
}

You don't close your Request. As soon as you hit the number of allowed connections, you have to wait for the earlier ones to time out. Try
using (var response = g.GetResponse())
{
// do stuff with your response
}