We Keep Coding

An unexpected error occurred on a receive. WebRequest already overriden

I never thought I have to ask a question here, but I am at my wit's end.
We try contact a server and receive XML Data via SOAP from there. Regulary it's running fine, but sometimes and without any hint why, we've got the the following error:
Translated from german:
The underlying connection was closed: An unexpected error occurred on a
receive.
So I looked for possible reasons. And I overrode the WebRequest Method.
public class MyHttpClientProtocol : SoapHttpClientProtocol
{
protected override WebRequest GetWebRequest(Uri uri)
{
HttpWebRequest webRequest = (HttpWebRequest) base.GetWebRequest(uri);
//Setting KeepAlive to false
webRequest.KeepAlive = false;
return webRequest;
}
}
First of all? - Is this correct or did I chose the wrong base class?
I am not sure at all, but I use SoapHttpClientProtocoll in my service, but in the Request Header (used Fiddler) the "Connection" stills "keep-alive".
The error is rare but it occours and if it does, it does it for a longer time.
So I tried to debug the problem. And if I do and I've got the error he repeats to try the delegate:
ServicePointManager.ServerCertificateValidationCallback += delegate { return true; };
I can't jump in. VS 2010 does not support it.
I am working with X509Certificate2.
Please give me an idea.. I can't see the solution.

Getting headers from WebRequest

Technically I'm asking a question for a friend who writes VB, but I post as C# since more people are on it. And I personally know neither.
I'm helping him connecting to a Mobile Backend as a Service, although the way he set up he is connecting it on behalf of someone loading his own web page with ASP.net (I think).
I'm connecting to the service just fine using Python. But he is getting a 422 server response. I would like to compare the request header & content difference between his and mine.
According to Chris Doggett's post on this page down below, you can't get the headers until the request is actually sent. However, as soon as request.GetResponse() is called, Visual Studio (or the Express, not sure) seems to just halt on a break point there and say there is a 422 error and some error message on the browser. So, he can't get to the next line where he wish to print out the headers.
Two questions:
Is that some sort of debugging turned on? I thought a 422 response is a response nevertheless and the program shouldn't just stop there.
How do I print out the the content as well, not just the headers? Preferably, I want to print out the entire request in text. There is this stuff sent in JSON and I don't think that belongs to the headers but I'm not so sure.

The Create method will return an HttpWebRequest for an http/https url. The 422 status code indicates that you are somehow sending incorrect formed data to the server. GetResponse() will throw a WebException because you don't receive the
status code 200.
To get the actual headers of the response you need to handle the exception
private static void Main(string[] args)
{
WebRequest request = WebRequest.Create("http://google.com/12345"); //generate 404
try
{
WebResponse response = request.GetResponse();
}
catch(WebException ex)
{
HttpWebResponse errorResponse = ex.Response as HttpWebResponse;
if (errorResponse == null)
throw; //errorResponse not of type HttpWebResponse
string responseContent = "";
using(StreamReader r = new StreamReader(errorResponse.GetResponseStream()))
{
responseContent = r.ReadToEnd();
}
Console.WriteLine("The server at {0} returned {1}", errorResponse.ResponseUri, errorResponse.StatusCode);
Console.WriteLine("With headers:");
foreach(string key in errorResponse.Headers.AllKeys)
{
Console.WriteLine("\t{0}:{1}", key, errorResponse.Headers[key]);
}
Console.WriteLine(responseContent);
}
Console.ReadLine();
}

POST data to WCF Service from WP7

I am working on a WP7 application. If an error happens, I want to log the error back to my server. To handle this, I have created a WCF service operation. I want this operation to be REST ful so that I can later use it with iPhone and Android apps. Because I'm writing information to the database, I thought the POST method would be best. For this reason, I'm using WebInvoke. To do this, I'm using the following code:
[OperationContract]
[WebInvoke(UriTemplate = "/LogError/{message}/{stackTrace}", ResponseFormat = WebMessageFormat.Json)]
public void LogError(string message, string stackTrace)
{
// Write info to the database
}
From my WP7 app, I want to call this operaiton via a WebClient. My question is, how do I do that? I don't understand how to call the LogError operation and pass along the required data via the WebClient.
Thank you for your help!

If I am getting your Service method correctly, that method is not a POST method. You can just call that with a WebClient
WebClient wc = new WebClient()
Uri uri = new Uri("http://yourUri/LogError/ABC/XYZ"); //ABC is your message and XYZ is your stacktrace string.
wc.DownloadStringAsync(uri);
Or if you are thinking about real HTTP 'POST' then below might help.
You can use HttpWebRequest to do a POST on to any service which is accepting POST
This link may be helpful - WCF REST POST XML - The remote server returned an error: (400) Bad Request

Something along the lines:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://serveraddress/LogError/{message}/{stackTrace}");
If you would want to send additional information later on, you can do so with:
request.Method = "POST";
request.BeginGetRequestStream(new AsyncCallback(ExecuteAction), request);
And have a callback:
void ExecuteAction(IAsyncResult result)
{
HttpWebRequest request = (HttpWebRequest)result.AsyncState;
using (Stream s = request.EndGetRequestStream(result))
{
s.Write(data, 0, data.Length);
}
}
If there is a specific string response from the service, you might as well include the data in the WebClient and use DownloadStringAsync to get the response data.

For starters, I found a website that should help you get started with calling the service from WP7.
Try this and let me know what you think

Have a look at this post http://blog.ike.to/2011/02/02/wp7-application-crash-reporter/
It sounds like it will pretty much do what you need already, although you might want to tweak it to suit your own service interface.

Why would C# HttpWebRequest return 500 error on ResponseStream, but not with PHP?

I would be very grateful if anyone could help me out with this problem. I've got some C# code which reads in the contents of a web page for parsing later on. The code is:
private StringReader ReadInUrl(string url)
{
string result = string.Empty;
System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(url);
request.Method = "GET";
using (var stream = request.GetResponse().GetResponseStream())
using (var reader = new StreamReader(stream, Encoding.UTF8))
{
result = reader.ReadToEnd();
}
return new StringReader(result);
}
The code works fine with most pages, but throws a 'The remote server returned an error: (500) Internal Server Error.' with some pages. An example of a page that throws the error would be : http://www.thehut.com/blu-ray/harry-potter-collection-years-1-6/10061821.html
The thing that confuses me is that I can view the page fine using a webbrowser, and I can also grab the contents of the file using PHP fopen and fread, and then parse it in PHP.
I really need to be able to do this in C# and I'm stumped as to why it is happening. If any one could let me know why I can read in the page using PHP and not C#, and whether there is a setting in C# that could get round this issue? Any answers gratefully received!

The web site drops requests that doesn't specify a user agent. So you need to specify it. Also I would recommend you using WebClient instead of HttpWebRequest, HttpWebResponse, StreamReader, StringReader and company:
class Program
{
public static void Main()
{
using (var client = new WebClient())
{
client.Headers[HttpRequestHeader.UserAgent] = "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.13) Gecko/20101203 Firefox/3.6.13";
string result = client.DownloadString("http://www.thehut.com/blu-ray/harry-potter-collection-years-1-6/10061821.html");
Console.WriteLine(result);
}
}
}
it's kinda shorter and works.

I suspect that PHP is including some header that WebRequest doesn't include by default - and the server is failing to handle it. I've just reproduced this myself, and it really is an internal server error at thehut.com. Here's the server-side exception as far as it's shown in the HTML that's returned:
org.apache.jasper.JasperException: java.lang.NullPointerException
org.apache.jasper.servlet.JspServletWrapper.handleJspException(JspServletWrapper.java:486)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:416)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:342)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:267)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:968)
org.apache.jsp.hut.errors.error_jsp._jspService(error_jsp.java:71)
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:374)
org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:342)
org.apache.jasper.servlet.JspServlet.service(JspServlet.java:267)
javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
com.thehut.elysium.filter.SiteFilter.forwardToErrorPage(Unknown Source)
com.thehut.elysium.filter.SiteFilter.doFilter(Unknown Source)
com.thehut.elysium.filter.SlowRequestFilter.doFilter(Unknown Source)
com.thehut.elysium.filter.SetCharacterEncodingFilter.doFilter(Unknown Source)
Not terribly helpful, but it does basically confirm that it's a server-side issue - and it certainly sounds suspiciously like a header which the servlet assumes will be present.
You could try making the same request from your PHP code, and see what headers it uses (using Wireshark). Add those headers to the .NET WebRequest one at a time, and see what it needs before it starts working.

How to perform a fast web request in C#

I have a HTTP based API which I potentially need to call many times. The problem is that I can't get the request to take less than about 20 seconds, though the same request made through a browser is near instantaneous. The following code illustrates how I have implemented it so far.
WebRequest r = HttpWebRequest.Create("https://example.com/http/command?param=blabla");
var response = r.GetResponse();
One solution would be to make an asynchronous request but I would like to know why it takes so long and if I can avoid it. I have also tried using the WebClient class but I suspect it uses a WebRequest internally.
Update:
Running the following code took about 40 seconds in Release Mode (measured with Stopwatch):
WebRequest g = HttpWebRequest.Create("http://www.google.com");
var response = g.GetResponse();
I'm working at a university where there might be different things in the network configuration affecting the performance, but the direct use of the browser illustrates that it should be near instant.
Update 2:
I uploaded the code to a remote machine and it worked fine so the conclusion must be that the .NET code does something extra compared to the browser or it has problems resolving the address through the university network (proxy issues or something?!).

This problem is similar to another post on StackOverflow:
Stackoverflow-2519655(HttpWebrequest is extremely slow)
Most of the time the problem is the Proxy server property. You should set this property to null, otherwise the object will attempt to search for an appropriate proxy server to use before going directly to the source. Note: this property is turn on by default, so you have to explicitly tell the object not to perform this proxy search.
request.Proxy = null;
using (var response = (HttpWebResponse)request.GetResponse())
{
}

I was having the 30 second delay on 'first' attempt - JamesR's reference to the other post mentioning setting proxy to null solved it instantly!
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(_site.url);
request.Proxy = null; // <-- this is the good stuff
...
HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Does your site have an invalid SSL cert? Try adding this
ServicePointManager.ServerCertificateValidationCallback = new System.Net.Security.RemoteCertificateValidationCallback(AlwaysAccept);
//... somewhere AlwaysAccept is defined as:
using System.Security.Cryptography.X509Certificates;
using System.Net.Security;
public bool AlwaysAccept(object sender, X509Certificate certification, X509Chain chain, SslPolicyErrors sslPolicyErrors)
{
return true;
}

You don't close your Request. As soon as you hit the number of allowed connections, you have to wait for the earlier ones to time out. Try
using (var response = g.GetResponse())
{
// do stuff with your response
}