C# HttpWebRequest vs browser - c#

I'm trying to integrate with Shopping.com REST service.
so I wrote a simple (minimum code) C# application to retrieve the XML data using HttpWebRequest class. I've used StopWatch for bench-marking and the response time (Including Stream.ReadToEnd()) is something like 1300-1700 milliseconds.
it might sound good, but then I've tested the response time in Chrome browser with Fiddler and the response time was about 600-800.
I've read few articles. some suggested set HttpWebRequest.Proxy to null / WebRequest.DefaultWebProxy but it didn't make significant improvement.
Here the request-url:
http://sandbox.api.shopping.com/publisher/3.0/rest/GeneralSearch?apiKey=78b0db8a-0ee1-4939-a2f9-d3cd95ec0fcc&trackingId=7000610&keyword=nikon
So, what should I do to reach this response time?

Are you enabling gzip and deflate?
Also, the first time you execute a web request in your code, there is a warmup to load the necessary assemblies, initialize the service point, and establish the HTTP connection, so be sure to look at the time for the second and subsequent executions.
EDIT: Sorry you will still need to decode the results using a GZipStream or DeflateStream, but this at least gives you the idea. See here for more info: HttpWebRequest & Native GZip Compression
var timer = Stopwatch.StartNew();
var url = "http://sandbox.api.shopping.com/publisher/3.0/rest/GeneralSearch?apiKey=78b0db8a-0ee1-4939-a2f9-d3cd95ec0fcc&trackingId=7000610&keyword=nikon";
var webRequest = WebRequest.Create(url);
webRequest.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip, deflate");
using (var webResponse = webRequest.GetResponse())
using (var responseStream = webResponse.GetResponseStream())
using (var streamReader = new StreamReader(responseStream))
{
var content = streamReader.ReadToEnd();
}
var timeSpan = timer.Elapsed;
Console.WriteLine(timeSpan);

Related

C# WebRequest timeout on second call

The first time I call this function everything works perfect.
When I call it the second time I get an exception Host is not responding correclty
I tried everything for hours but I dont get it to work.
Is it the website or is something wrong with my code?
string url = "https://www.nseindia.com/live_market/dynaContent/live_watch/option_chain/optionKeys.jsp";
string website = "";
HttpWebRequest httpRequest = (HttpWebRequest)WebRequest.Create(url);
httpRequest.AutomaticDecompression = DecompressionMethods.GZip | DecompressionMethods.Deflate;
httpRequest.Timeout = 5000;
using (var response = (HttpWebResponse)httpRequest.GetResponse())
{
using (StreamReader Reader = new StreamReader(response.GetResponseStream()))
{
website = Reader.ReadToEnd();
}
}
httpRequest.Abort();
Please, on every question that you are going to do here, try to put as maximum of details as possible. It will become easier for us to help you on your problems. Especially the exceptions that you are having doing the development.
About your code, the implementation of that is not recommended for Microsoft. Looking at the docs https://learn.microsoft.com/en-us/dotnet/api/system.net.httpwebrequest?view=netframework-4.8 there is a part that says: We don't recommend that you use HttpWebRequest for new development. Instead, use the System.Net.Http.HttpClient class.
Try to change your implementation to use the class HttpClient, and see if the error is going to occur again.

Measure HTTP Response Time from a REST Client

I am trying to create a REST Client to measure the time taken for a HTTP Request to execute, that is to measure the time between the Client's request and the response from the server after it reaches the client.(There are other easier approaches to find this like fiddler etc, but I need this anyway). I am following Microsoft's example provided here:
https://msdn.microsoft.com/en-us/library/debx8sh9%28v=vs.110%29.aspx
should I just note the time when the response is returned?
// Send the request:
DateTime T = System.DateTime.UtcNow; //--> Note the initial Time
HttpWebResponse response = (HttpWebResponse) req.GetResponse();
TimeSpan TT = System.DateTime.UtcNow - T; //--> Note the Time Difference
or should I be calculating the time after the response stream is read:
DateTime T = System.DateTime.UtcNow;//--> Note the initial Time
HttpWebResponse response = (HttpWebResponse) req.GetResponse();
// Get the stream containing content returned by the server.
dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
TimeSpan TT = System.DateTime.UtcNow - T;//--> Note the Time Difference
I am just not sure about the exact lines where the Request is made to the server and the Response from the Server are available to the Client.
According to the msdn documentation, GetResponse() will send the request to the server and retreive the response.
If you're just interested in timing the call, then the first option would be your best bet.

Cancel HTTPWebRequest when server goes down

I have created a Windows Service that calls a API (that returns JSON) with HTTPWebRequest.
The API doesn't return anything until it has something to "deliver". So I set the timeout quite high and lets the request wait until it receivs a response.
The problem is that when I test to turn off or disconnect the server running the API. The HTTPWebRequest doesn't stop the request. So I can't know if the API server has gone down.
The request code:
HttpWebRequest Request = (HttpWebRequest)HttpWebRequest.Create(Url);
Request.Method = "POST";
Request.ContentType = "application/x-www-form-urlencoded";
Request.ContentLength = PostData.Length;
Request.Timeout = Timeout;
Request.KeepAlive = true;
using (StreamWriter sw = new StreamWriter(Request.GetRequestStream()))
{
sw.Write(PostData);
}
using (HttpWebResponse Response = (HttpWebResponse)Request.GetResponse())
{
using (StreamReader sr = new StreamReader(Response.GetResponseStream()))
{
ResponseText = sr.ReadToEnd();
}
}
Is there anyway to "break" the request when the requested server goes down?
I have tried using the webbrowser to call the API server and after a while disconnect it and that return an error to the webpage.
you could use a background worker only cecking if the server is online. It has some disatvantages but may work fine.
It is always good to keep the requests asynchronous (See the BeginXXX methods in HttpWebRequest - http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.aspx).
Using the asynchronous APIs ensures that you are not blocked until you get a response from the server.
In addition to using the asynchronous APIs, you can have a heart-beat requests (that could be just a HEAD HTTP request to a ping service on the server, which returns an empty body and HTTP 200 status), to keep track that the server is alive. If this request times out, then server is not alive - in which case, you can cancel / just 'forget' that the request has been made.

Call to REST web service results in response of error 503 but not with a tool like Poster

I am trying to send an xml document to a REST web service. Using a tool like Poster, the call works fine (With the xml in the body of the "content"), but in my code, I get error 503 server unavailable.
Link to Poster: https://addons.mozilla.org/en-US/firefox/addon/poster/
My code is as follows (this is a test-harness so no error handling etc):
string s = "";
using (StreamReader sr = new StreamReader(#"c:\users\dev.admin\documents\visual studio 2010\Projects\WindowsFormsApplication2\WindowsFormsApplication2\XMLFile1.xml"))
{
s = sr.ReadToEnd();
}
string url = FULL_URL_WITH_PARAMETERS; // SAME URL AS USED IN POSTER
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.Method = "POST";
request.ContentType = "text/xml;charset=UTF-8";
string data = s;
Stream postStream = null;
using (StreamWriter requestStream = new StreamWriter(request.GetRequestStream()))
{
requestStream.Write(data);
}
HttpWebResponse pervasiveResponse = (HttpWebResponse)request.GetResponse();
StreamReader sr1 = new StreamReader(pervasiveResponse.GetResponseStream(), System.Text.Encoding.Default);
string backstr = sr1.ReadToEnd();
}
else
{
throw new ArgumentNullException();
}
REST Starter Kit? WCF Web API (Glenn Block's project, now part of AppFabric int he 4.5 time frame? (although I believe there will be a standalone))? Roll your own joint?
Regardless, I think working with binary is the correct method to go. For that reason, I would start with something like this article. Not because I tried the code and think it is a great article, but merely because the article shows sending XML and I don't have time to find the client I wrote a few months ago. ;-)
If I can break free long enough to find my code, I will post a sample.
Oh, another tool you should look at is SoapUI. No, it is not just for testing SOAP, as it supports REST (and other methodologies) as well as SOAP. One nice benefit of using SoapUI is the ability to move the tests over to LoadUI and stress testing a service. If you are merely consuming services, it might not have the greatest value, of course.

Can I send an empty HTTP POST WebRequest object from C# to IIS?

Do I need to just slap some random garbage data in a WebRequest object to get by the HTTP status code 411 restriction on IIS?
I have an HttpPost action method in an MVC 3 app that consumes a POST request with all the relevant information passed in the querystring (no body needed).
[HttpPost] public ActionResult SignUp(string email) { ... }
It worked great from Visual Studio's built in web host, Cassini. Unfortunately, once the MVC code was live on IIS [7.5 on 2008 R2], the server is pitching back an HTTP error code when I hit it from my outside C# form app.
The remote server returned an error:
(411) Length Required.
Here is the calling code:
WebRequest request = WebRequest.Create("http://somewhere.com/signup/?email=a#b.com");
request.Method = "POST";
using (WebResponse response = request.GetResponse())
using (Stream responseStream = response.GetResponseStream())
using (StreamReader responseReader = new StreamReader(responseStream)) {
// Do something with responseReader.ReadToEnd();
}
Turns out you can get this to go through by simply slapping an empty content length on the request before you send it.
WebRequest request = WebRequest.Create("http://somewhere.com/signup/?email=a#b.com");
request.Method = "POST";
request.ContentLength = 0;
Not sure how explicitly giving an empty length vs. implying one makes a difference, but IIS was happy after I did. There are probably other ways around this, but this seems simple enough.
I believe you are required to set a Content-Length header anytime you post a request to a web server:
http://msdn.microsoft.com/en-us/library/system.web.httprequest.contentlength.aspx
You could try a GET request to test it.

Categories