We Keep Coding

Retrieve response time of a web request within a catch

First of all, thanks for the help you can give me.
Okay, first, excuse my little knowledge, I'm learning, so forgive my ignorance.
I have a method that calls a service, and it is assumed that if any response other than the range of 200 is received, it goes to catch. Well, I need, within that catch, to retrieve from the header (or somewhere) of that response the time it took the service to respond.
I can't use timer or things like that (it was what I first thought of).
The thing is, I don't know how to recover this data from the exception.
Thank you very much!
{
try
{
// Reading the http response
using (HttpWebResponse webResponse = httpWebRequest.GetResponse () as HttpWebResponse)
{
///// Call to endpoint
}
}
catch (WebException ex)
{
//// Here I need to retrieve the time it took for the service to respond (with the error)
}
return response;
} ```

Adding an example to what kshkarin mentioned:
var watch = System.Diagnostics.Stopwatch.StartNew();
try
{
var request = WebRequest.Create("http://www.google.com");
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream answer = response.GetResponseStream())
{
// do something
watch.Stop();
Console.WriteLine($"Success at {watch.ElapsedMilliseconds}");
}
}
}
catch (WebException e)
{
// If we got here, it was a timeout exception.
watch.Stop();
Console.WriteLine($"Error occurred at {watch.ElapsedMilliseconds} \n {e}");
}

Indeed you must be set timeout in web request and check this with TimeoutException
if you have some scenario you can check this with simple DateTime
set DateTime.Now and get distance time

How to detect HTTPRequest Failure

I am sending 100000 Requests and in order to check if all the requests has been sent successfully , I have developed a simple web page that counts the number of requests that has been sent.
The problem is this that the receiver counts less than 50000 and I also can not detect which one of them has been failed in order to send them again as the sender gets statusscode=OK for all of them and also no exception has detected.
I also tried it after removing webReq.Method="HEAD" but had no effect.
Any hints is appreciated.
Here is the sender's code:
try
{
var content = new MemoryStream();
var webReq = (HttpWebRequest)WebRequest.Create(url);
webReq.Method = "HEAD";
using (WebResponse response = await webReq.GetResponseAsync())
{
HttpWebResponse res = (HttpWebResponse)response;
if (res.StatusCode != HttpStatusCode.OK)
{
UnsuccessfulURLsPhase1.Add(url);
}
}
}
catch (Exception e)
{
UnsuccessfulURLsPhase1.Add(url);
}
This is receiver's code:
protected void Page_Load(object sender, EventArgs e)
{
try
{
if (!IsPostBack)
{
counter1++;
txtCounter.Text = counter1.ToString();
}
}
catch (Exception ex)
{
Debug.WriteLine("\nException raised!");
Debug.WriteLine("Source :{0} ", ex.Source);
Debug.WriteLine("Message :{0} ", ex.Message);
}
}

You're sending these out as quickly as possible? IANA suggests (and recent editions of Windows respect) using the range 49152 to 65535 for ephemeral ports, which are ports that are reserved to receive the reply from IP socket connections. This means that there are 16383 ports available, each of which must be left in a TIME_WAIT state for (IIRC) 120 seconds after the connection is closed.
In perfect conditions (and with routing equipment that can sustain thousands of simultaneous connections... a cheap SOHO router will probably overheat and become unreliable as it runs out of memory), you're going to be limited to a maximum of around 16000 requests every two minutes.
In practice, HttpWebRequest (and therefore WebClient) will maintain only a specific number of connections to a specific host, and pipeline requests over those connections, so without tweaking ServicePointManager, or the ServicePoint associated with the host you're trying to hit, you're going to have an awful amount of queueing to squeeze 100000 requests through these connections. It's likely that you'll hit timeouts somewhere down this path.

Your Page_Load is swallowing the exceptions so the server is always returning 200, OK.
You have to let the exception be thrown or you have to explicitly set the response status when an error occurs.
Found in here How to send a Status Code 500 in ASP.Net and still write to the response?
that TrySkipIisCustomErrors should be set.
Something like
protected void Page_Load(object sender, EventArgs e)
{
try
{
if (!IsPostBack)
{
counter1++;
txtCounter.Text = counter1.ToString();
}
}
catch (Exception ex)
{
Debug.WriteLine("\nException raised!");
Debug.WriteLine("Source :{0} ", ex.Source);
Debug.WriteLine("Message :{0} ", ex.Message);
// Raise the exception
// throw;
// or assign the correct status and status code
Response.Clear();
Response.TrySkipIisCustomErrors = true
Response.ContentType = "text/plain";
Response.StatusCode = (int)HttpStatusCode.InternalServerError;
Response.Write(ex.Message);
// Send the output to the client.
Response.Flush();
}
}
(Hope it helps, its been a long time since I have done something in WebForms :S )

Keep trying to talk to server when the Internet is down

So my application is exchanging request/responses with a server (no problems), until the internet connection dies for a couple of seconds, then comes back. Then a code like this:
response = (HttpWebResponse)request.GetResponse();
will throw an exception, with a status like ReceiveFailure, ConnectFailure, KeepAliveFailure etc.
Now, it's quite important that if the internet connection comes back, I am able to continue communicating with the server, otherwise I'd have to start again from the beginning and that will take a long time.
How would you go about resuming this communication when the internet is back?
At the moment, I keep on checking for a possibility to communicate with the server, until it is possible (at least theoretically). My code attempt looks like this:
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
// We have a problem receiving stuff from the server.
// We'll keep on trying for a while
if (ex.Status == WebExceptionStatus.ReceiveFailure ||
ex.Status == WebExceptionStatus.ConnectFailure ||
ex.Status == WebExceptionStatus.KeepAliveFailure)
{
bool stillNoInternet = true;
// keep trying to talk to the server
while (stillNoInternet)
{
try
{
response = (HttpWebResponse)request.GetResponse();
stillNoInternet = false;
}
catch
{
stillNoInternet = true;
}
}
}
}
However, the problem is that the second try-catch statement keeps throwing an exception even when the internet is back.
What am I doing wrong? Is there another way to go about fixing this?
Thanks!

You should recreate the request each time, and you should execute the retries in a loop with a wait between each retry. The wait time should progressively increase with each failure.
E.g.
ExecuteWithRetry (delegate {
// retry the whole connection attempt each time
HttpWebRequest request = ...;
response = request.GetResponse();
...
});
private void ExecuteWithRetry (Action action) {
// Use a maximum count, we don't want to loop forever
// Alternativly, you could use a time based limit (eg, try for up to 30 minutes)
const int maxRetries = 5;
bool done = false;
int attempts = 0;
while (!done) {
attempts++;
try {
action ();
done = true;
} catch (WebException ex) {
if (!IsRetryable (ex)) {
throw;
}
if (attempts >= maxRetries) {
throw;
}
// Back-off and retry a bit later, don't just repeatedly hammer the connection
Thread.Sleep (SleepTime (attempts));
}
}
}
private int SleepTime (int retryCount) {
// I just made these times up, chose correct values depending on your needs.
// Progressivly increase the wait time as the number of attempts increase.
switch (retryCount) {
case 0: return 0;
case 1: return 1000;
case 2: return 5000;
case 3: return 10000;
default: return 30000;
}
}
private bool IsRetryable (WebException ex) {
return
ex.Status == WebExceptionStatus.ReceiveFailure ||
ex.Status == WebExceptionStatus.ConnectFailure ||
ex.Status == WebExceptionStatus.KeepAliveFailure;
}

I think what you are trying to do is this:
HttpWebResponse RetryGetResponse(HttpWebRequest request)
{
while (true)
{
try
{
return (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
if (ex.Status != WebExceptionStatus.ReceiveFailure &&
ex.Status != WebExceptionStatus.ConnectFailure &&
ex.Status != WebExceptionStatus.KeepAliveFailure)
{
throw;
}
}
}
}
When you want to retry something on failure then instead of thinking of this as something that you want to do when something fails, think of it instead as looping until you succeed. (or a failure that you don't want to retry on). The above will keep on retrying until either a response is returned or a different exception is thrown.
It would also be a good idea to introduce a maximum retry limit of some sort (for example stop retrying after 1 hour).

If it's still doing it when you get the connection back - then my guess is that it's simply returning the same result again.
You might want to to try recreating the request anew each time, other than that I don't see much wrong with the code or logic. Apart from the fact that you're forever blocking this thread. But then that might be okay if this is, in itself, a worker thread.

Unable to browse in all browsers to all sites after running threaded application

I have a weird problem and I am not sure where is it coming from.
Started last night when I started my application.
I have the following code:
DateTime startTime = DateTime.Now;
WebRequest req = (WebRequest)HttpWebRequest.Create("http://" + server + url + action);
WebResponse res = req.GetResponse();
try
{
using (StreamReader reader = new StreamReader(res.GetResponseStream()))
{
string response = reader.ReadToEnd();
TestInfo.CheckMsg retMsg;
// individual Test
retMsg = indFunc(req.RequestUri.ToString(), response);
printMessage(retMsg, req, res, startTime);
if (retMsg.SeverityLevel > TestInfo.CheckMsg.Severity.Warning)
return;
// group Test
retMsg = groupFunc(relatedGroup, req.RequestUri.ToString(), response);
printMessage(retMsg, req, res, startTime);
if (retMsg.SeverityLevel > TestInfo.CheckMsg.Severity.Warning)
return;
// common Test
retMsg = commonFunc(req.RequestUri.ToString(), response);
printMessage(retMsg, req, res, startTime);
if (retMsg.SeverityLevel > TestInfo.CheckMsg.Severity.Warning)
return;
reader.Close();
};
}
catch (Exception e)
{
Console.WriteLine("EXCEPTION WAS THROWN!!!!! " + e.Message);
}
finally
{
res.Close();
}
This function runs on several threads.
After several application debugging this code fails to connect claiming UNDERLYING CONNECTION WAS CLOSED.
when trying to browse in my computer (WIN XP) in all browsers to all sites I get either blank page or in chrome I get that the response from the server was empty.
Http only is screwed while other TCP connections work!
So I guess the problem might be in my code (maybe something is not released???)
Let me know what do you think of the code.
thanks

One thing I do see wrong is that you need to move
WebResponse res = req.GetResponse();
Into your try block bc it can throw an exception(MSDN). I would also add a null check to:
if(res != null){
res.Close()
}

Different performance between Java and c# code when testing URL

When running the following Java code, I get very accurate and consistent results in determining if the web page I'm testing is up.
protected synchronized boolean checkUrl(HttpURLConnection connection){
boolean error = false;
//HttpURLConnection connection = null;
GregorianCalendar calendar = new GregorianCalendar();
try{
if(connection != null){
connection.connect();
//200 is the expected HTTP_OK response
error = processResponseCode(connection.getResponseCode());
connection.disconnect();
} else{
error = false;
}
}catch(java.net.UnknownHostException uhe){
... }
catch(Exception e){
... }
return error;
}
The closest match to the Java pattern in c# has much higher results of false positives (mostly due to timeouts - which has a default period of 100000ms).
protected bool connectedToUrl = false;
response = null;
HttpWebRequest webreq = (HttpWebRequest)WebRequest.Create(this.getUri());
webreq.Credentials = CredentialCache.DefaultCredentials;
WebResponse res = null;// webreq.GetResponse();
try
{
WebRequest request = WebRequest.Create(this.getUri()) as WebRequest;
request.Credentials = CredentialCache.DefaultCredentials;
if (request != null)
{
// Get response
res = webreq.GetResponse();
connectedToUrl = processResponseCode(res);
}
else
{
logger.Fatal(getFatalMessage());
string error = string.Empty;
}
}
catch (Exception e)
{
throw e;
}
return connectedToUrl;
}
I have tried various patterns in c# to match the effectiveness of the quoted Java code, to no avail.
Any ideas?

I believe this is because you're not closing any of the request objects.

Simply change this:
res = webreq.GetResponse();
connectedToUrl = processResponseCode(res);
to
using (WebResponse res = webreq.GetResponse())
{
connectedToUrl = processResponseCode(res);
}
(Remove the declaration from earlier.)
Until you haven't closed/disposed the response (or it's been finalized), it's holding onto the connection. You can only have a certain number (2 by default, I believe) of connections to any one host at a time, hence the timeouts. When you dispose the response, it allows another request to use the same connection.

Also this:
catch (Exception e)
{
throw e;
}
Does nothing but destroy the stack trace on an exception that's been bubbled upwards. If you have error handling elsewhere in your code I suggest removing the try catch block. Otherwise you should log the exception and move on. Don't just catch it to throw it.

I think you're missing the GregorianCalendar in the C# version :-)
Why do you have two Request Objects in the C# version?