I have a windows service that calls a page after a certain interval of time. The page in turn creates some reports.
The problem is that the service stops doing anything after 2-3 calls. as in it calls the page for 2-3 times and then does not do any work though it shows that the service is running...i am using timers in my service..
please can someone help me with a solution here
thank you
the code:(where t1 is my timer)
protected override void OnStart(string[] args)
{
GetRecords();
t1.Elapsed += new ElapsedEventHandler(OnElapsedTime);
t1.Interval = //SomeTimeInterval
t1.Enabled = true;
t1.Start();
}
private void OnElapsedTime(object source, ElapsedEventArgs e)
{
try
{
GetRecords();
}
catch (Exception ex)
{
EventLog.WriteEntry(ex.Message);
}
}
public void GetRecords()
{
try
{
string ConnectionString = //Connection string from web.config
WebRequest Request = HttpWebRequest.Create(ConnectionString);
Request.Timeout = 100000000;
HttpWebResponse Response = (HttpWebResponse)Request.GetResponse();
}
catch (Exception ex)
{
}
}
Well, what does the code look like? WebClient is the easiest way to query a page:
string result;
using (WebClient client = new WebClient()) {
result = client.DownloadString(address);
}
// do something with `result`
The timer code might also be glitchy if it is stalling...
It's possible that HttpWebRequest will restrict the number of concurrent HTTP requests to a specific page or server, as is generally proper HTTP client practice.
The fact that you're not properly disposing your objects most likely means you are maintaining 2 or 3 connections to a specific page, each with large timout value, and HttpWebRequest is queueing or ignoring your requests until the first few complete (die from a client or server timeout, most likely the server in this case).
Add a 'finally' clause and dispose of your objects properly!
I think you're missing something about disposing your objects like StreamReader, WebRequest, etc.. You should dispose your expensive objects after using them.
possibly the way you are requesting athe page is throwing an unnhandled exception which leaves the service in an inoperable state.
Yes, we need code.
Marc's advice worked for me, in the context of a service
Using WebClient worked reliably, where WebRequest timed out.
#jscharf explanation looks as good as any to me.
Related
I'm writing a public-facing transaction processor. Naturally, we run on https:// and the payload carries all relevant detail so we'll only process legitimate transactions. However, as a public interface, any number of nefarious actors will no doubt be throwing shade at my server if for no other reason than to just be annoying.
When I detect such a request, is there anyway I can terminate processing at my end - not going to waste time on the transaction - but NOT send a response to the client? Basically, I'd like to force the nefarious clients into a timeout situation so that, if nothing else, it diminishes their capacity to annoy my server.
Here's the code:
public class Webhook : IHttpModule
{
/// <summary>
/// You will need to configure this module in the Web.config file of your
/// web and register it with IIS before being able to use it. For more information
/// see the following link: http://go.microsoft.com/?linkid=8101007
/// </summary>
private bool m_sslRequired = false;
#region IHttpModule Members
<snip...>
#endregion
private void OnBeginRequest(object sender, EventArgs e)
{
WriteTrace("Begin OnBeginRequest");
HttpContext ctx = HttpContext.Current;
try
{
string processor = ctx.Request.Params["p"];
if (processor != null && processor != "")
{
PluginProcessor(processor, ctx);
}
}
catch (Exception ex)
{
ctx.Response.StatusCode = 500;
ctx.Response.Write("ERROR");
}
ctx.ApplicationInstance.CompleteRequest();
WriteTrace("End OnBeginRequest");
}
private void PluginProcessor(string processor, HttpContext ctx)
{
string pluginSpec = AppConfig.GetAppSetting(processor.Trim().ToLower());
if (pluginSpec != "")
{
IWebhookProcessor proc = CreateProcessor(pluginSpec, ctx);
proc.Process(ctx);
}
}
private IWebhookProcessor CreateProcessor(string Processor, HttpContext ctx)
{
string assembly;
string typeName;
typeName = Processor.Substring(0, Processor.IndexOf(",")).Trim();
assembly = Path.Combine(ctx.Request.PhysicalApplicationPath, "bin", Processor.Substring(Processor.IndexOf(",") + 1).Trim());
var obj = Activator.CreateInstanceFrom(assembly, typeName);
return (Interfaces.IWebhookProcessor)obj.Unwrap();
}
}
So if the request doesn't map to a transaction handler, I'd like to 'hang' the client, but not in a way which will tie up resources on the server.
Thanks for your advice!
I think the best thing you can do is use HttpRequest.Abort(), which doesn't leave the client hanging, but it does immediately sever the TCP connection. Even the docs say it is for this kind of scenario:
You might use this method in response to an attack by a malicious HTTP client.
You would use it like this:
ctx.Request.Abort();
In a browser, you see a "connection reset" error.
Another option is to send back an unexpected HTTP status, like 400, or my personal favourite, 418.
Update: If you reaaallly want to make the client wait, you could implement your own HttpModule so that you can make an asynchronous BeginRequest event and then use Task.Delay().
The HttpModule class would look something like this:
public class AsyncHttpModule : IHttpModule {
public void Dispose() { }
public void Init(HttpApplication app) {
var wrapper = new EventHandlerTaskAsyncHelper(DoAsyncWork);
app.AddOnBeginRequestAsync(wrapper.BeginEventHandler, wrapper.EndEventHandler);
}
private async Task DoAsyncWork(object sender, EventArgs e) {
var app = (HttpApplication) sender;
var ctx = app.Context;
if (shouldDie) { //whatever your criteria is
await Task.Delay(60000); //wait for one minute
ctx.Request.Abort(); //kill the connection without replying
}
}
}
Then add the module in your web.config (replace the namespace with your app's namespace):
<system.webServer>
<modules>
<add name="AsyncHttpModule" type="MyNamespace.AsyncHttpModule" />
</modules>
</system.webServer>
Since this is asynchronous, it is not holding up a thread while it waits. Other requests that come in will use the same thread (I tested this).
However, it is still keeping the request context in memory, because the request is still in progress. So if they hit you with 1000+ requests, all of those 1000+ requests are held in memory for 60 seconds. Whereas if you just use HttpRequest.Abort() right away, those get removed from memory right away.
Hi I was making a crawler for a site. After about 3 hours of crawling, my app stopped on a WebException. below are my code in c#. client is predefined WebClient object that will be disposed every time gameDoc has already been processed. gameDoc is a HtmlDocument object (from HtmlAgilityPack)
while (retrygamedoc)
{
try
{
gameDoc.LoadHtml(client.DownloadString(url)); // this line caused the exception
retrygamedoc = false;
}
catch
{
client.Dispose();
client = new WebClient();
retrygamedoc = true;
Thread.Sleep(500);
}
}
I tried to use code below (to keep the webclient fresh) from this answer
while (retrygamedoc)
{
try
{
using (WebClient client2 = new WebClient())
{
gameDoc.LoadHtml(client2.DownloadString(url)); // this line cause the exception
retrygamedoc = false;
}
}
catch
{
retrygamedoc = true;
Thread.Sleep(500);
}
}
but the result is still the same. Then I use StreamReader and the result stays the same! below are my code using StreamReader.
while (retrygamedoc)
{
try
{
// using native to check the result
HttpWebRequest webreq = (HttpWebRequest)WebRequest.Create(url);
string responsestring = string.Empty;
HttpWebResponse response = (HttpWebResponse)webreq.GetResponse(); // this cause the exception
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
responsestring = reader.ReadToEnd();
}
gameDoc.LoadHtml(client.DownloadString(url));
retrygamedoc = false;
}
catch
{
retrygamedoc = true;
Thread.Sleep(500);
}
}
What should I do and check? I am so confused because I got am able to crawl on some pages, on the same site, then in about 1000 reasults, it cause the exception. the message from exception is only The request was aborted: The connection was closed unexpectedly. and the status is ConnectionClosed
PS. the app is a desktop form app.
update :
Now I am skipping the values and turned them to null so that the crawling can goes on. But if the data is really needed, I still have to update the crawling result manually, which is tiring because the result contains thousands of record. Please help me.
example :
it was like you have downloaded like about 1300 data from the website, then the application stopped saying The request was aborted: The connection was closed unexpectedly. while all your internet connection still on and on a good speed.
ConnectionClosed may indicate (and probably does) that the server you're downloading from is closing the connection. Perhaps it is noticing a large amount of requests from your client and is denying you additional service.
Since you can't control server-side shenanigans, I'd recommend you have some sort of logic to retry the download a bit later.
Got this error because it was returned as 404 from the server.
I am writing test harness to test a HTTP Post. Test case would send 8 http request using UploadValuesAsync in webclient class in 10 seconds interval. It sleeps 10 seconds after every 8 request. I am recording start time and end time of each request. When I compute the average response time. I am getting around 800 ms. But when I run this test case synchronously using UploadValues method in web client I am getting average response time 250 milliseconds. Can you tell me why is difference between these two methods? I was expecting the less response time in Aync but I did not get that.
Here is code that sends 8 requests async
var count = 0;
foreach (var nameValueCollection in requestCollections)
{
count++;
NameValueCollection collection = nameValueCollection;
PostToURL(collection,uri);
if (count % 8 == 0)
{
Thread.Sleep(TimeSpan.FromSeconds(10));
count = 0;
}
}
UPDATED
Here is code that sends 8 requests SYNC
public void PostToURLSync(NameValueCollection collection,Uri uri)
{
var response = new ServiceResponse
{
Response = "Not Started",
Request = string.Join(";", collection.Cast<string>()
.Select(col => String.Concat(col, "=", collection[col])).ToArray()),
ApplicationId = collection["ApplicationId"]
};
try
{
using (var transportType2 = new DerivedWebClient())
{
transportType2.Expect100Continue = false;
transportType2.Timeout = TimeSpan.FromMilliseconds(2000);
response.StartTime = DateTime.Now;
var responeByte = transportType2.UploadValues(uri, "POST", collection);
response.EndTime = DateTime.Now;
response.Response = Encoding.Default.GetString(responeByte);
}
}
catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
response.ResponseInMs = (int)response.EndTime.Subtract(response.StartTime).TotalMilliseconds;
responses.Add(response);
Console.WriteLine(response.ResponseInMs);
}
Here is the code that post to the HTTP URI
public void PostToURL(NameValueCollection collection,Uri uri)
{
var response = new ServiceResponse
{
Response = "Not Started",
Request = string.Join(";", collection.Cast<string>()
.Select(col => String.Concat(col, "=", collection[col])).ToArray()),
ApplicationId = collection["ApplicationId"]
};
try
{
using (var transportType2 = new DerivedWebClient())
{
transportType2.Expect100Continue = false;
transportType2.Timeout = TimeSpan.FromMilliseconds(2000);
response.StartTime = DateTime.Now;
transportType2.UploadValuesCompleted += new UploadValuesCompletedEventHandler(transportType2_UploadValuesCompleted);
transportType2.UploadValuesAsync(uri, "POST", collection,response);
}
}
catch (Exception exception)
{
Console.WriteLine(exception.ToString());
}
}
Here is the upload completed event
private void transportType2_UploadValuesCompleted(object sender, UploadValuesCompletedEventArgs e)
{
var now = DateTime.Now;
var response = (ServiceResponse)e.UserState;
response.EndTime = now;
response.ResponseInMs = (int) response.EndTime.Subtract(response.StartTime).TotalMilliseconds;
Console.WriteLine(response.ResponseInMs);
if (e.Error != null)
{
response.Response = e.Error.ToString();
}
else
if (e.Result != null && e.Result.Length > 0)
{
string downloadedData = Encoding.Default.GetString(e.Result);
response.Response = downloadedData;
}
//Recording response in Global variable
responses.Add(response);
}
One problem you're probably running into is that .NET, by default, will throttle outgoing HTTP connections to the limit (2 concurrent connections per remote host) that are mandated by the relevant RFC. Assuming 2 concurrent connections and 250ms per request, that means the response time for your first 2 requests will be 250ms, the second 2 will be 500ms, the third 750ms, and the last 1000ms. This would yield a 625ms average response time, which is not far from the 800ms you're seeing.
To remove the throttling, increase ServicePointManager.DefaultConnectionLimit to the maximum number of concurrent connections you want to support, and you should see your average response time go down alot.
A secondary problem may be that the server itself is slower handling multiple concurrent connections than handing one request at a time. Even once you unblock the throttling problem above, I'd expect each of the async requests to, on average, execute somewhat slower than if the server was only executing one request at a time. How much slower depends on how well the server is optimized for concurrent requests.
A final problem may be caused by test methodology. For example, if your test client is simulating a browser session by storing cookies and re-sending cookies with each request, that may run into problems with some servers that will serialize requests from a single user. This is often a simplification for server apps so they won't have to deal with locking cross-requests state like session state. If you're running into this problem, make sure that each WebClient sends different cookies to simulate different users.
I'm not saying that you're running into all three of these problems-- you might be only running into 1 or 2-- but these are the most likley culprits for the problem you're seeing.
As Justin said, I tried ServicePointManager.DefaultConnectionLimit but that did not fix the issue. I could not able reproduce other problems suggested by Justin. I am not sure how to reproduce them in first place.
What I did, I ran the same piece of code in peer machine that runs perfectly response time that I expected. The difference between the two machines is operating systems. Mine is running on Windows Server 2003 and other machine is running on Windows Server 2008.
As it worked on the other machines, I suspect that it might be one of the problem specified by Justin or could be server settings on 2003 or something else. I did not spend much time after that to dig this issue. As this is a test harness that we had low priority on this issue. We left off with no time further.
As I have no glue on what exactly fixed it, I am not accepting any answer other than this. Becuase at very least I know that switching to server 2008 fixed this issue.
i have this method:
private void sendSms(object url)
{
var Url = url.ToString();
webRequest = WebRequest.Create(Url);
// webRequest.BeginGetResponse(this.RespCallback, webRequest);
webResponse = webRequest.GetResponse();
// End the Asynchronous response.
var stream = new StreamReader(webResponse.GetResponseStream());
var response = stream.ReadToEnd().ToString();
if (response.Contains(Config.ValidResponse))
{
var queryString = HttpUtility.ParseQueryString(webRequest.RequestUri.Query);
OnMessageAccepted(this, new MessageAcceptedEventArgs(queryString["SN"], "n/a"));
}
else
{
OnMessageAccepted(this, new MessageAcceptedEventArgs("", "n/a"));
}
}
which i call inside a loop like this
While (true)
{
Send(url);
sleep(400);
}
Problem is after couples of hundreds of calls like 500 or 600 the performance of the calls gets slower and slower if i restart application it start so fast and good but then start slowing down ! i was wondering if there is any buffer or cache i should clear every now and then to make it stay fast ?
ps: i developed the server so im sure the server doesnt slow it down plus i tried that with different kind of server implementation that i developed and developed by others.
thanks in advance.
You need to dispose the response and response stream using using blocks.
Another question about Web proxy.
Here is my code:
IWebProxy Proxya = System.Net.WebRequest.GetSystemWebProxy();
Proxya.Credentials = CredentialCache.DefaultNetworkCredentials;
HttpWebRequest rqst = (HttpWebRequest)WebRequest.Create(targetServer);
rqst.Proxy = Proxya;
rqst.Timeout = 5000;
try
{
rqst.GetResponse();
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
This code hangs the first time it is called for a minute of two. After that on successive calls it works sometimes, but not others. It also never hits the catch block.
Now the weird part. If I add a MessageBox.Show(msg) call in the first section of code before the GetResponse() call this all will work every time without hanging. Here is an example:
try
{
// ========Here is where I make the call and get the response========
System.Windows.Forms.MessageBox.Show("Getting Response");
// ========This makes the whole thing work every time========
rqst.GetResponse();
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
I'm baffled about why it is behaving this way. I don't know if the timeout is not working (it's in milliseconds, not seconds, so should timeout after 5 seconds, right?...) or what is going on. The most confusing this is that the message box call makes it all work without hanging.
So any help and suggestions on what is happening is appreciated. These are the kind of bugs that drive me absolutely out of my mind.
EDIT and CORRECTION:
OK, so I've been testing this and the problem is caused when I try to download data from the URI that I am getting a response from. I am testing the connectivity using the GetResponse() method with a WebRequest, but am downloading the data with a WebClient. Here is the code for that:
public void LoadUpdateDataFromNet(string url, IWebProxy wProxy)
{
//Create web client
System.Net.WebClient webClnt = new System.Net.WebClient();
//set the proxy settings
webClnt.Proxy = wProxy;
webClnt.Credentials = wProxy.Credentials;
byte[] tempBytes;
//download the data and put it into a stream for reading
try
{
tempBytes = webClnt.DownloadData(url); // <--HERE IS WHERE IT HANGS
}
catch (WebException wex)
{
MessageBox.Show("NEW ERROR: " + wex.Message);
return;
}
//Code here that uses the downloaded data
}
The WebRequest and WebClient are both accessing the same URL which is a web path to an XML file and the proxy is the same one created in the method at the top of this post. I am testing to see if the created IWebProxy is valid for the specified path and file and then downloading the file.
The first piece of code I put above and this code using the WebClient are in separate classes and are called at different times, yet using a message box in the first bit of code still makes the whole thing run fine, which confuses me. Not sure what all is happening here or why message boxes and running/debugging in Visual Studio makes the program run OK. Suggestions?
So, I figured out the answer to the problem. The timeout for the we request is still 5 sec, but for some reason if it is not closed explicitly it makes consecutive web requests hang. Here is the code now:
IWebProxy Proxya = System.Net.WebRequest.GetSystemWebProxy();
//to get default proxy settings
Proxya.Credentials = CredentialCache.DefaultNetworkCredentials;
Uri targetserver = new Uri(targetAddress);
Uri proxyserver = Proxya.GetProxy(targetserver);
HttpWebRequest rqst = (HttpWebRequest)WebRequest.Create(targetserver);
rqst.Proxy = Proxya;
rqst.Timeout = 5000;
try
{
//Get response to check for valid proxy and then close it
WebResponse wResp = rqst.GetResponse();
//===================================================================
wResp.Close(); //HERE WAS THE PROBLEM. ADDING THIS CALL MAKES IT WORK
//===================================================================
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
Still not sure exactly how calling the message box was making everything work, but it doesn't really matter at this point. The whole thing works like a charm.