AsyncHttpWebRequest (web page title retrieve program) - c#

I'm trying to create a program that will retrieve page titles given a url. I've written code that works when I'm not using a AsyncCallback, but when I use a AsyncCallback the code doesn't seem to work.
public void GetWebPageTitle(string URL)
{
// make request for web page
HttpWebRequest myWebRequest = (HttpWebRequest)HttpWebRequest.Create(URL);
myWebRequest.Method = "GET";
myWebRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), myWebRequest);
zConsole.WriteLine("Beginning HttpWebRequest for: " + URL);
}
void FinishWebRequest(IAsyncResult result)
{
zConsole.WriteLine("...");
string title = "Unknown";
//Code under here doesnt get extcuted
HttpWebResponse myWebResponse = (HttpWebResponse)((HttpWebRequest)result.AsyncState).EndGetResponse(result);
StreamReader myWebSource = new StreamReader(myWebResponse.GetResponseStream());
string source = "";
source = myWebSource.ReadToEnd();
myWebResponse.Close();
title = Regex.Match(source, #"\<title\b[^>]*\>\s*(?<Title>[\s\S]*?)\</title\>", RegexOptions.IgnoreCase).Groups["Title"].Value;
zConsole.WriteLine(title);
}
Thanks.

I think, the problem is, your program ends, before async result is returned.
The main thread after doing Console.Writeline dies.
Rest looks okay. BeginGetResponse at MSDN

Put a try/catch block around the code inside the callback and see if anything in there is throwing an exception.
Otherwise some more details would be useful. When you say that the code doesn't get executed are you actually stepping through the code/using breakpoints or are you assuming this is the case based on your console output? Is this request being made from the main window thread of your application?

Related

how to do multitasking from same code at the same time?

Sorry if the title is not clear or correct, dont know what title should i put. Please correct if wrong.
I have this code to download images from IP camera and it can download the images.The problem is how can i do the images downloading process at the same time for all cameras if i have two or more cameras?
private void GetImage()
{
string IP1 = "example.IPcam1.com:81/snapshot.cgi;
string IP2 = "example.IPcam2.com:81/snapshot.cgi;
.
.
.
string IPn = "example.IPcamn.com:81/snapshot.cgi";
for (int i = 0; i < 10; i++)
{
string ImagePath = Server.MapPath("~\\Videos\\liveRecording2\\") + string.Format("{0}", i, i + 1) + ".jpeg";
string sourceURL = ip;
WebRequest req = (WebRequest)WebRequest.Create(sourceURL);
req.Credentials = new NetworkCredential("user", "password");
WebResponse resp = req.GetResponse();
Stream stream = resp.GetResponseStream();
Bitmap bmp = (Bitmap)Bitmap.FromStream(stream);
bmp.Save(ImagePath);
}
}
You should not run long-running code like that from an ASP.NET application. They are meant to simply respond to requests.
You should place this code in a service (Windows Services are easy), and control the service through a WCF service running inside of it.
You're also going to get into trouble because you don't have your WebResponse and Stream in using blocks.
There are several methods that will depend on how you want to report feedback to the user. It all comes down to multi-threading.
Here is one example, using the ThreadPool. Note that this is missing a bunch of error checking throughout... It is here as an example of how to use the ThreadPool, not as a robust application:
private Dictionary<String, String> _cameras = new Dictionary<String, String> {
{ "http://example.IPcam1.com:81/snapshot.cgi", "/some/path/for/image1.jpg" },
{ "http://example.IPcam2.com:81/snapshot.cgi", "/some/other/path/image2.jpg" },
};
public void DoImageDownload() {
int finished = 0;
foreach (KeyValuePair<String, String> pair in _cameras) {
ThreadPool.QueueUserWorkItem(delegate {
BeginDownload(pair.Key, pair.Value);
finished++;
});
}
while (finished < _cameras.Count) {
Thread.Sleep(1000); // sleep 1 second
}
}
private void BeginDownload(String src, String dest) {
WebRequest req = (WebRequest) WebRequest.Create(src);
req.Credentials = new NetworkCredential("username", "password");
WebResponse resp = req.GetResponse();
Stream input = resp.GetResponseStream();
using (Stream output = File.Create(dest)) {
input.CopyTo(output);
}
}
This example simply takes the work you are doing in the for loop and off-loads it to the thread pool for processing. The DoImageDownload method will return very quickly, as it is not doing much actual work.
Depending on your use case, you may need a mechanism to wait for the images to finish downloading from the caller of DoImageDownload. A common approach would be the use of event callbacks at the end of BeginDownload to notify when the download is complete. I have put a simple while loop here that will wait until the images finish... Of course, this needs error checking in case images are missing or the delegate never returns.
Be sure to add your error checking throughout... Hopefully this gives you a place to start.

HttpWebRequest GetResponse. How to wait for that response?

I'm sending some SMS using HttpWebRequest. But some times the messages appear to send double, three times and more reviewing all the proccess appear to be ok.
The only thing i think is HttpWebRequest is working asynchronus.
here is my code,
public bool sendSmsGateway(string tcDestino, string tcMensaje, string tcTitulo = "")
{
bool lReturn = false;
string contenido = HttpUtility.UrlEncode(tcMensaje);
string lcTitulo = "SMS Sender";
if (!String.IsNullOrEmpty(tcTitulo))
{
lcTitulo = tcTitulo;
}
lcTitulo = HttpUtility.UrlEncode(lcTitulo);
string fechaEnvio = DateTime.Now.ToString("s");
string url = string.Format(StrSMSGatewayURL, StrSMSGatewayUsuario, StrSMSGatewayPassword, tcDestino, contenido, lcTitulo, fechaEnvio);
HttpWebRequest enviar = (HttpWebRequest)WebRequest.Create(url);
string respuesta = new StreamReader(enviar.GetResponse().GetResponseStream()).ReadToEnd();
StrSMSGatewayRespuesta = respuesta.Trim();
if (StrSMSGatewayRespuesta == StrSMSGatewayRespuestaOk.Trim())
{
lReturn = true;
}
return lReturn;
}
this code runs in loop routine that send the message and mark it as sent when:
string respuesta = new StreamReader(enviar.GetResponse().GetResponseStream()).ReadToEnd();
return the propper code.
My question is.. there is a way to stop the proccess, or force the code to wait until the (httpWebRequest send the message and get the response) or (timeout succeed)
HttpWebRequest.GetResponse() is a synchronous call, so you don't need to have a different mechanism for blocking the current thread. If you were using the asynchronous method HttpWebRequest.BeginGetResponse(), then you would need to have another mechanism to wait for the response to be asynchronously received.
If you're sending multiple SMS messages, then this is probably caused by not receiving the correct response code when you get StrSMSGatewayRespuesta. If you get the wrong code but the message was actually sent, then the message is likely to be sent again since you're marking it as not sent.

C# How to stop a method if it takes longer than 2 seconds?

Following program will connect to the web and get html content of “msnbc.com” webpage and print out the result. If it takes longer than 2 seconds to get data from the webpage, I want my method to stop working and return. Can you please tell me how can I do this with an example?
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
gethtml();
MessageBox.Show("End of program");
}
public void gethtml()
{
HttpWebRequest WebRequestObject = (HttpWebRequest)HttpWebRequest.Create("http://msnbc.com/");
WebResponse Response = WebRequestObject.GetResponse();
Stream WebStream = Response.GetResponseStream();
StreamReader Reader = new StreamReader(WebStream);
string webcontent = Reader.ReadToEnd();
MessageBox.Show(webcontent);
}
}
Two seconds is far too long to block the UI. You should only block the UI if you are planning on getting the result in, say fifty milliseconds or less.
Read this article on how to do a web request without blocking the UI:
http://www.developerfusion.com/code/4654/asynchronous-httpwebrequest/
Note that this will all be much easier in C# 5, which is in beta release at present. In C# 5 you can simply use the await operator to asynchronously await the result of the task. If you would like to see how this sort of thing will work in C# 5, see:
http://msdn.microsoft.com/en-us/async
Set the Timeout property of your WebRequest object. Documentation
MSDN Example:
// Create a new WebRequest Object to the mentioned URL.
WebRequest myWebRequest=WebRequest.Create("http://www.contoso.com");
Console.WriteLine("\nThe Timeout time of the request before setting is : {0} milliseconds",myWebRequest.Timeout);
// Set the 'Timeout' property in Milliseconds.
myWebRequest.Timeout=10000;
// This request will throw a WebException if it reaches the timeout limit before it is able to fetch the resource.
WebResponse myWebResponse=myWebRequest.GetResponse();
As stated above .Timeout
public void gethtml()
{
HttpWebRequest WebRequestObject = (HttpWebRequest)HttpWebRequest.Create("http://msnbc.com/");
WebRequestObject.Timeout = (System.Int32)TimeSpan.FromSeconds(2).TotalMilliseconds;
try
{
WebResponse Response = WebRequestObject.GetResponse();
Stream WebStream = Response.GetResponseStream();
StreamReader Reader = new StreamReader(WebStream);
string webcontent = Reader.ReadToEnd();
MessageBox.Show(webcontent);
}
catch (System.Net.WebException E)
{
MessageBox.Show("Fail");
}
}
You can use the TimeOut property on HttpWebRequest
Consider switching to asynchronous downloading of the content. You will stop blocking UI thread and will be able to handle multiple requests easily. You will be able to increase timeout significantly without impact on UI, and can decide upon receiving response if you still want to fetch data.

WebRequests/Responses Slowing down after many calls

i have this method:
private void sendSms(object url)
{
var Url = url.ToString();
webRequest = WebRequest.Create(Url);
// webRequest.BeginGetResponse(this.RespCallback, webRequest);
webResponse = webRequest.GetResponse();
// End the Asynchronous response.
var stream = new StreamReader(webResponse.GetResponseStream());
var response = stream.ReadToEnd().ToString();
if (response.Contains(Config.ValidResponse))
{
var queryString = HttpUtility.ParseQueryString(webRequest.RequestUri.Query);
OnMessageAccepted(this, new MessageAcceptedEventArgs(queryString["SN"], "n/a"));
}
else
{
OnMessageAccepted(this, new MessageAcceptedEventArgs("", "n/a"));
}
}
which i call inside a loop like this
While (true)
{
Send(url);
sleep(400);
}
Problem is after couples of hundreds of calls like 500 or 600 the performance of the calls gets slower and slower if i restart application it start so fast and good but then start slowing down ! i was wondering if there is any buffer or cache i should clear every now and then to make it stay fast ?
ps: i developed the server so im sure the server doesnt slow it down plus i tried that with different kind of server implementation that i developed and developed by others.
thanks in advance.
You need to dispose the response and response stream using using blocks.

Why does Http Web Request and IWebProxy work at weird times

Another question about Web proxy.
Here is my code:
IWebProxy Proxya = System.Net.WebRequest.GetSystemWebProxy();
Proxya.Credentials = CredentialCache.DefaultNetworkCredentials;
HttpWebRequest rqst = (HttpWebRequest)WebRequest.Create(targetServer);
rqst.Proxy = Proxya;
rqst.Timeout = 5000;
try
{
rqst.GetResponse();
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
This code hangs the first time it is called for a minute of two. After that on successive calls it works sometimes, but not others. It also never hits the catch block.
Now the weird part. If I add a MessageBox.Show(msg) call in the first section of code before the GetResponse() call this all will work every time without hanging. Here is an example:
try
{
// ========Here is where I make the call and get the response========
System.Windows.Forms.MessageBox.Show("Getting Response");
// ========This makes the whole thing work every time========
rqst.GetResponse();
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
I'm baffled about why it is behaving this way. I don't know if the timeout is not working (it's in milliseconds, not seconds, so should timeout after 5 seconds, right?...) or what is going on. The most confusing this is that the message box call makes it all work without hanging.
So any help and suggestions on what is happening is appreciated. These are the kind of bugs that drive me absolutely out of my mind.
EDIT and CORRECTION:
OK, so I've been testing this and the problem is caused when I try to download data from the URI that I am getting a response from. I am testing the connectivity using the GetResponse() method with a WebRequest, but am downloading the data with a WebClient. Here is the code for that:
public void LoadUpdateDataFromNet(string url, IWebProxy wProxy)
{
//Create web client
System.Net.WebClient webClnt = new System.Net.WebClient();
//set the proxy settings
webClnt.Proxy = wProxy;
webClnt.Credentials = wProxy.Credentials;
byte[] tempBytes;
//download the data and put it into a stream for reading
try
{
tempBytes = webClnt.DownloadData(url); // <--HERE IS WHERE IT HANGS
}
catch (WebException wex)
{
MessageBox.Show("NEW ERROR: " + wex.Message);
return;
}
//Code here that uses the downloaded data
}
The WebRequest and WebClient are both accessing the same URL which is a web path to an XML file and the proxy is the same one created in the method at the top of this post. I am testing to see if the created IWebProxy is valid for the specified path and file and then downloading the file.
The first piece of code I put above and this code using the WebClient are in separate classes and are called at different times, yet using a message box in the first bit of code still makes the whole thing run fine, which confuses me. Not sure what all is happening here or why message boxes and running/debugging in Visual Studio makes the program run OK. Suggestions?
So, I figured out the answer to the problem. The timeout for the we request is still 5 sec, but for some reason if it is not closed explicitly it makes consecutive web requests hang. Here is the code now:
IWebProxy Proxya = System.Net.WebRequest.GetSystemWebProxy();
//to get default proxy settings
Proxya.Credentials = CredentialCache.DefaultNetworkCredentials;
Uri targetserver = new Uri(targetAddress);
Uri proxyserver = Proxya.GetProxy(targetserver);
HttpWebRequest rqst = (HttpWebRequest)WebRequest.Create(targetserver);
rqst.Proxy = Proxya;
rqst.Timeout = 5000;
try
{
//Get response to check for valid proxy and then close it
WebResponse wResp = rqst.GetResponse();
//===================================================================
wResp.Close(); //HERE WAS THE PROBLEM. ADDING THIS CALL MAKES IT WORK
//===================================================================
}
catch(WebException wex)
{
connectErrMsg = wex.Message;
proxyworks = false;
}
Still not sure exactly how calling the message box was making everything work, but it doesn't really matter at this point. The whole thing works like a charm.

Categories