We Keep Coding

How to make a loop with webrequest to check for changes in response

I am asking you today how to make a browser-like request, in a nutshell I want to make a web request to a website, keep it alive so I can check for changes in cookies and source code in a loop?
Here is my actual code:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost/test.php");
request.CookieContainer = new CookieContainer(); // needed to keep the session alive
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string value = null;
if (response.StatusCode == HttpStatusCode.OK)
{
bool search = true;
while(search == true)
{
for(int i = 0; i < response.Cookies.Count && search; i++)
{
if (response.Cookies[i].Name == "test")
{
value = response.Cookies[i].Value;
search = false;
}
}
}
}
if (value != null)
Console.WriteLine("found cookie : {0}", value);
response.Close();
Thank you in advance !

Just wrap your entire code in a loop and remove the while loop you already have.
If you want to check the difference between response by getting the entire stream and storing it or its hash somewhere then comparing it with the next response.
Please also remember to put a try/catch and that by default only two requests are allowed unless you tweak the configuration
bool search = true;
while (search == true) {
HttpWebRequest request = (HttpWebRequest)WebRequest.Create("http://localhost/test.php");
request.CookieContainer = new CookieContainer(); // needed to keep the session alive
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
string value = null;
if (response.StatusCode == HttpStatusCode.OK)
{
for (int i = 0; i < response.Cookies.Count && search; i++)
{
if (response.Cookies[i].Name == "test")
{
value = response.Cookies[i].Value;
search = false;
}
}
}
if (value != null)
{
Console.WriteLine("found cookie : {0}", value);
}
response.Close();
}

Consuming java web service getting error

I am getting error while consuming java web service in my win form application
The error is
You must provide a request body if you set ContentLength>0 or
SendChunked==true. Do this by calling [Begin]GetRequestStream before
[Begin]GetResponse.
My code to consume java service
public byte[] StringToByteArray(string stringData)
{
System.Text.UTF8Encoding Encoding = new System.Text.UTF8Encoding();
return Encoding.GetBytes(stringData);
}
private void button1_Click(object sender, EventArgs e)
{
string DATA = #"<RepositoryType>117</RepositoryType>
<RepositoryCategory>0</RepositoryCategory>
<ModifiedBy>2825</ModifiedBy>
<ReferenceCode>0</ReferenceCode>
<FromDate>2015-10-14T11:50:00</FromDate>
<ToDate>2015-10-14T11:51:00</ToDate>
<RepositoryName>ashok</RepositoryName>
<RepositoryShortName>kumar</RepositoryShortName>
<RepositoryDesc>nothing</RepositoryDesc>
<Fixed>F</Fixed>
<IsValid>true</IsValid>
<lstVisa />
<SortOrder>0</SortOrder>
</Repository>";`
byte[] postdata = null;
HttpWebRequest _WebRequest = null;
HttpWebResponse webresponse = null;
StreamReader ResponseStream = null;
string sReturnVal = string.Empty;
string
serviceAddress="http://172.16.12.21:8888/XML_RESPONSE/rest/test/xmltest/";
try
{
_WebRequest = (HttpWebRequest)WebRequest.Create(serviceAddress + "/" + DATA);
postdata = StringToByteArray(DATA);
if (_WebRequest != null)
{
if (postdata!=null)
{
_WebRequest.Method = "POST";
_WebRequest.ContentType= "text/xml";
_WebRequest.ContentLength = postdata.Length;
_WebRequest.UserAgent = "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1)";
_WebRequest.SendChunked = true;
}
**webresponse = (HttpWebResponse)_WebRequest.GetResponse();**
{
if (webresponse.Headers.Get("Content-Encoding") != null && webresponse.Headers.Get("Content-Encoding").ToLower() == "gzip")
ResponseStream = new StreamReader(new GZipStream(webresponse.GetResponseStream(), CompressionMode.Decompress));
else
{
Encoding enc = System.Text.Encoding.GetEncoding(1252);
ResponseStream = new StreamReader(webresponse.GetResponseStream(), enc);
}
if (ResponseStream != null)
{
XElement Root = XElement.Load(ResponseStream);
sReturnVal = Root.Value;
}
}
}
else
{
throw new Exception("Connection to " + " Service could not be Established.",
new Exception("Please Check whether " +
" Service is running Or Contact your System Administrator."));
}
}
catch(Exception ex)
{
}
}
the highlighted line is getting error.
Please help in this.

You're adding the data to the URL instead of posting it as the request body. Take a look at this question for working code you can use: HTTP POST using web service. With ASP.NET webservices you must set the SOAPAction HTTP header, you can skip this line if your service doesn't require it.

How do I reissue a webrequest? Must I recreate Webclient?

The following code gets stuck in a while loop when an exception is encountered. This is because GetResponse is caching the data.
According to fiddler, no data after the first request is issued.
Is it a best practice to recreate the webclient to solve the "refresh" issue I'm having?
private static ReportStatusEnum GetReportStatus(string domain, string oAuthKey, long permissionReportID)
{
string target = string.Format("https://{0}.egnyte.com/pubapi/v1/audit/jobs/{1}", domain, permissionReportID);
var client = new WebClient();
string result ="";
var request = (HttpWebRequest)WebRequest.Create(target);
request.ContentType = "application/json";
request.Headers.Add("Authorization", "Bearer " + oAuthKey);
request.AllowAutoRedirect = false;
bool callComplete = false;
while (callComplete != true)
{
try
{
using (var response = request.GetResponse())
{
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
}
}
JToken result2 = JObject.Parse(result);
var statusResult = result2.SelectToken("status");
ReportStatusEnum ret = ReportStatusEnum.error;
Enum.TryParse<ReportStatusEnum>(statusResult.ToString(), out ret);
Console.WriteLine("The returned variable is:" + ret);
callComplete = true;
return ret;
}
catch (System.Net.WebException e)
{
if (e.Response != null)
if (e.Response.ContentLength > 0)
{
if (e.Response.Headers["X-Mashery-Error-Code"] == "ERR_403_DEVELOPER_OVER_QPS")
{
Thread.Sleep(60000); Console.Write("*QPS HIT*");
}
}
}
}
return ReportStatusEnum.error;
}

No. HttpWebRequests are not reusable.
Just move the creation of your WebRequest into the body of your loop:
string result ="";
bool callComplete = false;
while (callComplete != true)
{
var request = (HttpWebRequest)WebRequest.Create(target);
request.ContentType = "application/json";
request.Headers.Add("Authorization", "Bearer " + oAuthKey);
request.AllowAutoRedirect = false;
//...

httpwebrequest gives timeout until restarted

I am working on a desktop application developed in C# (.NET environment).
This application connects to remote server using HttpWebRequest. If due to any reason my PC is disconnected from the internet and I re-connect it my application always gives request timeout for HttpWebRequest until I restart my whole application and if I again add new thread to my application after network d/c it works fine.
Is there any way to reset my network or anyone can tell me how does it work?
//my code is..
public String request(String add, String post, int time, String reff, int id, int rwtime)
{
try
{
if (rwtime == 0)
{
rwtime = 100000;
}
string result = "";
string location = "";
// Create the web request
HttpWebRequest req = WebRequest.Create(add) as HttpWebRequest;
req.ReadWriteTimeout = rwtime;
req.KeepAlive = true;
req.Headers.Add(HttpRequestHeader.AcceptEncoding, "gzip,deflate");
req.Accept = "application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,*/*;q=0.5";
req.ContentType = "application/x-www-form-urlencoded";
req.Timeout = time;
req.Referer = reff;
req.AllowAutoRedirect = false;
req.CookieContainer = statictk.cc[id];
req.PreAuthenticate = true;
if (post != "")
{
req.Method = "POST";
string postData = post;
ASCIIEncoding encoding = new ASCIIEncoding();
byte[] byte1 = encoding.GetBytes(postData);
// Set the content type of the data being posted.
req.ContentType = "application/x-www-form-urlencoded";
// Set the content length of the string being posted.
req.ContentLength = byte1.Length;
Stream newStream = req.GetRequestStream();
newStream.Write(byte1, 0, byte1.Length);
newStream.Close();
}
else
{
req.Method = "GET";
}
// Get response
try
{
HttpWebResponse response = req.GetResponse() as HttpWebResponse;
// Get the response stream
location = response.GetResponseHeader("Location");
if (location == "")
{
Stream responseStream = response.GetResponseStream();
if (response.ContentEncoding.ToLower().Contains("gzip"))
responseStream = new GZipStream(responseStream, CompressionMode.Decompress);
else if (response.ContentEncoding.ToLower().Contains("deflate"))
responseStream = new DeflateStream(responseStream, CompressionMode.Decompress);
StreamReader reader = new StreamReader(responseStream, Encoding.Default);
// Read the whole contents and return as a string
result = reader.ReadToEnd();
}
else
{
result = location;
}
response.Close();
if (result == "") result = "retry";
return result;
}
catch (Exception e)
{
log.store("errorinresponce", e.Message);
if (statictd.status[id] != "removed")
{
return "retry";
}
else
{
return "error";
}
}
}
catch(Exception f)
{
log.store("Networkerrorretry", f.Message);
if (f.Message == "The operation has timed out")
{
return "retry";
}
string ans = MessageBox.Show("There was a Network Error..Wish to Retry ?\nError msg : "+ f.Message, "Title", MessageBoxButtons.YesNo).ToString();
if (ans == "Yes")
return "retry";
else
{
Invoketk.settxt(id, "Not Ready");
return "error";
}
}
}

It sounds like your application is missing some error handling. A disconnect can happen at any time and your application should be able to handle it. Try to surround the network loop with a try-catch statement, and then catch for the different kinds of exceptions. Depending on what exception was thrown, you can then decide if you reconnect to the server silently or if you want to generate an error message.

can I check if a file exists at a URL?

I know I can locally, on my filesystem, check if a file exists:
if(File.Exists(path))
Can I check at a particular remote URL?

If you're attempting to verify the existence of a web resource, I would recommend using the HttpWebRequest class. This will allow you to send a HEAD request to the URL in question. Only the response headers will be returned, even if the resource exists.
var url = "http://www.domain.com/image.png";
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
Of course, if you want to download the resource if it exists it would most likely be more efficient to send a GET request instead (by not setting the Method property to "HEAD", or by using the WebClient class).

If you want to just copy & paste Justin's code and get a method to use, here's how I've implemented it:
using System.Net;
public class MyClass {
static public bool URLExists (string url) {
bool result = false;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try {
response = (HttpWebResponse)webRequest.GetResponse();
result = true;
} catch (WebException webException) {
Debug.Log(url +" doesn't exist: "+ webException.Message);
} finally {
if (response != null) {
response.Close();
}
}
return result;
}
}
I'll keep his observation:
If you want to download the resource, and it exists, it would be more efficient to send a GET request instead by not setting the Method property to "HEAD" or by using the WebClient class.

Below is a simplified version of the code:
public bool URLExists(string url)
{
bool result = true;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
try
{
webRequest.GetResponse();
}
catch
{
result = false;
}
return result;
}

If you are using a unc path or a mapped drive, this will work fine.
If you are using a web address (http, ftp etc) you are better off using WebClient - you will get a WebException if it doesn't exist.

public static bool UrlExists(string file)
{
bool exists = false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(file);
request.Method = "HEAD";
request.Timeout = 5000; // milliseconds
request.AllowAutoRedirect = false;
try
{
response = (HttpWebResponse)request.GetResponse();
exists = response.StatusCode == HttpStatusCode.OK;
}
catch
{
exists = false;
}
finally
{
// close your response.
if (response != null)
response.Close();
}
return exists;
}

I had the same problem to solve in asp.net core, I've solved with HttpClient
private async Task<bool> isFileExist(string url)
{
using (HttpClient client = new HttpClient())
{
var restponse = await client.GetAsync(url);
return restponse.StatusCode == System.Net.HttpStatusCode.OK;
}
}

My version:
public bool IsUrlExist(string url, int timeOutMs = 1000)
{
WebRequest webRequest = WebRequest.Create(url);
webRequest.Method = "HEAD";
webRequest.Timeout = timeOutMs;
try
{
var response = webRequest.GetResponse();
/* response is `200 OK` */
response.Close();
}
catch
{
/* Any other response */
return false;
}
return true;
}

WebRequest will waiting long time(ignore the timeout user set) because not set proxy, so I change to use RestSharp to do this.
var client = new RestClient(url);
var request = new RestRequest(Method.HEAD);
request.Timeout = 5000;
var response = client.Execute(request);
result = response.StatusCode == HttpStatusCode.OK;

Thanks for all answers.
And I would like to add my implementation which includes default state when we get errors, for specific cases like mine.
private bool HTTP_URLExists(String vstrURL, bool vResErrorDefault = false, int vTimeOut = 1200)
{
bool vResult = false;
WebRequest webRequest = WebRequest.Create(vstrURL);
webRequest.Timeout = vTimeOut; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)webRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) vResult = true;
else if (response.StatusCode == HttpStatusCode.NotFound) vResult = false;
else vResult = vResErrorDefault;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
{
var resp01 = (HttpWebResponse)ex.Response;
if (resp01.StatusCode == HttpStatusCode.NotFound)
{
vResult = false;
}
else
{
vResult = vResErrorDefault;
}
}
else
{
vResult = vResErrorDefault;
}
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
return vResult;
}

Anoter version with define timeout :
public bool URLExists(string url,int timeout = 5000)
{
...
webRequest.Timeout = timeout; // miliseconds
...
}

This works for me:
bool HaveFile(string url)
{
try
{
using (WebClient webClient = new WebClient())
{
webClient.DownloadString(url);
}
return true;
}
catch (Exception)
{
return false;
}
}