We Keep Coding

GetRequestStream() timeout exception

I would like to make a simple Http request using the Webclient:
public string PostRequest(object json, string contentType, string server)
{
try
{
var request = (HttpWebRequest)WebRequest.Create(server);
request.ContentType = contentType;
request.Method = "POST";
request.Timeout = 10000;
using (var streamWriter = new StreamWriter(request.GetRequestStream()))
{
streamWriter.Write(JsonConvert.SerializeObject(json));
}
var response = (HttpWebResponse)request.GetResponse();
using (var streamReader = new StreamReader(response.GetResponseStream()))
{
return streamReader.ReadToEnd();
}
}
catch (Exception e)
{
throw e;
}
}
The problem is that the request.GetRequestStream() part does never return and will always timeout (with the default 100s and the 10s as well). I am using a samsung xcover 4 with android 7 and later android 8.1. the server string works perfectly when copyed into my standard browser on the PC.On the device browser itself it does not work (timeout). The contentType is "application/json".
Is there something I can do to fix this problem or is there another method to send httprequests in xamarin that are not broken?
the server itself is working and I can ping it form my device:
public int PingHost(string nameOrAddress)
{
int pingCount = 0;
Ping pinger = null;
for (int i = 0; i < 4; i++)
{
try
{
pinger = new Ping();
PingReply reply = pinger.Send(nameOrAddress);
pingCount += reply.Status == IPStatus.Success ? 1:0;
}
catch (Exception){ pingCount = - 1; }
finally
{
pinger?.Dispose();
}
if (pingCount == -1) return -1;
}
return pingCount;
}
thanks for your time.

Well I have a working code in my application and it is something like this:
public HttpClient apiClient;
GetType API
public async Task<string> GetServiceData(string srvUrl)
{
try
{
apiClient = new HttpClient(new NativeMessageHandler(throwOnCaptiveNetwork: false, customSSLVerification: false)); // SSL true if you have custom SLL in your API
apiClient.BaseAddress = new Uri(_yourbaseUrl); // Url where the service is hosted
apiClient.DefaultRequestHeaders.Add("",""); //defualt req header key value in case any
var respon = await apiClient.GetAsync(srvUrl).Result.Content.ReadAsStringAsync(); // svrUrl is the name of the api that you want to consume
if (respon != string.Empty)
{
return respon;
}
}
catch (HttpRequestException reqEx)
{
return string.Empty;
}
catch (Exception ex)
{
return string.Empty;
}
}
PostType API
public async Task<string> PostServiceData(string srvUrl, object srvModel)
{
try
{
var myContent = JsonConvert.SerializeObject(srvModel);//your parameter to the API
var stringContent = new StringContent(myContent, Encoding.UTF8, "application/json");
apiClient = new HttpClient(new NativeMessageHandler(throwOnCaptiveNetwork: false, customSSLVerification: true));// SSL true if you have custom SLL in your API
apiClient.BaseAddress = new Uri(_yourbaseUrl); // Url where the service is hosted
apiClient.DefaultRequestHeaders.Add("",""); //defualt req header key value in case any
apiClient.DefaultRequestHeaders.Accept
.Add(new MediaTypeWithQualityHeaderValue("application/json"));//ACCEPT header
var respon = await apiClient.PostAsync(srvUrl, stringContent);
var resReslt = respon.Content.ReadAsStringAsync().Result;
return resReslt;
}
else
return string.Empty;
}
catch (HttpRequestException reqEx)
{
return string.Empty;
}
catch (Exception ex)
{
return string.Empty;
}
Good luck In case of queries revert!

httpwebrequest GET retry when connection is lost

My application uses a httpwebrequest to GET certain information from my WebAPI. What I'm trying to do is retry the request if the connection is lost or if there is no connection at all.
public static string httpsGET(string passedweburi, string BCO)
{
string content = "";
//GET method
HttpWebRequest HttpRequest = (HttpWebRequest)WebRequest.Create(passedweburi + BCO);
HttpRequest.Method = "GET";
//Response
HttpWebResponse response = (HttpWebResponse)HttpRequest.GetResponse();
StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.GetEncoding("UTF-8"));
content = sr.ReadToEnd();
string resp = content.TrimStart('[').TrimEnd(']').TrimStart('"').TrimEnd('"');
if (resp == "\"The request is invalid.\"")
{
return "VALIDATE Me";
}
else
{
return resp;
}
}
It usually stops at the response variable then throws the exception from the method that calls this method, that there is no connection. I am thinking of making a while loop to make a countdown to reconnect for about an hour perhaps. Something like this:
int rt = 0;
while (rt < 60)
{
if (resp == "\"Unable to connect to the remote server.\"")
{
Console.Writeline("Connection Timed Out");
Console.Writeline("Re-establishing connection...");
DateTime startTime = DateTime.Now;
while (true)
{
if (DateTime.Now.Subtract(startTime).TotalMilliseconds > 60000)
break;
}
rt++;
Console.Writeline("Retrying " + rt.ToString() + " times");
}
if (rt >= 60)
{
Console.Writeline("Failed to reconnect.");
}
Any advise?

//this is by no means pretty, but im using your code verbatim
` public static string httpsGET(string passedweburi, string BCO)
{
string content = "";
//GET method
HttpWebRequest HttpRequest = (HttpWebRequest)WebRequest.Create(passedweburi + BCO);
HttpRequest.Method = "GET";
//Response
try
{
HttpWebResponse response = (HttpWebResponse)HttpRequest.GetResponse();
}
catch(Exception ex)
{
return "failed";
}
StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.GetEncoding("UTF-8"));
content = sr.ReadToEnd();
string resp = content.TrimStart('[').TrimEnd(']').TrimStart('"').TrimEnd('"');
if (resp == "\"The request is invalid.\"")
{
return "VALIDATE Me";
}
else
{
return resp;
}
}
//calling your method
string resp = "";
while (rt < 60)
{
if (rt >= 60)
{
Console.Writeline("Failed to reconnect.");
}
resp = YourStaticObj.httpsGET("http://bla","bco")
if (resp == "failed")
{
Console.Writeline("Connection Timed Out");
Console.Writeline("Re-establishing connection...");
DateTime startTime = DateTime.Now;
System.Threading.Thread.Sleep(60000);
Console.Writeline("Retrying " + rt.ToString() + " times");
}
}

performance problems of Multithreading check remote file exists or not

void test() //apply multithreading
{
ThreadPool.SetMaxThreads(int.Parse(TxtThread.Text), int.Parse(TxtThread.Text) + 10);
foreach (string url in list_url)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(CheckFile), url);
}
}
void CheckFile(object url) //incoming url to check files exists or not
{
HttpWebResponse response = null;
foreach (string str in filenameArr)
{
try
{
string strUrlFile2 = UriFile(url.ToString(), str);
response = Com.WebResponse(strUrlFile2);
if (response.StatusCode == HttpStatusCode.OK && response.ContentType.ToLower() != "text/html")
{
}
}
catch (Exception ex)
{
}
finally
{
if (response != null)
{
response.Close();
}
}
}
}
public static HttpWebResponse WebResponse(string strUrlFile) //check method
{
HttpWebRequest req = null;
try
{
//System.GC.Collect();
req = (HttpWebRequest)WebRequest.Create(strUrlFile);
req.Method = "HEAD";
req.Timeout = 100;
req.ProtocolVersion = HttpVersion.Version11;
req.AllowAutoRedirect = false;
req.Accept = "*/*";
req.KeepAlive = false;
HttpWebResponse res = (HttpWebResponse)req.GetResponse();
return res;
}
catch (Exception)
{
return null;
}
}
list_url:url ArrayList
filenameArr:filename dictionary array
Questions：
1、After a while when httprequesting to check telefile several times，task didn't finish.And all threads stopped.
2、When the number of thread pool threads to grow to a certain number, and all sub-thread operation is not the main UI thread, but the program interface slow phenomenon

C# Async WebRequests: OnComplete Event

The following asynchronous C# code runs through a list of 7 URLs and tries to get HTML from each one. Right now I just have it outputting simple debug responses to the console like, "Site HTML", "No Response", or "Bad URL". It seems to work fine, but I need to fire off an event once all 7 queries have been made. How would I do this? It's important that all cases are taken into account: 1) Site HTML has been received, 2) Site timed out, 3) Site was an improper URL and couldn't be loaded. I'm covering all these cases already, but can't figure out how to connect everything to trigger a global "OnComplete" event.
Thank you.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.IO;
using System.Net;
using System.Threading;
using System.Timers;
using System.Collections.Concurrent;
using System.Diagnostics;
namespace AsyncApp_05
{
class Program
{
static int _count = 0;
static int _total = 0;
static void Main(string[] args)
{
ArrayList alSites = new ArrayList();
alSites.Add("http://www.google.com");
alSites.Add("http://www.yahoo.com");
alSites.Add("http://www.ebay.com");
alSites.Add("http://www.aol.com");
alSites.Add("http://www.bing.com");
alSites.Add("adsfsdfsdfsdffd");
alSites.Add("http://wwww.fjasjfejlajfl");
alSites.Add("http://mundocinema.com/noticias/the-a-team-2/4237");
alSites.Add("http://www.spmb.or.id/?p=64");
alSites.Add("http://gprs-edge.ru/?p=3");
alSites.Add("http://blog.tmu.edu.tw/MT/mt-comments.pl?entry_id=3141");
_total = alSites.Count;
//Console.WriteLine(_total);
ScanSites(alSites);
Console.Read();
}
private static void ScanSites(ArrayList sites)
{
foreach (string uriString in sites)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriString);
request.Method = "GET";
request.Proxy = null;
RequestState state = new RequestState();
state.Request = request;
IAsyncResult result = request.BeginGetResponse(new AsyncCallback(ResponseCallback), state);
// Timeout comes here
ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle,
new WaitOrTimerCallback(TimeOutCallback), request, 100, true);
}
catch (Exception ex)
{
Console.WriteLine("Bad URL");
Interlocked.Increment(ref _count);
}
}
}
static void ReadCallback(IAsyncResult result)
{
try
{
// Get RequestState
RequestState state = (RequestState)result.AsyncState;
// determine how many bytes have been read
int bytesRead = state.ResponseStream.EndRead(result);
if (bytesRead > 0) // stream has not reached the end yet
{
// append the read data to the ResponseContent and...
state.ResponseContent.Append(Encoding.ASCII.GetString(state.BufferRead, 0, bytesRead));
// ...read the next piece of data from the stream
state.ResponseStream.BeginRead(state.BufferRead, 0, state.BufferSize,
new AsyncCallback(ReadCallback), state);
}
else // end of the stream reached
{
if (state.ResponseContent.Length > 0)
{
Console.WriteLine("Site HTML");
// do something with the response content, e.g. fill a property or fire an event
//AsyncResponseContent = state.ResponseContent.ToString();
// close the stream and the response
state.ResponseStream.Close();
state.Response.Close();
//OnAsyncResponseArrived(AsyncResponseContent);
}
}
}
catch (Exception ex)
{
// Error handling
RequestState state = (RequestState)result.AsyncState;
if (state.Response != null)
{
state.Response.Close();
}
}
}
static void ResponseCallback(IAsyncResult result)
{
Interlocked.Increment(ref _count);
Console.WriteLine("Count: " + _count);
try
{
// Get and fill the RequestState
RequestState state = (RequestState)result.AsyncState;
HttpWebRequest request = state.Request;
// End the Asynchronous response and get the actual resonse object
state.Response = (HttpWebResponse)request.EndGetResponse(result);
Stream responseStream = state.Response.GetResponseStream();
state.ResponseStream = responseStream;
// Begin async reading of the contents
IAsyncResult readResult = responseStream.BeginRead(state.BufferRead, 0, state.BufferSize, new AsyncCallback(ReadCallback), state);
}
catch (Exception ex)
{
// Error handling
RequestState state = (RequestState)result.AsyncState;
if (state.Response != null)
{
state.Response.Close();
}
Console.WriteLine("No Response");
}
}
static void TimeOutCallback(object state, bool timedOut)
{
if (timedOut)
{
HttpWebRequest request = state as HttpWebRequest;
if (request != null)
{
request.Abort();
}
}
}
}
public class RequestState
{
public int BufferSize { get; private set; }
public StringBuilder ResponseContent { get; set; }
public byte[] BufferRead { get; set; }
public HttpWebRequest Request { get; set; }
public HttpWebResponse Response { get; set; }
public Stream ResponseStream { get; set; }
public RequestState()
{
BufferSize = 1024;
BufferRead = new byte[BufferSize];
ResponseContent = new StringBuilder();
Request = null;
ResponseStream = null;
}
}
}

You can use a CountdownEvent to find out when all the sites have been scanned. It'd be initially set to sites.Count, and then wait on that event. On each completion (either by error, timeout or success) you'd signal the event. When the event count reaches zero, the wait will return and you can have your "OnComplete" event.

The simplest way IMHO is to create a semaphore, make each OnComplete handler to Release it and to WaitOne on it N times in a master thread (where N is number of sites).
private static void ScanSites(ArrayList sites)
{
var semaphore = new Semaphore(0,sites.Count);
foreach (string uriString in sites)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriString);
request.Method = "GET";
request.Proxy = null;
RequestState state = new RequestState();
state.Request = request;
state.Semaphore = semaphore;
IAsyncResult result = request.BeginGetResponse(new AsyncCallback(ResponseCallback), state);
// Timeout comes here
ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle,
(o, timeout => { TimeOutCallback }, request, 100, true);
}
catch (Exception ex)
{
Console.WriteLine("Bad URL");
Interlocked.Increment(ref _count);
}
}
for(var i =0; i <sites.Count; i++) semaphore.WaitOne();
}
static void ReadCallback(IAsyncResult result)
{
try
{ ... }
finally{
var state = result.State as RequestState;
if (state != null) state.Semaphore.Release();
}
}
Another option is to pass some WaitHandle (ManualResetEvent fits well) to each of handler and WaitHandle.WaitAll for them in master thread.
private static void ScanSites(ArrayList sites)
{
var handles = new List<WaitHandle>();
foreach (string uriString in sites)
{
try
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uriString);
request.Method = "GET";
request.Proxy = null;
RequestState state = new RequestState();
state.Request = request;
IAsyncResult result = request.BeginGetResponse(new AsyncCallback(ResponseCallback), state);
handles.Add(result.AsyncWaitHandle);
// Timeout comes here
ThreadPool.RegisterWaitForSingleObject(result.AsyncWaitHandle,
new WaitOrTimerCallback(TimeOutCallback), request, 100, true);
}
catch (Exception ex)
{
Console.WriteLine("Bad URL");
Interlocked.Increment(ref _count);
}
}
WaitHandle.WaitAll(handles.ToArray());
}
Surely, you can achieve the same with Interlocked as well, by using, e.g. Exchange or CompareExchange methods but IMHO, WaitHandles are more straightforward here (and performance hit for using them is not significant).

can I check if a file exists at a URL?

I know I can locally, on my filesystem, check if a file exists:
if(File.Exists(path))
Can I check at a particular remote URL?

If you're attempting to verify the existence of a web resource, I would recommend using the HttpWebRequest class. This will allow you to send a HEAD request to the URL in question. Only the response headers will be returned, even if the resource exists.
var url = "http://www.domain.com/image.png";
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
try
{
response = (HttpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
/* A WebException will be thrown if the status of the response is not `200 OK` */
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
Of course, if you want to download the resource if it exists it would most likely be more efficient to send a GET request instead (by not setting the Method property to "HEAD", or by using the WebClient class).

If you want to just copy & paste Justin's code and get a method to use, here's how I've implemented it:
using System.Net;
public class MyClass {
static public bool URLExists (string url) {
bool result = false;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try {
response = (HttpWebResponse)webRequest.GetResponse();
result = true;
} catch (WebException webException) {
Debug.Log(url +" doesn't exist: "+ webException.Message);
} finally {
if (response != null) {
response.Close();
}
}
return result;
}
}
I'll keep his observation:
If you want to download the resource, and it exists, it would be more efficient to send a GET request instead by not setting the Method property to "HEAD" or by using the WebClient class.

Below is a simplified version of the code:
public bool URLExists(string url)
{
bool result = true;
WebRequest webRequest = WebRequest.Create(url);
webRequest.Timeout = 1200; // miliseconds
webRequest.Method = "HEAD";
try
{
webRequest.GetResponse();
}
catch
{
result = false;
}
return result;
}

If you are using a unc path or a mapped drive, this will work fine.
If you are using a web address (http, ftp etc) you are better off using WebClient - you will get a WebException if it doesn't exist.

public static bool UrlExists(string file)
{
bool exists = false;
HttpWebResponse response = null;
var request = (HttpWebRequest)WebRequest.Create(file);
request.Method = "HEAD";
request.Timeout = 5000; // milliseconds
request.AllowAutoRedirect = false;
try
{
response = (HttpWebResponse)request.GetResponse();
exists = response.StatusCode == HttpStatusCode.OK;
}
catch
{
exists = false;
}
finally
{
// close your response.
if (response != null)
response.Close();
}
return exists;
}

I had the same problem to solve in asp.net core, I've solved with HttpClient
private async Task<bool> isFileExist(string url)
{
using (HttpClient client = new HttpClient())
{
var restponse = await client.GetAsync(url);
return restponse.StatusCode == System.Net.HttpStatusCode.OK;
}
}

My version:
public bool IsUrlExist(string url, int timeOutMs = 1000)
{
WebRequest webRequest = WebRequest.Create(url);
webRequest.Method = "HEAD";
webRequest.Timeout = timeOutMs;
try
{
var response = webRequest.GetResponse();
/* response is `200 OK` */
response.Close();
}
catch
{
/* Any other response */
return false;
}
return true;
}

WebRequest will waiting long time(ignore the timeout user set) because not set proxy, so I change to use RestSharp to do this.
var client = new RestClient(url);
var request = new RestRequest(Method.HEAD);
request.Timeout = 5000;
var response = client.Execute(request);
result = response.StatusCode == HttpStatusCode.OK;

Thanks for all answers.
And I would like to add my implementation which includes default state when we get errors, for specific cases like mine.
private bool HTTP_URLExists(String vstrURL, bool vResErrorDefault = false, int vTimeOut = 1200)
{
bool vResult = false;
WebRequest webRequest = WebRequest.Create(vstrURL);
webRequest.Timeout = vTimeOut; // miliseconds
webRequest.Method = "HEAD";
HttpWebResponse response = null;
try
{
response = (HttpWebResponse)webRequest.GetResponse();
if (response.StatusCode == HttpStatusCode.OK) vResult = true;
else if (response.StatusCode == HttpStatusCode.NotFound) vResult = false;
else vResult = vResErrorDefault;
}
catch (WebException ex)
{
if (ex.Status == WebExceptionStatus.ProtocolError && ex.Response != null)
{
var resp01 = (HttpWebResponse)ex.Response;
if (resp01.StatusCode == HttpStatusCode.NotFound)
{
vResult = false;
}
else
{
vResult = vResErrorDefault;
}
}
else
{
vResult = vResErrorDefault;
}
}
finally
{
// Don't forget to close your response.
if (response != null)
{
response.Close();
}
}
return vResult;
}

Anoter version with define timeout :
public bool URLExists(string url,int timeout = 5000)
{
...
webRequest.Timeout = timeout; // miliseconds
...
}

This works for me:
bool HaveFile(string url)
{
try
{
using (WebClient webClient = new WebClient())
{
webClient.DownloadString(url);
}
return true;
}
catch (Exception)
{
return false;
}
}