Inside a Portable Class Library, I've the following method which post data to a specific Url. The method works great. However I'd like to specify a more aggressive timeout (the default is 100 seconds).
Considering that there's no Timeout property on the HttpWebRequest class from the Portable Class Library, how can I make sure that the call is abandoned if it takes longer than a few seconds?
public async Task<HttpResponse> PostAsync(Uri uri, string data)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(uri);
request.Method = "POST";
request.ContentType = "application/x-www-form-urlencoded";
using (Stream requestStream = await request.GetRequestStreamAsync())
{
byte[] postBytes = Encoding.UTF8.GetBytes(data);
requestStream.Write(postBytes, 0, postBytes.Length);
}
HttpWebResponse response = (HttpWebResponse)await request.GetResponseAsync();
return new HttpResponse(response.StatusCode, await new StreamReader(response.GetResponseStream()).ReadToEndAsync());
}
Below code either will return a HttpWebResponse or null if timed out.
HttpWebResponse response = await TaskWithTimeout(request.GetResponseAsync(), 100);
if(response != null)
{
....
}
Task<HttpWebResponse> TaskWithTimeout(Task<WebResponse> task, int duration)
{
return Task.Factory.StartNew(() =>
{
bool b = task.Wait(duration);
if (b) return (HttpWebResponse)task.Result;
return null;
});
}
--EDIT--
Creating an extension method would be even better
public static class SOExtensions
{
public static Task<T> WithTimeout<T>(this Task<T> task, int duration)
{
return Task.Factory.StartNew(() =>
{
bool b = task.Wait(duration);
if (b) return task.Result;
return default(T);
});
}
}
Usage would be:
var response = (HttpWebResponse)await request.GetResponseAsync().WithTimeout(1000);
--EDIT 2--
Another way of doing it
public async static Task<T> WithTimeout<T>(this Task<T> task, int duration)
{
var retTask = await Task.WhenAny(task, Task.Delay(duration))
.ConfigureAwait(false);
if (retTask is Task<T>) return task.Result;
return default(T);
}
// Abort the request if the timer fires.
private static void TimeoutCallback(object state, bool timedOut) {
if (timedOut) {
HttpWebRequest request = state as HttpWebRequest;
if (request != null) {
request.Abort();
}
}
}
Yes it is the responsibility of the client application to implement its own time-out mechanism. You can do this from the code above which sets the timeout and uses the ThreadPool.RegisterWaitForSingleObject method. See http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.abort.aspx for the full details. Essentially it does an abort from GetResponse, BeginGetResponse, EndGetResponse, GetRequestStream, BeginGetRequestStream, or EndGetRequestStream.
Related
This is my method, to which I am passing the url to check if it's active.
The link is being activated on the wowza service so it takes some time until it's "alive"
GetResponse is returning the 404 Error because the url is not reached.
Is there a way to get the timeout instead of 404 error if the url is not alive after specified time?
public async Task<IActionResult> GetLinkIsAlive([FromQuery] string url, [FromQuery] int timeout)
{
HttpWebResponse webResponse;
try
{
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(url);
webRequest.Timeout = timeout;
webRequest.Method = "GET";
webResponse = webRequest.GetResponse() as HttpWebResponse;
return Ok(webResponse.StatusCode);
}
catch (WebException ex)
{
return BadRequest(ex.Message);
}
}
You can use connection pooling.
It’s using IHttpClientFactory that helps to maintain the pooling and lifetime of clients
In your startup class :
services.AddHttpClient<NameofyourService, NameofyourService>()
.AddTransientHttpErrorPolicy(
p => p.WaitAndRetryAsync(new[]
{
TimeSpan.FromSeconds(1),
TimeSpan.FromSeconds(5),
TimeSpan.FromSeconds(10)
}));
It requires Microsoft.Extensions.Http.Polly
You need to use HttpClient for your service. All added configurations will apply automatically.
My solution to this was calling the link in the while loop every 1s and await the whole Task.
private async Task<bool> IsLiveStreamAlive(string streamUrl, int retriesCount = 30)
{
try
{
bool res = false;
HttpClient client = new HttpClient();
Uri uri = new Uri(streamUrl);
HttpWebRequest webRequest = (HttpWebRequest)WebRequest.Create(uri);
var request = new HttpRequestMessage(HttpMethod.Get, uri);
while (retriesCount > 1)
{
await Task.Delay(1000);
retriesCount--;
HttpResponseMessage httpResponse = await client.GetAsync(uri);
res = httpResponse.StatusCode == HttpStatusCode.OK ? true : false;
if (res)
{
Log.Info(string.Format("Stream alive: {0}", streamUrl));
break;
}
}
return res;
}
catch (WebException ex)
{
Log.Error(ex);
}
return false;
}
I have an application that sends a lot of httpwebrequests and reads data. So I decided to run each request on a different task. I ended up with having about 60 tasks to run. the problem is that the these requests does not get executed when I use task.run. In fact some of the requests gets sent to the server and responses back with data and a lot of them are just ignored. I was wondering if this is because of the CPU limitations and if so how would it be possible to handle such requests.
Example of the the method that requests data:
private static bool getResponseData(string url, string pageNumber)
{
url = url + "page=" + pageNumber;
Uri myUri = new Uri(url, UriKind.Absolute);
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(myUri);
request.AutomaticDecompression = DecompressionMethods.GZip;
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
if (response.StatusCode == HttpStatusCode.OK)
{
Stream receiveStream = response.GetResponseStream();
StreamReader readStream = null;
if (String.IsNullOrWhiteSpace(response.CharacterSet))
readStream = new StreamReader(receiveStream);
else
readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
string data = readStream.ReadToEnd();
//dealing with data by storing it in the data base
return true;
}
return false;
}
}
}
A method to loop throughout url pages
private void startrequest(string url)
{
bool data2 = getResponseData(url, "0");
int counter2 = 0;
while (data2 && counter2 < 4)
{
counter2++;
data2 = getResponseData(url, counter2.ToString());
}
}
Example of the use of the task.run in my application is as follows:
Task task1 = Task.Run(() => startrequest("www.x1.com"));
Task task2 = Task.Run(() => startrequest("www.x2.com"));
Task task3 = Task.Run(() => startrequest("www.x3.com"));
Task task4 = Task.Run(() => startrequest("www.x4.com"));
Task.WaitAll(task1, task2, task3, task4);
if I run the method startrequest manually without the use of the Task.Run I always end up by getting the correct data. I am running my PC on I5.
I build this method (c#) in order to receive the HTTP response status code from an URL.
whene I run this method ones it's works fine, but when I run it in a loop, the third time its stuck. any clue??
public static string isAlive(string url)
{
Console.WriteLine("start: Is Alive Test");
WebRequest request = WebRequest.Create(url);
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
return Convert.ToString((int)response.StatusCode);
}
catch(WebException ex)
{
HttpWebResponse res = (HttpWebResponse)ex.Response;
return Convert.ToString((int)res.StatusCode);
}
}
the loop
for (int i = 0; i < 5; i++)
{
string a = isAlive("https://www.yahoo.com/");
Console.WriteLine(a);
}
You're not calling Dispose on the HttpWebResponse object, which means that the connection is still lying around. If you change your code to the following:
public static string isAlive(string url)
{
Console.WriteLine("start: Is Alive Test");
WebRequest request = WebRequest.Create(url);
try
{
using(HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return Convert.ToString((int)response.StatusCode);
}
}
catch(WebException ex)
{
using(HttpWebResponse res = (HttpWebResponse)ex.Response)
{
return Convert.ToString((int)res.StatusCode);
}
}
}
the using statement will implicitly call Dispose for you, which will close the connection.
The reason your code is halting after the second iteration is because .Net has a built in maximum number of connections it will open to a website, which is by default 2. This is controlled by System.Net.ServicePointManager.DefaultConnectionLimit which you can increase should you need to.
You need to wrap HttpWebResponse var into using statement because it's disposable
Before checking ex.Response.StatusCode make sure that ex.Status is a ProtocolError
And also consider making your method asynchronous for performance considerations
Since your method is returning a status code, there might be a better name for it than isAlive
Sample:
public static async Task<string> GetStatusCode(string url)
{
var request = (HttpWebRequest)WebRequest.Create(url);
try
{
using (var response = (HttpWebResponse)await request.GetResponseAsync())
{
return response.StatusCode.ToString();
}
}
catch (WebException ex)
{
return ex.Status == WebExceptionStatus.ProtocolError ?
((HttpWebResponse)e.Response).StatusCode.ToString() : null;
}
}
It might have to do with you not closing the HttpWebResponse.
Add a finally to that try catch which closes the response.
Also close the WebException response within the catch.
Use "using" and it will work well.
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
return Convert.ToString((int)response.StatusCode);
}
How can I use HttpWebRequest (.NET, C#) asynchronously?
Use HttpWebRequest.BeginGetResponse()
HttpWebRequest webRequest;
void StartWebRequest()
{
webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}
void FinishWebRequest(IAsyncResult result)
{
webRequest.EndGetResponse(result);
}
The callback function is called when the asynchronous operation is complete. You need to at least call EndGetResponse() from this function.
By far the easiest way is by using TaskFactory.FromAsync from the TPL. It's literally a couple of lines of code when used in conjunction with the new async/await keywords:
var request = WebRequest.Create("http://www.stackoverflow.com");
var response = (HttpWebResponse) await Task.Factory
.FromAsync<WebResponse>(request.BeginGetResponse,
request.EndGetResponse,
null);
Debug.Assert(response.StatusCode == HttpStatusCode.OK);
If you can't use the C#5 compiler then the above can be accomplished using the Task.ContinueWith method:
Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse,
request.EndGetResponse,
null)
.ContinueWith(task =>
{
var response = (HttpWebResponse) task.Result;
Debug.Assert(response.StatusCode == HttpStatusCode.OK);
});
Considering the answer:
HttpWebRequest webRequest;
void StartWebRequest()
{
webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}
void FinishWebRequest(IAsyncResult result)
{
webRequest.EndGetResponse(result);
}
You could send the request pointer or any other object like this:
void StartWebRequest()
{
HttpWebRequest webRequest = ...;
webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), webRequest);
}
void FinishWebRequest(IAsyncResult result)
{
HttpWebResponse response = (result.AsyncState as HttpWebRequest).EndGetResponse(result) as HttpWebResponse;
}
Greetings
Everyone so far has been wrong, because BeginGetResponse() does some work on the current thread. From the documentation:
The BeginGetResponse method requires some synchronous setup tasks to
complete (DNS resolution, proxy detection, and TCP socket connection,
for example) before this method becomes asynchronous. As a result,
this method should never be called on a user interface (UI) thread
because it might take considerable time (up to several minutes
depending on network settings) to complete the initial synchronous
setup tasks before an exception for an error is thrown or the method
succeeds.
So to do this right:
void DoWithResponse(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
Action wrapperAction = () =>
{
request.BeginGetResponse(new AsyncCallback((iar) =>
{
var response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
responseAction(response);
}), request);
};
wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
{
var action = (Action)iar.AsyncState;
action.EndInvoke(iar);
}), wrapperAction);
}
You can then do what you need to with the response. For example:
HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
Console.Write(body);
});
public static async Task<byte[]> GetBytesAsync(string url) {
var request = (HttpWebRequest)WebRequest.Create(url);
using (var response = await request.GetResponseAsync())
using (var content = new MemoryStream())
using (var responseStream = response.GetResponseStream()) {
await responseStream.CopyToAsync(content);
return content.ToArray();
}
}
public static async Task<string> GetStringAsync(string url) {
var bytes = await GetBytesAsync(url);
return Encoding.UTF8.GetString(bytes, 0, bytes.Length);
}
I ended up using BackgroundWorker, it is definitely asynchronous unlike some of the above solutions, it handles returning to the GUI thread for you, and it is very easy to understand.
It is also very easy to handle exceptions, as they end up in the RunWorkerCompleted method, but make sure you read this: Unhandled exceptions in BackgroundWorker
I used WebClient but obviously you could use HttpWebRequest.GetResponse if you wanted.
var worker = new BackgroundWorker();
worker.DoWork += (sender, args) => {
args.Result = new WebClient().DownloadString(settings.test_url);
};
worker.RunWorkerCompleted += (sender, e) => {
if (e.Error != null) {
connectivityLabel.Text = "Error: " + e.Error.Message;
} else {
connectivityLabel.Text = "Connectivity OK";
Log.d("result:" + e.Result);
}
};
connectivityLabel.Text = "Testing Connectivity";
worker.RunWorkerAsync();
.NET has changed since many of these answers were posted, and I'd like to provide a more up-to-date answer. Use an async method to start a Task that will run on a background thread:
private async Task<String> MakeRequestAsync(String url)
{
String responseText = await Task.Run(() =>
{
try
{
HttpWebRequest request = WebRequest.Create(url) as HttpWebRequest;
WebResponse response = request.GetResponse();
Stream responseStream = response.GetResponseStream();
return new StreamReader(responseStream).ReadToEnd();
}
catch (Exception e)
{
Console.WriteLine("Error: " + e.Message);
}
return null;
});
return responseText;
}
To use the async method:
String response = await MakeRequestAsync("http://example.com/");
Update:
This solution does not work for UWP apps which use WebRequest.GetResponseAsync() instead of WebRequest.GetResponse(), and it does not call the Dispose() methods where appropriate. #dragansr has a good alternative solution that addresses these issues.
public void GetResponseAsync (HttpWebRequest request, Action<HttpWebResponse> gotResponse)
{
if (request != null) {
request.BeginGetRequestStream ((r) => {
try { // there's a try/catch here because execution path is different from invokation one, exception here may cause a crash
HttpWebResponse response = request.EndGetResponse (r);
if (gotResponse != null)
gotResponse (response);
} catch (Exception x) {
Console.WriteLine ("Unable to get response for '" + request.RequestUri + "' Err: " + x);
}
}, null);
}
}
Follow up to the #Isak 's answer, which is very good. Nonetheless it's biggest flaw is that it will only call the responseAction if the response has status 200-299. The best way to fix this is:
private void DoWithResponseAsync(HttpWebRequest request, Action<HttpWebResponse> responseAction)
{
Action wrapperAction = () =>
{
request.BeginGetResponse(new AsyncCallback((iar) =>
{
HttpWebResponse response;
try
{
response = (HttpWebResponse)((HttpWebRequest)iar.AsyncState).EndGetResponse(iar);
}
catch (WebException ex)
{
// It needs to be done like this in order to read responses with error status:
response = ex.Response as HttpWebResponse;
}
responseAction(response);
}), request);
};
wrapperAction.BeginInvoke(new AsyncCallback((iar) =>
{
var action = (Action)iar.AsyncState;
action.EndInvoke(iar);
}), wrapperAction);
}
And then as #Isak follows:
HttpWebRequest request;
// init your request...then:
DoWithResponse(request, (response) => {
var body = new StreamReader(response.GetResponseStream()).ReadToEnd();
Console.Write(body);
});
I've been using this for async UWR, hopefully it helps someone
string uri = "http://some.place.online";
using (UnityWebRequest uwr = UnityWebRequest.Get(uri))
{
var asyncOp = uwr.SendWebRequest();
while (asyncOp.isDone == false) await Task.Delay(1000 / 30); // 30 hertz
if(uwr.result == UnityWebRequest.Result.Success) return uwr.downloadHandler.text;
Debug.LogError(uwr.error);
}
I have this piece of code to make an asynchronous HTTP request:
public static void MakeRequest(Uri uri, Action<RequestCallbackState> responseCallback)
{
WebRequest request = WebRequest.Create(uri);
request.Proxy = null;
Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null).ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(new RequestCallbackState(response.GetResponseStream()));
responseStream.Close();
response.Close();
});
}
It works, but I need to set a request timeout. I tried to use request.Timeout but don't seem to do anything. Is there a way to set up a task timeout in this code?
Edited to add a new timeout callback.
New code:
public static void MakeRequest(Uri uri, Action<RequestCallbackState> responseCallback)
{
WebRequest request = WebRequest.Create(uri);
request.Proxy = null;
IAsyncResult t = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null).ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(new RequestCallbackState(response.GetResponseStream()));
responseStream.Close();
response.Close();
});
ThreadPool.RegisterWaitForSingleObject(t.AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), request, 1000, true);
}
private static void TimeoutCallback(object state, bool timedOut)
{
if (timedOut)
{
Console.WriteLine("Timeout");
WebRequest request = (WebRequest)state;
if (state != null)
{
request.Abort();
}
}
}
Testing with:
HttpSocket.MakeRequest(new Uri("http://www.google.comhklhlñ"), callbackState =>
{
if (callbackState.Exception != null)
throw callbackState.Exception;
Console.WriteLine(GetResponseText(callbackState.ResponseStream));
});
Thread.Sleep(10000);
From the Timeout documentation:
The Timeout property has no effect on asynchronous requests made with the BeginGetResponse or BeginGetRequestStream method.
This is because the framework forces you to handle the timeout yourself. You should be able to use the example code here, except pass the Task returned from the FromAsync call to the ThreadPool.RegisterWaitForSingleObject method.
Edit:
You need to put the registration on the original task, not the continuation:
public static void MakeRequest(Uri uri, Action<Stream> responseCallback)
{
WebRequest request = WebRequest.Create(uri);
request.Proxy = null;
const int TimeoutPeriod = 1000;
Task<WebResponse> t = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
ThreadPool.RegisterWaitForSingleObject((t as IAsyncResult).AsyncWaitHandle, TimeoutCallback, request, TimeoutPeriod, true);
t.ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(response.GetResponseStream());
responseStream.Close();
response.Close();
});
}
This works for me (if I set the timeout duration very short, and hit this, I always timeout appropriately).