We Keep Coding

How exactly GetRequestAsync works?

i think i'm missing something about how HttpWebRequest works via streaming when uploading large files.
basicly, i found out that i receive timeout exception when sending large files to the server, so a post suggested to do it via Async and handle the timeout myself.
The thing is, that after debugging, i found out that "GetRequestStreamAsync" method, and writing to it does nothing at the server side, the server is called only when doing GetResponseAsync
so my question is:
- code marked as //1 - it writes the file to the request stream, but i don't see that the memory is increasing, or the server even getting any request - where does the streaming go to?
This is basicly my code:
HttpWebRequest request = RESTUtils.InitializeRequest(...);
request.AllowWriteStreamBuffering = false;
request.ContentLength = i_InputStream.Length;
request.Timeout = 5000;
using (Stream requestStream = request.GetRequestStreamWithTimeout())
{
if (requestStream != null) //1
{
// We will write the stream to the request
byte[] buffer = new byte[UPLOAD_FILE_BUFFER_SIZE];
int read = i_InputStream.Read(buffer, 0, buffer.Length);
while (read > 0)
{
requestStream.Write(buffer, 0, read);
read = i_InputStream.Read(buffer, 0, buffer.Length);
}
}
}
using (var response = request.GetResponseWithTimeout(-1))
{
using (var responseStream = response.GetResponseStream())
{
}
}
public static class WebRequestExtensions
{
public static Stream GetRequestStreamWithTimeout(
this WebRequest request,
int? millisecondsTimeout = null)
{
return AsyncToSyncWithTimeout(
request.BeginGetRequestStream,
request.EndGetRequestStream,
millisecondsTimeout ?? request.Timeout);
}
public static WebResponse GetResponseWithTimeout(
this HttpWebRequest request,
int? millisecondsTimeout = null)
{
return AsyncToSyncWithTimeout(
request.BeginGetResponse,
request.EndGetResponse,
millisecondsTimeout ?? request.Timeout);
}
private static T AsyncToSyncWithTimeout<T>(
Func<AsyncCallback, object, IAsyncResult> begin,
Func<IAsyncResult, T> end,
int millisecondsTimeout)
{
var iar = begin(null, null);
if (!iar.AsyncWaitHandle.WaitOne(millisecondsTimeout))
{
var ex = new TimeoutException();
throw new WebException(ex.Message, ex, WebExceptionStatus.Timeout, null);
}
return end(iar);
}
}
Thanks!
== Edit 9/9/15 ==
Something even weirder happens, i'm attaching breakpoint right after GetResponseAsync, then i see that the server receives the call.
after that, i'm closing the process of the client -> the server is uploading the file successfully.
this happens also if i do "Abort".
anyone knows why?

Instead of using the old-style begin/end async pattern, you should consider switching to async/await which would greatly simplify your code.
You would then set the Timeout property against the request to a large value to accommodate your waiting time; then instead of using the callback-based async code, you could just do:
var request = SomeMethodToCreateRequest();
request.Timeout = int.MaxValue; // (don't do this)
var response = await request.GetResponse();
The timeout should be respected internally, and you get to simplify your code.

Leave open a StreamReader in Unity c#

I am developing a game in which I need to retrieve data from a stream (that hasn't end).
I have a class called StreamingChannel which creates the streaming channel
public StreamingChannel (){
//stuff to set the stream
webResponse = (HttpWebResponse) webRequest.GetResponse();
responseStream = new StreamReader (webResponse.GetResponseStream (), encode);
}
and to read from it i have this method
public string Read(){
try{
string jsonText = responseStream.ReadLine();
return jsonText;
}catch(ObjectDisposedException){
return null;
}
}
I perform the reading every tot secs with an InvokeRepeating and I do that for the whole game.
It works great except that for the fact that my stream lasts for about a couple of minute. After that it throws an ObjectDisposedException.
At first I wanted to restore the connection but I didn't manage to do that without reinstantiate the whole connection. In this case the problem is that the game lags for about a seconds.
So how can I tell the StreamReader that has to leave open the channel?
ps I cannot use the constructor
public StreamReader(
Stream stream,
Encoding encoding,
bool detectEncodingFromByteOrderMarks,
int bufferSize,
bool leaveOpen)
because it has been introduced in the version 4.5 of the .NET Framework, and Unity doesn't support that.

A streaming API expects your code to pull data out of Stream pretty aggressively. You may not be able to wait for Unity to schedule your ReadLine method. I think a better model is to use a separate thread to pull data as fast as possible from the Stream and store it in a buffer. (I think this is possible in Unity.) Then you can pull the stream data out of your buffer in the standard Unity thread without worrying about the pull rate. A ConcurrentQueue would be a great buffer, but Unity doesn't support it, so I've used a locked List.
Using a separate thread also allows you to restart after failures without blocking the main game.
using System.Collections.Generic;
using System.Threading;
public class StreamingChannel
{
private readonly List<string> backgroundLinesList;
private readonly object listLock = new object();
private Thread streamReaderThread;
public StreamingChannel()
{
streamReaderThread = new Thread(this.ReadWebStream);
streamReaderThread.Start();
}
public List<string> Read()
{
if (!streamReaderThread.IsAlive)
{
streamReaderThread = new Thread(this.ReadWebStream);
streamReaderThread.Start();
}
List<string> lines = null;
lock (listLock)
{
if (backgroundLinesList != null)
{
lines = backgroundLinesList;
backgroundLinesList = null;
}
}
return lines;
}
private void ReadWebStream()
{
try
{
//stuff to set the stream
HttpWebRequest webRequest;
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
StreamReader responseStream = new StreamReader(webResponse.GetResponseStream(), encode);
while (!responseStream.EndOfStream)
{
var line = responseStream.ReadLine()
lock (listLock)
{
if (backgroundLinesList == null)
{
backgroundLinesList = new List<string>();
}
backgroundLinesList.Add(line);
}
}
log.Debug("Stream closed");
}
catch (Exception e)
{
log.Debug("WebStream thread failure: " + e + " Stack: " + e.StackTrace);
}
}
}

Show spinner in MonoTouch while downloading data

So in my app I have a button that talks to a lib that downloads some data from the internet and filters it. When the app is doing this the screen freezes and it looks to the user like the app crashed. But this is not the case because its downloading data.
Here is my code:
GetDetailsBtn.TouchUpInside += (sender, e) => {
var defaults = NSUserDefaults.StandardUserDefaults;
if (RefNr.Text != string.Empty && RefNr.Text != null) {
FilteredDataRef = _FetchingData.getTrackTraceData (defaults.StringForKey ("SecurityToken"), RefNr.Text);
if (FilteredDataRef == null) {
UIAlertView InvalidAlert = new UIAlertView ("Reference number invalid", "The reference number that you have entered is not linked to the current security code. You can change your security code in the settings.", null, "OK", null);
InvalidAlert.Show ();
} else {
FilteredDataReceived = _FetchingData.FilteringOnReceived (FilteredDataRef);
FilteredDataPlanned = _FetchingData.FilteringOnPlanned (FilteredDataRef);
FilteredDataLoadingETA = _FetchingData.FilteringOnLoadingETA (FilteredDataRef);
FilteredDataLoadingFinal = _FetchingData.FilteringOnLoadingFinal (FilteredDataRef);
FilteredDataUnloadingETA = _FetchingData.FilteringOnUnloadingETA (FilteredDataRef);
FilteredDataUnloadingFinal = _FetchingData.FilteringOnUnloadingFinal (FilteredDataRef);
this.PerformSegue (MoveToTrackTraceDetailsSegue, this);
//foreach (string s in FilteredDataPlanned)
// Console.WriteLine (s);
}
} else {
UIAlertView InvalidAlert = new UIAlertView ("Reference number cannot be empty", "You did not provide a reference number. We need your reference number to trace identify the shipment you would like to trace.", null, "OK", null);
InvalidAlert.Show ();
}
};
Downloading of the data:
public IEnumerable<string> getTrackTraceData (string securityCode, string referenceNumber)
{
WebRequest request = WebRequest.Create ("http://plex.janssen1877.com/app/life/" + securityCode);
HttpWebResponse response = (HttpWebResponse)request.GetResponse ();
Stream dataStream = response.GetResponseStream ();
StreamReader reader = new StreamReader (dataStream);
string FetchedData = reader.ReadToEnd ();
reader.Close ();
dataStream.Close ();
response.Close ();
var FetchingDataItems = FetchedData.Split (new char[] { '\n' });
if (FetchingDataItems != null) {
var filteredResult = FetchingDataItems.Where (x => x.Contains (referenceNumber));
return filteredResult;
} else {
return null;
}
}
Now I want to use a component called BTProgressHUD. This is just a fancy spinner. I thought that if I would put BTProgressHUD.show(); to the top of the button action and BTProgressHUD.Dismiss(); to the button it would show when the loading starts and dismiss when its done loading.
This is not the case. It shows very quickly in the new view controller and dismisses again within a second. What am I doing wrong?
Edit for exemple:
public IEnumerable<string> getTrackTraceData (string securityCode, string referenceNumber)
{
string url = string.Format ("http://plex.janssen1877.com/app/life/" + securityCode);
HttpWebRequest HttpRequest = (HttpWebRequest)WebRequest.Create (url);
string FetchedData = new StreamReader (HttpRequest.GetResponse ().GetResponseStream ()).ReadToEnd ();
var FetchingDataItems = FetchedData.Split (new char[] { '\n' });
if (FetchingDataItems != null) {
var filteredResult = FetchingDataItems.Where (x => x.Contains (referenceNumber));
return filteredResult;
} else {
return null;
}
}

Florian,
According to .NET documentation and HttpWebRequest GetResponse. How to wait for that response? you need to perform the download (and maybe the parsing) in a async fashion.
The behavior of your actual application is correct. When you perform a sync request in the main thread you freeze the application and hence the UI elements are not updated. The main thread processes execution in a serial fashion.
To avoid this you have two different solutions. On the one hand, you need to move to an async request. On the other hand, you can create a background thread (a different path of execution) with a sync request. I prefer the former. So, for example, after starting the async request, show the indicator. When you have finished (in the callback), dismiss the indicator and perform the segue.
For example, you can follow the following discussion on how to achieve this: How to use HttpWebRequest (.NET) asynchronously?.
To understand how the main thread (and the run loop) works, I suggest to read about The pogo stick of NSRunLoop.
Hope that helps.
Edit
You should use a pattern like the following (source How to use HttpWebRequest (.NET) asynchronously?):
HttpWebRequest webRequest;
void StartWebRequest()
{
webRequest.BeginGetResponse(new AsyncCallback(FinishWebRequest), null);
}
void FinishWebRequest(IAsyncResult result)
{
webRequest.EndGetResponse(result);
}

It sounds like you're attempting to do all this in the same thread as your UI, which is why your app freezes and waits for processing to finish without showing anything. I would perform this download operation in a backgroundworker of some sort. Then I am not sure in Mono but access the main thread before and after and show and dismiss your loading component.

C# How to stop a method if it takes longer than 2 seconds?

Following program will connect to the web and get html content of “msnbc.com” webpage and print out the result. If it takes longer than 2 seconds to get data from the webpage, I want my method to stop working and return. Can you please tell me how can I do this with an example?
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
gethtml();
MessageBox.Show("End of program");
}
public void gethtml()
{
HttpWebRequest WebRequestObject = (HttpWebRequest)HttpWebRequest.Create("http://msnbc.com/");
WebResponse Response = WebRequestObject.GetResponse();
Stream WebStream = Response.GetResponseStream();
StreamReader Reader = new StreamReader(WebStream);
string webcontent = Reader.ReadToEnd();
MessageBox.Show(webcontent);
}
}

Two seconds is far too long to block the UI. You should only block the UI if you are planning on getting the result in, say fifty milliseconds or less.
Read this article on how to do a web request without blocking the UI:
http://www.developerfusion.com/code/4654/asynchronous-httpwebrequest/
Note that this will all be much easier in C# 5, which is in beta release at present. In C# 5 you can simply use the await operator to asynchronously await the result of the task. If you would like to see how this sort of thing will work in C# 5, see:
http://msdn.microsoft.com/en-us/async

Set the Timeout property of your WebRequest object. Documentation
MSDN Example:
// Create a new WebRequest Object to the mentioned URL.
WebRequest myWebRequest=WebRequest.Create("http://www.contoso.com");
Console.WriteLine("\nThe Timeout time of the request before setting is : {0} milliseconds",myWebRequest.Timeout);
// Set the 'Timeout' property in Milliseconds.
myWebRequest.Timeout=10000;
// This request will throw a WebException if it reaches the timeout limit before it is able to fetch the resource.
WebResponse myWebResponse=myWebRequest.GetResponse();

As stated above .Timeout
public void gethtml()
{
HttpWebRequest WebRequestObject = (HttpWebRequest)HttpWebRequest.Create("http://msnbc.com/");
WebRequestObject.Timeout = (System.Int32)TimeSpan.FromSeconds(2).TotalMilliseconds;
try
{
WebResponse Response = WebRequestObject.GetResponse();
Stream WebStream = Response.GetResponseStream();
StreamReader Reader = new StreamReader(WebStream);
string webcontent = Reader.ReadToEnd();
MessageBox.Show(webcontent);
}
catch (System.Net.WebException E)
{
MessageBox.Show("Fail");
}
}

You can use the TimeOut property on HttpWebRequest

Consider switching to asynchronous downloading of the content. You will stop blocking UI thread and will be able to handle multiple requests easily. You will be able to increase timeout significantly without impact on UI, and can decide upon receiving response if you still want to fetch data.

Different performance between Java and c# code when testing URL

When running the following Java code, I get very accurate and consistent results in determining if the web page I'm testing is up.
protected synchronized boolean checkUrl(HttpURLConnection connection){
boolean error = false;
//HttpURLConnection connection = null;
GregorianCalendar calendar = new GregorianCalendar();
try{
if(connection != null){
connection.connect();
//200 is the expected HTTP_OK response
error = processResponseCode(connection.getResponseCode());
connection.disconnect();
} else{
error = false;
}
}catch(java.net.UnknownHostException uhe){
... }
catch(Exception e){
... }
return error;
}
The closest match to the Java pattern in c# has much higher results of false positives (mostly due to timeouts - which has a default period of 100000ms).
protected bool connectedToUrl = false;
response = null;
HttpWebRequest webreq = (HttpWebRequest)WebRequest.Create(this.getUri());
webreq.Credentials = CredentialCache.DefaultCredentials;
WebResponse res = null;// webreq.GetResponse();
try
{
WebRequest request = WebRequest.Create(this.getUri()) as WebRequest;
request.Credentials = CredentialCache.DefaultCredentials;
if (request != null)
{
// Get response
res = webreq.GetResponse();
connectedToUrl = processResponseCode(res);
}
else
{
logger.Fatal(getFatalMessage());
string error = string.Empty;
}
}
catch (Exception e)
{
throw e;
}
return connectedToUrl;
}
I have tried various patterns in c# to match the effectiveness of the quoted Java code, to no avail.
Any ideas?

I believe this is because you're not closing any of the request objects.

Simply change this:
res = webreq.GetResponse();
connectedToUrl = processResponseCode(res);
to
using (WebResponse res = webreq.GetResponse())
{
connectedToUrl = processResponseCode(res);
}
(Remove the declaration from earlier.)
Until you haven't closed/disposed the response (or it's been finalized), it's holding onto the connection. You can only have a certain number (2 by default, I believe) of connections to any one host at a time, hence the timeouts. When you dispose the response, it allows another request to use the same connection.

Also this:
catch (Exception e)
{
throw e;
}
Does nothing but destroy the stack trace on an exception that's been bubbled upwards. If you have error handling elsewhere in your code I suggest removing the try catch block. Otherwise you should log the exception and move on. Don't just catch it to throw it.

I think you're missing the GregorianCalendar in the C# version :-)
Why do you have two Request Objects in the C# version?