We Keep Coding

C# Asychronous REST query not working: WaitAll or WhenAll combined with .Result hangs up

I am building a C# Winforms application and I have many REST calls to process. Each call takes about 10 sec till I receive an answer, so in the end, my application is running quite a while. Mostly spending time waiting for the REST service to answer.
I am not coming forward because no matter what I try (configureAwait, waitAll or whenAll), the application hangs or when I want to access each tasks result, it is going back to the Main methods or hangs. Here is what I currently have:
I am building up a list of tasks to fill my objects :
List<Task> days = new List<Task>();
for (DateTime d = dtStart; d <= dtEnd; d = d.AddDays(1))
{
if (UseProduct)
{
Task _t = AsyncBuildDay(d, Project, Product, fixVersion);
var t = _t as Task<Day>;
days.Add(t);
}
else
{
Task _t = AsyncBuildDay(d, Project, fixVersion);
var t = _t as Task<Day>;
days.Add(t);
}
}
Then I am starting and waiting until every task is finished and the objects are built:
Task.WaitAll(days.ToArray());
When I try this, then the tasks are waiting for activation:
var tks = Task.WhenAll(days.ToArray());
What is running asynchronously inside the tasks (AsyncBuildDay
) is a query to JIRA:
private async Task<string> GetResponse(string url)
{
WebRequest request = WebRequest.Create(url);
request.Method = "GET";
request.Headers["Authorization"] = "Basic " + Convert.ToBase64String(Encoding.Default.GetBytes(JIRAUser + ":" + JIRAPassword));
request.Credentials = new NetworkCredential(JIRAUser, JIRAPassword);
WebResponse response = await request.GetResponseAsync().ConfigureAwait(false);
// Get the stream containing all content returned by the requested server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content fully up to the end.
string json = reader.ReadToEnd();
return json;
}
And now I would like to access all my objects with .Result, but then the whole code freezes again.
foreach (Task<Day> t in days)
{
dc.colDays.Add(t.Result);
}
I don't find a wait to get to my objects and I'm really going nuts with this stuff. Any ideas are much appreciated!

You're overcomplicating this.
Task.WhenAll is the way to go; it returns a new Task that completes when the provided tasks have all completed.
It's also non-blocking.
By awaiting the Task returned by Task.WhenAll, you unwrap it's results into an array:
List<Task<Day>> dayTasks = new();
// ...
Day[] days = await Task.WhenAll(dayTasks);
You can then add this to dc.colDays:
dc.colDays.AddRange(days);
Or if dc.colDays doesnt have an AddRange method:
foreach (var day in days) dc.colDays.Add(day);

It might be better to await any completion, and remove the completed task from the list.
while (days.Count > 0)
{
Task completedTask = await Task.WhenAny(days);
// Do something with result.
days.Remove(completedTask);
}

GetResponseAsync stuck in Scheduled status

I'm using below code (this is slightly simplified) to make a webrequest:
public async Task<string> GetResponseAsync()
{
WebRequest webrequest = WebRequest.Create(url);
WebResponse response = null;
string content = string.Empty;
webrequest.Method = "GET";
webrequest.Timeout = 10000; // 10 seconds
response = await webrequest.GetResponseAsync();//this seems to not get started
using (Stream dataStream = response.GetResponseStream())
{
StreamReader reader = new StreamReader(dataStream);
content = await reader.ReadToEndAsync();
}
response?.Close();
return content;
}
This code has been working in production for months. Recently some changes have been made to the load balancer of the underlying service and now intermittently the line with GetResponseAsync gets stuck.
Below is a screenshot from the tasks debugging window. It will stay in this state for hours and the timeout does not work. The tasks window only shows tasks which are either "Awaiting" or "Scheduled". There is no task in any other state. Double clicking the task in red will go to line with GetResponseAsync method.
I feel like I might be missing something obvious here. What can be the reason of this getting stuck?

As per the link below, use ConfigureAwait to prevent deadlocks. Please read extensive doc on deadlocks due to async calls
https://blog.stephencleary.com/2012/07/dont-block-on-async-code.html
public static async Task<JObject> GetJsonAsync(Uri uri)
{
// (real-world code shouldn't use HttpClient in a using block; this is just example code)
using (var client = new HttpClient())
{
var jsonString = await client.GetStringAsync(uri).ConfigureAwait(false);
return JObject.Parse(jsonString);
}
}

It's likely that code outside of this method is placing restrictions on the ExecutionContext or SynchronizationContext your task needs to resume execution.

It turned out that the SSL handshake failed and that the timeout does not work in this case. The solution was to pass a CancellationToken with the timeout like this:
await webrequest.GetResponseAsync(new CancellationTokenSource(millisecondsDelay: 10000).Token)

Show Android ProgressBar with Xamarin

I am building an Android app through Xamarin/MonoTouch (so using C# rather than Java) and would like to display a ProgressBar while the app is communicating with the server. I have been attempting to do this using async, but I don't have a good understanding of how threading works, and I've been running into the same issue for the past few hours - the ProgressBar shows after my method call rather than before. My code is in the OnCreate() method, which I have also overridden to be async. Here it is:
loginButton.Click += async (sender, e) =>
{
progbar.Visibility = ViewStates.Visible;
var userFetcher = new UserFetcher();
var json = await userFetcher.FetchUserDetailsAsync(/*parameters*/);
//the above method is async and returns a Task<JsonValue>
ParseUserDetails(json); //another method I created
progbar.Visibility = ViewStates.Invisible;
//do some stuff using the parsed data
};
The issue I'm running into is that the FetchUserDetailsAsync seems to be blocking my thread. I have been testing this by shutting the server off so that it takes a long response time (but even when I test stuff like Thread.Sleep(5000) I have the same issue). After the method has been called, it runs both progbar.Visibility = ViewStates.Visible; and progbar.Visibility = ViewStates.Invisible; right after one another - I know this because when I comment out the Invisible part, the ProgressBar appears after my method got a "response" from the server. The compiler has also been giving me messages like "Skipped 67 frames! The application may be doing too much work on its main thread."
Like I said earlier, I'm not really experienced with threading, so it's very possible I'm just naively doing something wrong. Does anyone have a solution to this issue?
EDIT: Here is the source code for FetchUserDetailsAsync:
public async Task<JsonValue> FetchUserDetailsAsync(string url, string username, string password)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.ContentType = "application/json";
request.Method = "GET";
string myBasicHeader = AuthenticationHelper.MakeHeader(username, password);
request.Headers.Add("Authorization", "Basic " + myBasicHeader);
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
JsonValue jsonDoc = await Task.Run(() => JsonObject.Load(stream));
return jsonDoc;
}
}
}
catch (WebException e)
{
Console.WriteLine(e.ToString());
if (e.Status == WebExceptionStatus.ProtocolError)
{
var response = e.Response as HttpWebResponse;
if (response != null)
{
Console.WriteLine("HTTP Status Code: " + (int)response.StatusCode);
}
else
{
Console.WriteLine("No http status code available");
}
}
return null;
}
}

It looks like either FetchUserDetailsAsync or ParseUserDetails is blocking the UI thread. Because Task.Sleep(5000) is running synchronously. Try await Task.Delay(5000); in the line of FetchUserDetailsAsync to see if the progressbar show up. If it does, then you probably need to go into FetchUserDetailsAsync implementation to make sure it is implemented async

I figured out the issue. I was calling GetResponse() instead of GetResponseAsync() in my FetchUserDetailsAsync method.

Multiple WebRequests to the same resource blocking each other?

So I have multiple threads trying to get a response from a resource, but for some reason - even though they are running in seperate threads, each response will only return when all others are either still waiting or closed. I tried using
WebResponse response = await request.GetResponseAsync(); but first of all that seems redundant to me, since I'm already running seperate threads, and also visual studio tells me
The 'await' operator can only be used within an async method. Consider marking this method with the 'async' modifier and changing its return type to 'Task'.
What's going on here?
EDIT (Code):
Start method (called from a single thread)
public void Start()
{
if (!Started)
{
ByteAt = 0;
request = (HttpWebRequest)WebRequest.Create(URL);
request.Method = "GET";
request.AddRange(ByteStart, ByteStart + ByteLength);
downloadThread = new Thread(DownloadThreadWorker);
downloadThread.Start();
Started = true;
Paused = false;
}
}
Download threads:
private void DownloadThreadWorker()
{
WebResponse response = request.GetResponse();
if (response != null)
{
if (!CheckRange(response))
Abort(String.Format("Multi part downloads not supported (Requested length: {0}, response length: {1})", ByteLength, response.ContentLength));
else
{ ...

Per HTTP 1.1 RFC a client should make no more than 2 concurrent connections. Not sure about the latest versions of IE, but previously IE honored this limitation (could be changed via a registry key) and only had no more than 2 connections to the same host at any one time. This could be the limitation you're experiencing...
Or try setting ServicePointManager.DefaultConnectionLimit above 2.

ManualResetEvent.set not getting fired in C#

I am working on Visual Web Express 2010.
I am trying to upload a file to server and block the calling function and release it once uploading is complete. However, the main thread never gets unblocked.
public partial class MainPage : UserControl
{
private FileStream fileStream;
private static String responseStr = "";
private static ManualResetEvent evt = new ManualResetEvent(false);
public MainPage()
{
InitializeComponent();
HtmlPage.RegisterScriptableObject("App", this);
}
public void sendPhoto()
{
uploadFile();
}
private void uploadFile()
{
uploadDataToServer(url);
evt.WaitOne();
postProcess();
}
public static void postProcess()
{
HtmlPage.Window.Invoke("postProcess2", responseStr);
}
void uploadDataToServer(String url)
{
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.ContentType = "multipart/form-data; boundary=---------------------------" + _boundaryNo;
request.Method = "POST";
request.BeginGetRequestStream(writeCallback, request);
}
private void writeCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
MemoryStream memoryStream = new MemoryStream();
fileStream.CopyTo(memoryStream);
if (memoryStream != null)
{
memoryStream.Position = 0;
byte[] img = memoryStream.ToArray();
Stream postStream = request.EndGetRequestStream(asynchronousResult);
postStream.Write(img, 0, img.Length);
request.BeginGetResponse(new AsyncCallback(GetResponseCallback), request);
}
}
private void GetResponseCallback(IAsyncResult asynchronousResult)
{
HttpWebRequest request = (HttpWebRequest)asynchronousResult.AsyncState;
HttpWebResponse response = (HttpWebResponse)request.EndGetResponse(asynchronousResult);
Stream streamResponse = response.GetResponseStream();
StreamReader streamRead = new StreamReader(streamResponse);
string responseString = streamRead.ReadToEnd();
streamRead.Close();
streamResponse.Close();
response.Close();
responseStr = responseString;
evt.Set();
}
}
Now, when I use evt.WaitOne() in uploadFile, the whole app goes waiting and no request is send to server, i.e. the code never reaches getResponseCallBack and hence the application never wakes up.
However, if I don't use evt.WaitOne(), then the request is successful, however I can't read the response text since it's set in writeCallBack() function and the request is async one. What should I do to get over this problem?
I can't figure out:
1. If the request is multi-threaded / Async, then why evt.WaitOne() makes the complete app waiting and the request doesn't complete?
2. If the request is single threaded, then why postProcess() [removing the evt.WaitOne()] on trying to access responseStr [set in the getResponseCallBack()] doesn't get the proper response set in it.
[Sorry, I am new to this and am confused].
Thanks.
Sorry, I forgot to mention one thing that, I am using silverlight.

There is no point in starting an async operation and then waiting for it to complete. Instead of async request.BeginGetRequestStream(writeCallback, request); use the synchronous request.GetRequestStream(...) and consume it's result. ManualResetEvent would not be needed then.

Here is the solution to the problem that I figured out:
Some have advised to use HttpWebRequest.GetRequestStream and make the call synchronous. However, as it seems that silverlight doesn't allow this to take place. Moreover, there was many confusion related to threading and parrallel-processing.
The main problem was that I wasn't able to access the UI thread from my callback function, so I used the dispatcher method and since it always makes sure that the actions are performed in UI thread, my objective got fulfilled. The following did the trick.
System.Windows.Deployment.Current.Dispatcher.BeginInvoke(() => { postProcess(); });