We Keep Coding

Unity C# - Execute async task within a Coroutine and wait for it to finish

I'm completely new to Unity development and I'm trying to integrate an asynchronous functionality into an existing Coroutine but I have faced several issues so far.
My issues:
The Unity app completely freezes, probably because it was blocked by a thread. I implemented this code in traditional C# (console app) without having any issues.(Fixed now in Unity after some modifications)
The download task begins but it never finishes . This happens only when I run it as APK. On Unity debugging on PC works fine.
My code:
public void Validate()
{
StartCoroutine(DoWork());
}
private IEnumerator DoWork()
{
bool success;
//Perform some calculations here
..
..
success = true;
//
yield return new WaitForSeconds(3);
if(success){
GetInfo(_files);
}
}
async void GetInfo(List<string> files)
{
await StartDownload(files);
//After completing the download operation, perform some other actions in the background
...
...
//
//When done, change the active status of specific game objects
}
public async Task StartDownload(List<string> files){
var t1 = GetFileSizesAsync(files);
var t2 = DownloadFilesAsync(files);
await Task.WhenAll(t1, t2);
}
public async Task GetFileSizesAsync(List<string> urls)
{
foreach (var url in urls)
GetFileSize(url);
txtSize.text = totalSizeMb +"MB";
}
private void GetFileSize(string url)
{
var uri = new Uri(url);
var webRequest = HttpWebRequest.Create(uri);
webRequest.Method = "GET";
try
{
var webResponse = webRequest.GetResponse();
var fileSize = webResponse.Headers.Get("Content-Length");
var fileSizeInMegaByte = Math.Round(Convert.ToDouble(fileSize) / 1024.0 / 1024.0, 2);
totalSizeMb = totalSizeMb + fileSizeInMegaByte;
}
catch (Exception ex)
{
}
finally
{
webRequest.Abort();
}
}
public async Task<List<string>> DownloadFilesAsync(List<string> urls)
{
var result = new List<string>();
foreach (var url in urls)
var download = await DownloadFile(url);
if(download)
result.Add(url);
return response;
}
private async Task<bool> DownloadFile(string url)
{
WebClient webClient = new WebClient();
var uri = new Uri(url);
var saveHere = "C:\\...";
try
{
await webClient.DownloadFileTaskAsync(uri, saveHere);
return true;
}
catch (Exception ex)
{
return false;
}
}
Can you tell me what I'm doing wrong here? I've tried several ways but couldn't manage to find a proper solution.
Thank you!

I would rather make it
async Task GetInfo(List<string> files){ ... }
And then in your Coroutine do
var t = Task.Run(async () => await GetInfo(files));
while(!t.IsCompleted)
{
yield return null;
}
// Or also
//yield return new WaitUntil(() => t.IsCompleted);
if(!t.IsCompletedSuccesfully)
{
Debug.LogError("Task failed or canceled!");
yield break;
}
Note however:
When done, change the active status of specific game objects
This can't be done async! It has to happen in the Unity main thread! Therefore you would probably rather return something from your GetInfo task and activate the objects in the Coroutine when done.
after the loop and yielding you could then access the return value via
var result = t.Result;
Your web requests are currently totally redundant! You start get requests only to check how big the received content is but immediately throw away that received content ... then you start a second request to actually download the files (again).
In general I would recommend to rather use a UnityWebRequest.Get you can directly yield in the Coroutine in combination with a DownloadHandlerFile which allows you to directly download the content into a local file instead of into runtime memory.
Also
var saveHere = "C:\\...";
is hopefully not what you are trying to use as path on Android ;)

First of all.
The freezing is definitely being done by the IEneumerator as I don't see a yield statement. What's a yield statement? Uhhhh...Google it. lel
Second of all.
Wouldn't a normal void work just fine?
Third of all.
I don't know much about async's since I'm pretty new to them
but I'm fairly certain you don't need them for:
Task GetFileSizesAsync(List urls)
AND
async Task<List> DownloadFilesAsync(List urls)
I may be wrong tho.

Getting HTML response fails respectively after first fail

I have a program which gets html code for ~500 webpages every 5 minutes
it runs correctly until first fail(unable to download source in 6 seconds)
after that all threads will fail
and if I restart program, again it runs correctly until ...
where I'm wrong, what I should do to do it better?
this function runs every 5 mins:
foreach (Company company in companies)
{
string link = company.GetLink();
Thread t = new Thread(() => F(company, link));
t.Start();
if (!t.Join(TimeSpan.FromSeconds(6)))
{
Debug.WriteLine( company.Name + " Fails");
t.Abort();
}
}
and this function download html code
private void F(Company company, string link)
{
try
{
string htmlCode = GetInformationFromWeb.GetHtmlRequest(link);
company.HtmlCode = htmlCode;
}
catch (Exception ex)
{
}
}
and this class:
public class GetInformationFromWeb
{
public static string GetHtmlRequest(string url)
{
using (MyWebClient client = new MyWebClient())
{
client.Encoding = Encoding.UTF8;
string htmlCode = client.DownloadString(url);
return htmlCode;
}
}
}
and web client class
public class MyWebClient : WebClient
{
protected override WebRequest GetWebRequest(Uri address)
{
HttpWebRequest request = base.GetWebRequest(address) as HttpWebRequest;
request.AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip;
return request;
}
}

IF your foreach is looping over 500 companies, and each is creating a new thread, it could be that your internet speed could become a bottleneck and you will receive timeouts over 6 seconds, and fail very often.
I suggest you to try with parallelism. Note MaxDegreeOfParallelism, which sets maximum amount of parallel executions. You can tune this to suit your needs.
Parallel.ForEach(companies, new ParallelOptions { MaxDegreeOfParallelism = 10 }, (company) =>
{
try
{
string htmlCode = GetInformationFromWeb.GetHtmlRequest(company.link);
company.HtmlCode = htmlCode;
}
catch(Exception ex)
{
//ignore or process exception
}
});

I have four basic suggestions:
Use HttpClient instead of obsolete WebClient. HttpClient can deal with asynchronous operations natively and has far more flexibility to take advantage of. You can even read downloaded contents to strings/streams on different thread since you can configure await not to schedule back your operations. Or even program the HttpClientHandler to break after 6 seconds and raise TaskCanceledException if this was exceeded.
Avoid swallowing exceptions (like you do in your F function) as it breaks debugging and obfuscates the real cause of problems. Correctly-written program will never raise an exception during normal operation.
You are using threads in an useless way, in which they are not even overlapping; they are just waiting for each other to start, because you are locking the calling loop after each thread's start. In .NET it would be better to do multitasking using Tasks (for example, by calling them as Task.Run(async delegate() { await yourTask(); }) (or AsyncContext.Run(...) if you need UI access) and it won't block anything.
The whole GetInformationFromWeb class is pointless in the moment - and you are spawning multiple client objects also pointlessly, since one HTTP client object can handle multiple requests (if you'd use HttpClient even without additional bloat - you just instantiate it once as static global variable with all necessary configuration and then call it from any place using as little code as client.GetStringAsync(Uri uri).
OT: Is it some kind of an academic project?

Show Android ProgressBar with Xamarin

I am building an Android app through Xamarin/MonoTouch (so using C# rather than Java) and would like to display a ProgressBar while the app is communicating with the server. I have been attempting to do this using async, but I don't have a good understanding of how threading works, and I've been running into the same issue for the past few hours - the ProgressBar shows after my method call rather than before. My code is in the OnCreate() method, which I have also overridden to be async. Here it is:
loginButton.Click += async (sender, e) =>
{
progbar.Visibility = ViewStates.Visible;
var userFetcher = new UserFetcher();
var json = await userFetcher.FetchUserDetailsAsync(/*parameters*/);
//the above method is async and returns a Task<JsonValue>
ParseUserDetails(json); //another method I created
progbar.Visibility = ViewStates.Invisible;
//do some stuff using the parsed data
};
The issue I'm running into is that the FetchUserDetailsAsync seems to be blocking my thread. I have been testing this by shutting the server off so that it takes a long response time (but even when I test stuff like Thread.Sleep(5000) I have the same issue). After the method has been called, it runs both progbar.Visibility = ViewStates.Visible; and progbar.Visibility = ViewStates.Invisible; right after one another - I know this because when I comment out the Invisible part, the ProgressBar appears after my method got a "response" from the server. The compiler has also been giving me messages like "Skipped 67 frames! The application may be doing too much work on its main thread."
Like I said earlier, I'm not really experienced with threading, so it's very possible I'm just naively doing something wrong. Does anyone have a solution to this issue?
EDIT: Here is the source code for FetchUserDetailsAsync:
public async Task<JsonValue> FetchUserDetailsAsync(string url, string username, string password)
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(new Uri(url));
request.ContentType = "application/json";
request.Method = "GET";
string myBasicHeader = AuthenticationHelper.MakeHeader(username, password);
request.Headers.Add("Authorization", "Basic " + myBasicHeader);
try
{
using (HttpWebResponse response = (HttpWebResponse)request.GetResponse())
{
using (Stream stream = response.GetResponseStream())
{
JsonValue jsonDoc = await Task.Run(() => JsonObject.Load(stream));
return jsonDoc;
}
}
}
catch (WebException e)
{
Console.WriteLine(e.ToString());
if (e.Status == WebExceptionStatus.ProtocolError)
{
var response = e.Response as HttpWebResponse;
if (response != null)
{
Console.WriteLine("HTTP Status Code: " + (int)response.StatusCode);
}
else
{
Console.WriteLine("No http status code available");
}
}
return null;
}
}

It looks like either FetchUserDetailsAsync or ParseUserDetails is blocking the UI thread. Because Task.Sleep(5000) is running synchronously. Try await Task.Delay(5000); in the line of FetchUserDetailsAsync to see if the progressbar show up. If it does, then you probably need to go into FetchUserDetailsAsync implementation to make sure it is implemented async

I figured out the issue. I was calling GetResponse() instead of GetResponseAsync() in my FetchUserDetailsAsync method.

Parallel.For WebRequests - The first requests is just doing nothing

I've got a list of Accounts. I want to login with all accounts on a website. I want to use Parallel.ForEach to process all accounts.
This is what my code looks like:
Parallel.ForEach(Accounts,
acc =>
{
acc.WebProxy = null;
acc.Channel = "pelvicpaladin__";
Debug.WriteLine("Connecting to {0}.", new object[] { acc.Username });
acc.ConnectToIrc().Wait();
});
Everything works fine except one single problem:
The first Account in the Accounts-list does not work. Internal I have to use more than one request (it is a bit more than just logging in). The first request just does nothing. If I break the debugger, there is no available source.
I've got about 12 accounts. I've tried to remove the first account from the list. But the problem is still the same (now the new first (old second) account fails).
And now the very strange point:
If I don't use Parallel.For, everything works fine.
foreach (var acc in Accounts)
{
acc.WebProxy = null;
Debug.WriteLine("Connecting to {0}.", new object[] { acc.Username });
await acc.ConnectToIrc();
}
Again: Everything works except the first account from the list. It is always the first (it does not depend on how much accounts the list contains or which account is the first account).
Does anyone has any idea?
EDIT: This is how I create WebRequests:
private async Task<string> GetResponseContent(HttpWebRequest request)
{
if (request == null)
throw new ArgumentNullException("request");
using (var response = await request.GetResponseAsync())
{
return await GetResponseContent((HttpWebResponse)response);
}
}
private async Task<string> GetResponseContent(HttpWebResponse response)
{
if (response == null)
throw new ArgumentNullException("response");
using (var responseStream = response.GetResponseStream())
{
return await new StreamReader(responseStream).ReadToEndAsync();
}
}
private HttpWebRequest GetRequest(string url)
{
if (String.IsNullOrWhiteSpace(url))
throw new ArgumentNullException("url");
try
{
HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(url);
request.CookieContainer = _cookieContainer;
request.Referer = url;
request.ContentType = "application/x-www-form-urlencoded; charset=UTF-8";
request.UserAgent = "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.1.4) Gecko/20091016 Firefox/3.5.4 GTB6 (.NET CLR 3.5.30729)";
if (_webProxy != null)
request.Proxy = WebProxy.WebProxy;
request.KeepAlive = true;
request.Timeout = 30000;
return request;
}
catch (Exception ex)
{
ErrorLogger.Log(String.Format("Could not create Request on {0}.", url), ex);
return null;
}
}

You're running into the typical await deadlock situation. The problem is that your calls to ConnectToIrc are using await and capturing the synchronization context. They're trying to marshall the continuations to the main thread. The problem is that your main thread is busy blocking on the call to Parallel.ForEach. It's not allowing any of those continuations to run. The main thread is waiting on the continuations to continue, the continuations are waiting on the main thread to be free to run. Deadlock.
This is (one reason) why you shouldn't be synchronously waiting on asynchronous operations.
Instead just start up all of the asynchronous operations and use WhenAll to wait for them all to finish. There's no need to create new threads, or use the thread pool, etc.
var tasks = new List<Task>();
foreach (var acc in Accounts)
{
acc.WebProxy = null;
Debug.WriteLine("Connecting to {0}.", new object[] { acc.Username });
tasks.Add(acc.ConnectToIrc());
}
await Task.WhenAll(tasks);
This, unlike your second example, will perform all of the async operations in parallel, while still waiting asynchronously.

Updated again:
var tasks = Accounts.Select(MyTask).ToList();
await Task.WhenAll(tasks);
then you can write a named method:
private Task MyTask(Account acc)
{
acc.WebProxy = null;
Debug.WriteLine("Connecting to {0}.", new object[] { acc.Username });
return acc.ConnectToIrc();
}
thanks for the tip

C# Threading with functions that return variables

Okay so basically I have a function that returns a string, but to get that string it uses webrequest which means while it's doing that webrequest the form is locking up unless I put it in a different thread.
But I can't figure out a way to capture the returned data in a thread since it's started using thread.start and that's a void.
Any help please?
Current code if it matters to anyone:
string CreateReqThread(string UrlReq)
{
System.Threading.Thread NewThread = new System.Threading.Thread(() => CreateReq(UrlReq));
string ReturnedData = "";
return ReturnedData;
}
string CreateReq(string url)
{
try
{
WebRequest SendReq = WebRequest.Create(url);
SendReq.Credentials = CredentialCache.DefaultCredentials;
SendReq.Proxy = WebRequest.DefaultWebProxy; //For closed port networks like colleges
SendReq.Proxy.Credentials = CredentialCache.DefaultCredentials;
SendReq.Timeout = 15000;
System.IO.StreamReader Reader = new System.IO.StreamReader(SendReq.GetResponse().GetResponseStream());
string Response = Reader.ReadToEnd();
Reader.Close();
return Response;
}
catch (WebException e)
{
EBox(e.Message, "Unknown Error While Connecting");
return null;
}
}

A common means of doing this is to use a Task<T> instead of a thread:
Task<string> CreateReqThread(string UrlReq)
{
return Task.Factory.StartNew() => CreateReq(UrlReq));
// In .NET 4.5, you can use (or better yet, reimplement using await/async directly)
// return Task.Run(() => CreateReq(UrlReq));
}
You can then call Task<T>.Result to get the returned value (later), when it's needed, or schedule a continuation on the task which will run when it completes.
This could look something like:
var request = CreateReqThread(theUri);
request.ContinueWith(t =>
{
// Shove results in a text box
this.textBox.Text = t.Result;
}, TaskScheduler.FromCurrentSynchronizationContext());
This also works perfectly with the new await/async support in C# 5.