I've been struggling with async / await for a week now. Some background: the code below is part of an MVC4 website project. The website has a large number of API calls happening. The goal is to get those API calls happening in parallel instead of synchronous to improve site responsiveness. Right now all the API calls block each other. So if one page requires 4 calls...longer load time.
I've built out individual methods for both the synchronous and async versions of all the API calls. The problem I have is the await never responds. I think it's related to this question. However, I'm really not sure how to solve it. I've tried the ConfigureAwait(false), and that hasn't helped me.
Here's the code:
The initial call in the controller it looks like so:
BaseData bdata = API.GetBaseData().Result;
I'd love to use await here, but that's not an option without an AsyncController, which we can't use due to needing request / response access. The other methods are in the API class:
internal static async Task<BaseData> GetBaseData() {
var catTask = GetParentCategoriesAsync();
var yearTask = GetYearsAsync();
await Task.WhenAll(new Task[] { catTask, yearTask });
var bdata = new BaseData {
years = await yearTask,
cats = await catTask
};
return bdata;
}
internal static async Task<List<APICategory>> GetParentCategoriesAsync() {
try {
WebClient wc = new WebClient();
wc.Proxy = null;
string url = getAPIPath();
url += "GetFullParentCategories";
url += "?dataType=JSON";
Uri targeturi = new Uri(url);
List<APICategory> cats = new List<APICategory>();
var cat_json = await wc.DownloadStringTaskAsync(targeturi);
cats = JsonConvert.DeserializeObject<List<APICategory>>(cat_json);
return cats;
} catch (Exception) {
return new List<APICategory>();
}
}
internal static async Task<List<double>> GetYearsAsync() {
WebClient wc = new WebClient();
wc.Proxy = null;
Uri targeturi = new Uri(getAPIPath() + "getyear?dataType=JSON");
var year_json = await wc.DownloadStringTaskAsync(targeturi);
List<double> years = JsonConvert.DeserializeObject<List<double>>(year_json);
return years;
}
When these methods are called, I can put breakpoints in GetYearsAsync() and GetParentCategoriesAsync(). Everything fires until the await wc.DownloadStringTaskAsync(targeturi) command. That's where stops.
I've added ConfigureAwait(continueOnCapturedContext: false) to all the tasks, but that hasn't helped. I'm assuming the problem is that the threads are not in the same context. However, I'm not certain. I am certain, however, that I'm doing something wrong. I'm just not sure what. It's either that or I'm just trying to do something that can't be done with .NET MVC4. Any thoughts would be supremely appreciated.
The problem is actually due to WebClient, which will always sync back to the request context (which is blocked due to the Result call).
You can use HttpClient instead, combined with ConfigureAwait(false):
internal static async Task<BaseData> GetBaseDataAsync() {
var catTask = GetParentCategoriesAsync();
var yearTask = GetYearsAsync();
await Task.WhenAll(catTask, yearTask).ConfigureAwait(false);
var bdata = new BaseData {
years = await yearTask,
cats = await catTask
};
return bdata;
}
internal static async Task<List<APICategory>> GetParentCategoriesAsync() {
try {
var client = new HttpClient();
string url = getAPIPath();
url += "GetFullParentCategories";
url += "?dataType=JSON";
Uri targeturi = new Uri(url);
List<APICategory> cats = new List<APICategory>();
var cat_json = await client.GetStringAsync(targeturi).ConfigureAwait(false);
cats = JsonConvert.DeserializeObject<List<APICategory>>(cat_json);
return cats;
} catch (Exception) {
return new List<APICategory>();
}
}
internal static async Task<List<double>> GetYearsAsync() {
var client = new HttpClient();
Uri targeturi = new Uri(getAPIPath() + "getyear?dataType=JSON");
var year_json = await client.GetStringAsync(targeturi).ConfigureAwait(false);
List<double> years = JsonConvert.DeserializeObject<List<double>>(year_json);
return years;
}
That should enable you to call it as such:
BaseData bdata = API.GetBaseDataAsync().Result;
However, I strongly recommend that you call it as such:
BaseData bdata = await API.GetBaseDataAsync();
You'll find that the code both before and after the await can access the request and response context just fine.
I ended up following a combination of Servy's and Stephen Cleary's advice. Based on Stephen's response, I realized I could access the request/response as long as I did it before I made any asynchronous calls in the controller.
If I stored the HttpContext in a variable, I could pass that context around to any model/service method that needed access to it. This allowed me to just go async all the way up like Servy suggested. After that, I had no issues with anything I wanted with the async patterns.
Related
I understand the implications of using an async lambda with Parallel.ForEach which is why I'm not using it here. This then forces me to use .Result for each of my Tasks that make Http requests. However, running this simple scraper through the performance profiler shows that .Result has an elapsed exclusive time % of ~98% which is obviously due to the blocking nature of the call.
My question is: is there any possibility of optimizing this for it to still be async? I'm not sure that will help in this case since it may just take this long to retrieve the HTML/XML.
I'm running a 4 core processor with 8 logical cores (hence the MaxDegreesOfParallelism = 8. Right now I'm looking at around 2.5 hours to download and parse ~51,000 HTML/XML pages of simple financial data.
I was leaning towards using XmlReader instead of Linq2XML to speed up the parsing, but it appears the bottleneck is at the .Result call.
And although it should not matter here, the SEC limits scraping to 10 requests/sec.
public class SECScraper
{
public event EventHandler<ProgressChangedEventArgs> ProgressChangedEvent;
public SECScraper(HttpClient client, FinanceContext financeContext)
{
_client = client;
_financeContext = financeContext;
}
public void Download()
{
_numDownloaded = 0;
_interval = _financeContext.Companies.Count() / 100;
Parallel.ForEach(_financeContext.Companies, new ParallelOptions {MaxDegreeOfParallelism = 8},
company =>
{
RetrieveSECData(company.CIK);
});
}
protected virtual void OnProgressChanged(ProgressChangedEventArgs e)
{
ProgressChangedEvent?.Invoke(this, e);
}
private void RetrieveSECData(int cik)
{
// move this url elsewhere
var url = "https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=" + cik +
"&type=10-q&dateb=&owner=include&count=100";
var srBody = ReadHTML(url).Result; // consider moving this to srPage
var srPage = new SearchResultsPage(srBody);
var reportLinks = srPage.GetAllReportLinks();
foreach (var link in reportLinks)
{
url = SEC_HOSTNAME + link;
var fdBody = ReadHTML(url).Result;
var fdPage = new FilingDetailsPage(fdBody);
var xbrlLink = fdPage.GetInstanceDocumentLink();
var xbrlBody = ReadHTML(SEC_HOSTNAME + xbrlLink).Result;
var xbrlDoc = new XBRLDocument(xbrlBody);
var epsData = xbrlDoc.GetAllEPSData();
//foreach (var eps in epsData)
// Console.WriteLine($"{eps.StartDate} to {eps.EndDate} -- {eps.EPS}");
}
IncrementNumDownloadedAndNotify();
}
private async Task<string> ReadHTML(string url)
{
using var response = await _client.GetAsync(url);
return await response.Content.ReadAsStringAsync();
}
}
The task is not CPU bound, but rather network bound so there is no need to use multiple threads.
Make multiple async calls on one thread. just don't await them. Put the tasks on a list. When you get a certain amount there (say you want 10 going at once), start waiting for the first one to finish (Look up 'task, WhenAny' for more info).
Then put more on :-) You can then control the size of the lits of tasks by #/second using other code.
is there any possibility of optimizing this for it to still be async?
Yes. I'm not sure why you're using Parallel in the first place; it seems like the wrong solution for this kind of problem. You have asynchronous work to do across a collection of items, so a better fit would be asynchronous concurrency; this is done using Task.WhenAll:
public class SECScraper
{
public async Task DownloadAsync()
{
_numDownloaded = 0;
_interval = _financeContext.Companies.Count() / 100;
var tasks = _financeContext.Companies.Select(company => RetrieveSECDataAsync(company.CIK)).ToList();
await Task.WhenAll(tasks);
}
private async Task RetrieveSECDataAsync(int cik)
{
var url = "https://www.sec.gov/cgi-bin/browse-edgar?action=getcompany&CIK=" + cik +
"&type=10-q&dateb=&owner=include&count=100";
var srBody = await ReadHTMLAsync(url);
var srPage = new SearchResultsPage(srBody);
var reportLinks = srPage.GetAllReportLinks();
foreach (var link in reportLinks)
{
url = SEC_HOSTNAME + link;
var fdBody = await ReadHTMLAsync(url);
var fdPage = new FilingDetailsPage(fdBody);
var xbrlLink = fdPage.GetInstanceDocumentLink();
var xbrlBody = await ReadHTMLAsync(SEC_HOSTNAME + xbrlLink);
var xbrlDoc = new XBRLDocument(xbrlBody);
var epsData = xbrlDoc.GetAllEPSData();
}
IncrementNumDownloadedAndNotify();
}
private async Task<string> ReadHTMLAsync(string url)
{
using var response = await _client.GetAsync(url);
return await response.Content.ReadAsStringAsync();
}
}
Also, I recommend you use IProgress<T> for reporting progress.
I'm really looking for the solution and can't find proper instruction.
I have async method in RestService.cs
public async static Task<List<Convert>> CheckBTCUSDAsync()
{
HttpClient client = new HttpClient();
string restUrl =
"https://bitbay.net/API/Public/BTCUSD/trades.json";
HttpResponseMessage responseGet = await
client.GetAsync(restUrl);
if (responseGet.IsSuccessStatusCode)
{
var response = await responseGet.Content.ReadAsStringAsync();
List<Convert> currencies = Convert.FromJson(response);
//Debug.WriteLine(currencies[0].Date);
return currencies;
}
else
{
//Debug.WriteLine("***************");
//Debug.WriteLine("*****FALSE*****");
//Debug.WriteLine("***************");
return null;
}
}
I want to use it in my MainPage but of course I cant use await in sync method. I found that some devs suggest putting async tasks in eg OnStart method: https://xamarinhelp.com/xamarin-forms-async-task-startup/
I need to to Bind the returned list to picker in Xaml but of course when trying to use:
var convert = RestService.CheckBTCUSDAsync().Result;
It hangs the UI thread. Anyone knows what is the best/easiest way to resolve this?
This is how I got it to work on my app
var convert = Task.Run(() => RestService.CheckBTCUSDAsync()).Result;
I currently am tasked with implementing some new functionality into an API that will eventually replace the existing code once everything (images) are transferred to the new system.
I have a controller which is currently not async
public HttpResponseMessage GetImage(long imageId)
{
//Old sync code
var imageStream = GetImage(imageId);
var response = Request.CreateResponse();
response.Content = new StreamContent(imageStream );
response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
return response;
}
There is of course a bunch of synchronous calls to the DB and different resources already in this method.
If I made this method async and returned before the synchronous code in the new code, will the old code still function that same way and just be executed synchronously?
For example:
public async Task<HttpResponseMessage> GetImage(long imageId)
{
Stream imageStream = new MemoryStream();
var newImage = await IsNewImage(imageId);
if (newImage)
{
//Do async work
imageStream = await GetNewImage(imageId);
}
else
{
//Old sync code
imageStream = GetImage(imageId);
}
var response = Request.CreateResponse();
response.Content = new StreamContent(imageStream);
response.Content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
return response;
}
The reason for this is to try to minimize touching the old code that will be going away, and implementing the new code async. I just want to make sure introducing async functionality into this is not going to change the existing sync code? I do not have the option to create new controllers, it has to be changes to the existing controller in this instance.
I know it has been asked a lot, but my problem is, that my method won't wait for the request to be completet, even though i have implemented a TaskCompletionSource, which should have done the job, but it doesn't.
public DecksViewModel(bool local)
{
DList = new List<Deck>();
if (local)
InitializeLocalDeckList();
else
{
Dereffering();
}
}
public async void Dereffering()
{
var e = await InitilaizeWebDeckList();
List<DeckIn> decksIn = JsonConvert.DeserializeObject<List<DeckIn>>(e);
foreach (DeckIn d in decksIn)
{
Deck dadd = new Deck();
dadd.CardCount = 0;
dadd.Name = d.name;
dadd.PicturePath = d.image;
dadd.InstallDirectory = false;
DList.Add(dadd);
}
DataSource = AlphaKeyGroup<Deck>.CreateGroups(DList, System.Threading.Thread.CurrentThread.CurrentUICulture, (Deck s) => { return s.Name; }, true);
}
public Task<String> InitilaizeWebDeckList()
{
var tcs = new TaskCompletionSource<string>();
var client = new RestClient("blabla.com");
var request = new RestRequest("");
request.AddHeader("Authorization", "Basic blabla");
client.ExecuteAsync(request, response =>
{
test = response.Content;
tcs.SetResult(response.Content);
});
return tcs.Task;
}
So when I call the DecksViewModel constructor, I asyncally try to request the data from a webserver and fill the model.
The point is, that the corresponding view "doesn't wait" for the request to fill the model, so it's displayed empty.
I use the
List<AlphaKeyGroup<Deck>> DataSource
to fill a LongListSelector via DataBinding. But DataSource isn't yet set, when it is binded.
I hope you can help
You're calling an async method without awaiting it inside the constructor. That's why "it doesn't wait" (because it has nothing to wait on).
It's usually a bad idea to call an async method inside the constructor for that reason combined with the fact that constructors can't be async.
You should redesign your solution accordingly. An option is to have an async static method that creates an instance and awaits the procedure:
public static async Task CreateInstance(bool local)
{
var model = new DecksViewModel();
if (local)
{
await InitializeLocalDeckList();
}
else
{
await Dereffering();
}
}
That would allow you to not use async void which should only be used in UI even handlers.
You can read more about other options in Stephen Cleary's blog
You are using async void, which means nobody's gonna wait for that. It's just fire and forget.
I see some misunderstanding in the async keyword here:
Your code will only wait for the result of an async method, if you use await. Otherwise that call will just start the async method, but you don't know when it is actually gonna run.
You cannot use await in constructors though.
I am starting to create some classes who will fire asynchronous operations and I want the client to register a callback to receive some results. Finally I reached the following code. It is just an example, and I would like to know if there is a better way of doing it using TaskFactory and Action<>, Func<>
Here is the client basic example:
Client client2 = new Client();
client2.GetClientList(ClientCallBack);
private static void ClientCallBack(List<Client> listado)
{
//Receive the list result and do some stuff in UI
}
Here is the Client class GetCLientList asynchronous example:
public void GetClientList(Action<List<Client>> Callback)
{
List<Client> listado=null;
Task.Factory.StartNew(() =>
{
listado = new List<Client>{
new Client{ apellidos="Landeras",nombre="Carlos",edad=25},
new Client{ apellidos="Lopez", nombre="Pepe", edad=22},
new Client{ apellidos="Estevez", nombre="Alberto", edad=28}
};
//Thread.Sleep to simulate some load
System.Threading.Thread.Sleep(4000);
}).ContinueWith((prevTask) =>
{
Callback(listado);
}
);
}
Is there a better way of doing it?. I know I can return Task from my function and register the continueWith in client side, but I want to wrap it inside the class.
EDIT
I'm posting another example. I was trying to make sync/async versions of a webrequest.
Is this approach correct?:
public string ExecuteRequest(string url)
{
HttpWebRequest httpwebrequest = (HttpWebRequest) WebRequest.Create(url);
HttpWebResponse httpwebresponse = (HttpWebResponse) httpwebrequest.GetResponse();
using (StreamReader sr = new StreamReader(httpwebresponse.GetResponseStream()))
{
return sr.ReadToEnd();
}
}
public void ExecuteRequestAsync(string uri,Action<string> callbackResult)
{
Task.Factory.StartNew(() => ExecuteRequest(uri), CancellationToken.None)
.ContinueWith((task) => callbackResult(task.Result));
}
First, your method doesn't seem to be actually asynchronous, so you're not going to gain much from making it look like one. If the user of your method decides to run it on another thread, they can do it themselves.
Second, if you can use C# 5.0, you should follow the Task-based asynchronous pattern, to make your method easier to use. With that (and assuming you have a reason to ignore my first point above), your code could look like this:
public Task<List<Client>> GetClientListAsync()
{
return Task.Run(() =>
{
var list = new List<Client>
{
new Client { apellidos="Landeras", nombre="Carlos", edad=25 },
new Client { apellidos="Lopez", nombre="Pepe", edad=22 },
new Client { apellidos="Estevez", nombre="Alberto", edad=28 }
};
//Thread.Sleep to simulate some load
System.Threading.Thread.Sleep(4000);
return list;
});
}
Or, if you wanted to make your code actually asynchronous:
public async Task<List<Client>> GetClientListAsync()
{
var list = new List<Client>
{
new Client { apellidos="Landeras", nombre="Carlos", edad=25 },
new Client { apellidos="Lopez", nombre="Pepe", edad=22 },
new Client { apellidos="Estevez", nombre="Alberto", edad=28 }
};
//Task.Delay() to simulate some load
await Task.Delay(4000);
return list;
}
In both cases, you could then use this method without using callbacks from an async method like this:
var client = new Client();
var list = await client.GetClientListAsync();
//Receive the list result and do some stuff in UI
Third, if you didn't want to (or couldn't) use async-await, then your code is close, but quite right. The problem is that the callback won't actually execute on the UI thread. For that, you would need to use TaskScheduler.FromCurrentSynchronizationContext().
Also, your design where Client has a GetClientList() method seems suspicious to me. Such method probably belongs to some sort of repository object, not to Client.