I have the following method:
public async Task ExecuteAsync()
{
Task<IEnumerable<Comment>> gettingComments = RetrieveComments();
Dictionary<string, ReviewManager> reviewers = ConfigurationFacade.Repositories.ToDictionary(name => name, name => new ReviewManager(name));
IEnumerable<Comment> comments = await gettingComments;
Parallel.ForEach(reviewers, (reviewer) => {
Dictionary<Comment, RevisionResult> reviews = reviewer.Value.Review(comments);
int amountModerated = ModerateComments(reviews.Where(r => r.Value.IsInsult), "hide");
});
}
My ModerateComments method looks like the following:
private Task<int> ModerateComments(IEnumerable<Comment> comments, string operation)
{
return Task.Factory.StartNew(() =>
{
int moderationCount = 0;
Parallel.ForEach(comments, async (comment) =>
{
bool moderated = await ModerateComment(comment, operation); //Problem here
if(moderated)
moderationCount++;
}
return moderationCount;
};
}
And finally:
private async Task<bool> ModerateComment(Comment comment, string operation, string authenticationToken = null)
{
if(comment == null) return false;
if(String.IsNullOrWhiteSpace(authenticationToken))
authenticationToken = CreateUserToken(TimeSpan.FromMinutes(1));
string moderationEndpoint = ConfigurationFacade.ModerationEndpoint;
using(HttpRequestMessage request = new HttpRequestMessage())
{
request.Method = HttpMethod.Post;
request.RequestUri = new Uri(moderationEndpoint);
using(HttpResponseMessage response = await _httpClient.SendAsync(request)) //Problem here
{
if(!response.IsSuccessStatusCode)
{
if(response.StatusCode == HttpStatusCode.Unauthorized)
return await ModerateComment(comment, operation, null); //Retry operation with a new access token
else if(response.StatusCode == HttpStatusCode.GatewayTimeout)
return await ModerateComment(comment, operation, authenticationToken); //Retry operation
return false;
}
}
}
return true;
}
I'm having a strange problem at runtime. All the above code is working fine except when it reaches the line:
using(HttpResponseMessage response = await _httpClient.SendAsync(request)) {
//...
}
When I debug my application, this instruction is executed but just after that, it does not throw any exception, nor return anything, it just finish executing and I am derived to the next statement on the Parallel.ForEach loop.
It is really hard to explain so I'll post some images:
All good so far, I reach the following line of code:
The execution keeps going well and I reach the call to the Moderation API
Even if I press F10 (Next statement) in the debugger, the execution flow jumps to the next loop in the Parallel.ForEach loop.
As you can see I have breakpoints in the try-catch just i ncase any exception is thrown, but the breakpoint is never activated, neither is activated the breakpoint in if(moderacion) commentCount++.
So what happens here? Where did my execution flow went? It just dissapears after sending the POST request to the API.
After continuing the execution, all the elements in the enumerable do the same jump, and therefore, my commentCount variable ends up being equal to 0
You don't need Parallel.ForEach or Task.Factory.StartNew to do IO bound work:
private async Task<int> ModerateCommentsAsync(IEnumerable<Comment> comments, string operation)
{
var commentTasks = comments.Select(comment => ModerateCommentAsync(comment, operation));
await Task.WhenAll(commentTasks);
return commentTasks.Count(x => x.Result);
}
Common practice is to add the Async postfix to an async method.
Excellent description for a common problem. Parallel.ForEach does not support async lambdas. async methods return once they hit the first await that would need to block. This happens when you issue the HTTP request.
Use one of the common patterns for a parallel async foreach loop.
Related
I am making two httpClient.GetAsync() calls.
According to the requirement, one of two calls will always throw a "No host known" exception and one will return a proper HttpResponseMessage object.
My problem is determining the bool value only after both async httpClient calls finish.
public async Task<bool> BothHttpCallsTask(string hostName)
{
bool response1 = false;
bool response2 = false;
try
{
//httpClient and the GetAsync() are inside this method
response1 = await CheckRedirectionBool(url1);
response2 = await CheckRedirectionBool(url2);
}
catch(Exception e)
{
//when exception is caught here, the method evaluates as false
//as response2 is still not executed
}
return response1 || response2 ;
}
How do I make the execution only evaluate when both async calls complete successfully (keeping in mind the mandatory exception makes the return statement evaluate before response 2 can get a value from its execution)
I need to return true if even one of the two http calls are successful.
My problem is determining the bool value only after both async httpClient calls finish.
If you want to treat an exception the same as returning false, then I recommend writing a little helper method:
async Task<bool> TreatExceptionsAsFalse(Func<Task<bool>> action)
{
try { return await action(); }
catch { return false; }
}
Then it becomes easier to use Task.WhenAll:
public async Task<bool> BothHttpCallsTask(string hostName)
{
var results = await Task.WhenAll(
TreatExceptionsAsFalse(() => CheckRedirectionBool(url1)),
TreatExceptionsAsFalse(() => CheckRedirectionBool(url2))
);
return results[0] || results[1];
}
Could you simply wrap each in its own exception handler?
For instance:
try{
response1 = ...
}
catch(Exception e){
//set some flag here
}
try{
response2 = ...
}
catch(Exception e){
//set some flag here
}
This way you know which one past vs which one didn't and set some flags based on that condition, etc.
This is my code, I'm using AsyncEx in my library to try to get around potential deadlocks, but I ended up in there anyway:
return AsyncContext.Run(async () =>
{
var get = await httpClient.GetAsync(url);
if (get.IsSuccessStatusCode && (get.StatusCode == HttpStatusCode.OK))
{
var content = await get.Content.ReadAsStringAsync();
return content;
}
return "";
});
I'm running this from a command line app, calling it with multiple different url values in a row, but synchronously in one big for loop. Given enough calls, it will eventually stop dead in its tracks. Am I doing something wrong?
What I think happens is that after calling GetAsync the continuation cannot switch back to the same thread since the other thread is waiting for the continuation to start. Does the following code work?
return AsyncContext.Run(async () =>
{
var get = await httpClient.GetAsync(url).ConfigureAwait(false);
if (get.IsSuccessStatusCode && (get.StatusCode == HttpStatusCode.OK))
{
var content = await get.Content.ReadAsStringAsync().ConfigureAwait(false);
return content;
}
return "";
});
Please forgive me for any noobish mistakes seen below, I'm learning some of the concepts I'm attempting to work with.
Problem:
While debugging my app, I was able to call an async function with Task.Start(). I felt that the app was in a working state for the phase I'm in so removed all breakpoints with CTRL + SHIFT + F9.
Once I ran the app with no breakpoints it would fail due to a property not getting populated. Now when I try to debug any breakpoint I set in the async function that handles most of the work is longer hit. It's like it is getting skipped. Can anyone see a reason why GetWowAuctionFileInfo isn't being called?
GetWowAuctionFileInfo is what is not getting called, or at least appears to be not getting called.
Thanks.
Relevant Code
Caller Function
private void buttonTestJSFCHI_Click(object sender, RoutedEventArgs e)
{
JSON_Worker w = new JSON_Worker();
w.StartTask("FileInfo", "https://us.api.battle.net/wow/auction/data/medivh?locale=en_US&apikey=<guid>");
foreach (string res in w.ReturnedData)
{
textBoxResults.Text += res;
}
}
Called Functions
public void StartTask(string TaskName, string optionalUri= "no_uri_passed")
{
if (TaskName == "FileInfo")
{
//Need to use a lamba expression to call a delegate with a parameter
if (!(optionalUri == "no_uri_passed"))
{
Task t = new Task(() => GetWowAuctionFileInfo(optionalUri));
t.Start();
//Func<string> function = new Func<string>(() => GetWowAuctionFileInfo(optionalUri));
//Task<string> tInfo = Task<string>.Factory.StartNew(() => GetWowAuctionFileInfo(optionalUri));
}
}
}
private async void GetWowAuctionFileInfo(string auctionInfoUri)
{
RealmJSFileCheck realmInfoObject;
List<string> returnValue = new List<string>();
try
{
using (HttpClient client = new HttpClient())
{
for (int attempt = 0; attempt < 3; attempt++)
{
var response = await client.GetAsync(auctionInfoUri);
if (response.IsSuccessStatusCode)
{
string content = await response.Content.ReadAsStringAsync();
realmInfoObject = JsonConvert.DeserializeObject<RealmJSFileCheck>(content);
returnValue = ConvertFileInfoToConsumableList(realmInfoObject);
//returnValue = realmInfoObject.files.ToString();
break;
}
}
}
}
catch (InvalidOperationException iOpEx)
{
//recieved this when an invalid uri was passed in
}
ReturnedData = returnValue;
}
private List<string> ConvertFileInfoToConsumableList(RealmJSFileCheck jsfc)
{
List<string> returnData = new List<string>();
if (jsfc.files.Count > 0)
{
StringBuilder sb = new StringBuilder();
sb.Append("File URL: ");
sb.Append(jsfc.files[0].url);
returnData.Add(sb.ToString());
sb = new StringBuilder();
sb.AppendLine("Last Modified: ");
sb.Append(jsfc.files[0].lastModified);
returnData.Add(sb.ToString());
}
else
{
returnData.Add("No File Info Found");
}
return returnData;
}
UPDATE
Thanks again all for the detailed commentary. I've gone through much documentation regarding Task usage and learned a lot in this exercise. I'm marking the answer from #Johnathon as the solution because it provided exactly what I was asking for and provided a very helpful link for more information.
Your GetWowAuctionFileInfo method is an asynchronous method, and you await an async call within it without returning a Task. In general it is bad practice to use async void. Instead, turn your GetWowAuctionFileInfo method into async Task<List<string>> GetWowAuctionFileInfo. This will let you await the GetAsync call, parse the data, and return the collection to the caller without having to use a ReturnObject.
private async Task<List<string>> GetWowAuctionFileInfo(string auctionInfoUri)
{
RealmJSFileCheck realmInfoObject;
List<string> returnValue = new List<string>();
try
{
using (HttpClient client = new HttpClient())
{
for (int attempt = 0; attempt < 3; attempt++)
{
var response = await client.GetAsync(auctionInfoUri);
if (response.IsSuccessStatusCode)
{
string content = await response.Content.ReadAsStringAsync();
realmInfoObject = JsonConvert.DeserializeObject<RealmJSFileCheck>(content);
// You can just return the List<T> now.
return ConvertFileInfoToConsumableList(realmInfoObject);
//returnValue = realmInfoObject.files.ToString();
break;
}
}
}
}
catch (InvalidOperationException iOpEx)
{
//recieved this when an invalid uri was passed in
}
}
Because the method was originally async void, you could not await the calling of it in your buttonTestJSFCHI_Click. Now that we've made it all Task based, you can await it within your event handler. Note that event handlers are generally the only acceptable place to use async void. Any time you are responsible for the creation of the methods, and not constrained by a contract (like event handlers), you should always return a Task on your async methods.
private async void buttonTestJSFCHI_Click(object sender, RoutedEventArgs e)
{
JSON_Worker w = new JSON_Worker();
List<string> results = await w.StartTask("FileInfo", "https://us.api.battle.net/wow/auction/data/medivh?locale=en_US&apikey=<guid>");
foreach (string res in results)
{
textBoxResults.Text += res;
}
}
public async Task<List<string>> StartTask(string TaskName, string optionalUri= "no_uri_passed")
{
if (TaskName == "FileInfo")
{
//Need to use a lamba expression to call a delegate with a parameter
if (!(optionalUri == "no_uri_passed"))
{
// Since the GetWowAuctionFileInfo now returns Task, we don't need to create a new one. Just await the Task given back to us, and return the given result.
return await GetWowAuctionFileInfo(optionalUri);
}
}
}
The reason you saw the expected result while debugging is because the debug session was slow enough that the async operation completed in time for your code to use it. When running the app outside of the debugger, it runs faster than the async operation could complete, preventing you from seeing the data. Thus the need to await the entire async call stack, so you can prevent further execution from happening down that code-path until you receive all of the desired data.
Microsoft has a good write up on Task based programming, I'd take a read through it to help you understand it some.
EDIT
Just to clarify, when you return a Task<T> on your methods, you will be given the result when you await. For example:
List<string> result = await StartTask();
Even though StartTask returns Task<List<string>>, the await operation will wait for the StartTask() method to complete, and then unwrap the result from the Task<T> object and give you the result back automatically. So don't let the method signature fool you, if you await it, you will be given back the resulting data, and not the actual Task itself. There won't be any need for you to pull the data out of the Task manually.
Because you not waiting for result.
You loop with ReturnedData before it was assigned with data.
I think you don't need to create new Task at all. Make GetWowAuctionFileInfo method properly asynchronous which returns Task.
private async Task GetWowAuctionFileInfo(string auctionInfoUri)
{
// same code
}
Change StartTask to return Task. Because we not awaiting result here we don't need make method asynchronous.
Suggest to change name of this method to LoadData for example, which give more information about what this method does.
public Task StartTask(string TaskName, string optionalUri= "no_uri_passed")
{
if (TaskName == "FileInfo")
{
//Need to use a lamba expression to call a delegate with a parameter
if (!(optionalUri == "no_uri_passed"))
{
return GetWowAuctionFileInfo(optionalUri) // this will return Task
}
}
// if validation fails - return completed task or throw exception
return Task.CompletedTask;
}
Then you can call it in Button_Click event handler
private async void buttonTestJSFCHI_Click(object sender, RoutedEventArgs e)
{
JSON_Worker w = new JSON_Worker();
await w.StartTask("FileInfo", "yourUrl");
// This line will be executed only after asynchronous methods completes succesfully
// or exception will be thrown
foreach (string res in w.ReturnedData)
{
textBoxResults.Text += res;
}
}
I have an ASP.NET MVC application which needs to check if something exists at 3 remote API servers. The application passes an ID to each API and it returns either true or false. The code looks like this.
public class PingController
{
public async Task<bool> IsFound(int id)
{
var servers = new ['a.com', b.com', 'c.com'];
var result = await foundAtServers(id, servers);
return result;
}
private async Task<bool> foundAtServers(int id, string[] servers)
{
var tasks = from server in servers
select checkServer(id, server);
return await.Task.WhenAll(tasks.ToArray());
}
private async Task<bool> checkServer(id, server)
{
var request = new HttpRequestMessage(HttpMethod.Get, server+"/api/exists"+id);
var client = new HttpClient();
var task = await client.SendAsync(request);
var response = await task.Content.ReadAsStringAsync();
return bool.Parse(response);
}
}
This code currently checks all 3 APIs asynchronously but will wait until ALL of the HttpClient calls have completed before the MVC Action can return.
As soon as one API returns true I want to immediately return true on the Action, rather than wait for the other tasks to complete.
The C# Task class has .WaitAll and .WaitAny, but these won't work either. As I need to cancel the other HttpClient request, I presume I need to use a CancellationToken but I don't know how to use it with this structure.
Cheers.
If you want to immediately return, you can use Task.WhenAny instead of Task.WhenAll. This won't cancel the on-going tasks, but it will enable you to return as soon as possible:
private async Task<bool> FoundAtServersAsync(int id, string[] servers)
{
var tasks = (from server in servers
select checkServer(id, server)).ToList();
while (tasks.Count > 0)
{
var finishedTask = await Task.WhenAny(tasks);
if (finishedTask.Result)
{
return finishedTask.Result;
}
tasks.Remove(finishedTask);
}
return false;
}
This will discard the other tasks. This means that if any exception is thrown inside one of them, it will be swallowed.
Edit:
If you care about actually canceling the other tasks, consider passing your CancellationToken to the overload of SendAsync which takes one, and calling CancellationTokenSource.Cancel once a value is received. Note this will mean you'll also need to handle the OperationCanceledException they will throw.
If they don't matter, i'd simply discard them as above.
This problem is made easier by using the following method to take a sequence of tasks and order them based on when they are completed.
public static IEnumerable<Task<T>> Order<T>(this IEnumerable<Task<T>> tasks)
{
var taskList = tasks.ToList();
var taskSources = new BlockingCollection<TaskCompletionSource<T>>();
var taskSourceList = new List<TaskCompletionSource<T>>(taskList.Count);
foreach (var task in taskList)
{
var newSource = new TaskCompletionSource<T>();
taskSources.Add(newSource);
taskSourceList.Add(newSource);
task.ContinueWith(t =>
{
var source = taskSources.Take();
if (t.IsCanceled)
source.TrySetCanceled();
else if (t.IsFaulted)
source.TrySetException(t.Exception.InnerExceptions);
else if (t.IsCompleted)
source.TrySetResult(t.Result);
}, CancellationToken.None,
TaskContinuationOptions.PreferFairness,
TaskScheduler.Default);
}
return taskSourceList.Select(tcs => tcs.Task);
}
With this you can write:
public static async Task<bool> WhenAny(this IEnumerable<Task<bool>> tasks)
{
foreach (var task in tasks.Order())
if (await task)
return true;
return false;
}
You could wait for the first task to complete - if it's successful, return true immediately. Otherwise, wait for the next one to complete, and so on and so forth.
private async Task<bool> foundAtServers(int id, string[] servers)
{
var tasks = servers.Select(server => checkServer(id, server))
.ToList();
while(tasks.Any())
{
var task = await Task.WhenAny(tasks);
if(task.Result)
return true;
tasks.Remove(task);
}
return false;
}
I am using the DropNet API for connecting to DropBox. I am struggling around async/await concepts though.
I have a method is calling the api GetTokenAsync. The return type is void, and there is a success and failure action to callback.
public async Task<GetTokenResult> GetAuthorizationUrl()
{
var result = new GetTokenResult();
_dropNetClient.GetTokenAsync(
login =>
{
result.Url = _dropNetClient.BuildAuthorizeUrl(_authorizationCallback.ToString());
result.Success = true;
},
exception =>
{
result.Error = exception.ToDiagnosticString();
result.Success = false;
}
);
return result;
}
The problem? I am thinking that changing the return type to just GetTokenResult may return faster than the actions will, therefore my results will never get set. I cannot await the async method as it returns void.
This is the one concept around async/await that I cannot wrap my head around.
You might want to consider using a TaskCompletionSource:
public Task<GetTokenResult> GetAuthorizationUrl()
{
var tcs = new TaskCompletionSource<GetTokenResult>();
_dropNetClient.GetTokenAsync(
login => tcs.SetResult(new GetTokenResult {
Url = _dropNetClient.BuildAuthorizeUrl(
_authorizationCallback.ToString()),
Success = true
},
exception => tcs.SetResult(new GetTokenResult {
Error = exception.ToDiagnosticString(),
Success = true
});
return tcs.Task;
}
The returned task will only complete when the GetTokenAsync operation completes (via one of the callbacks), and you can await it from an async method.
I would personally use SetException instead of SetResult on failure though - so that if you await the returned task, it will throw the appropriate failure on an exception, rather than just setting the value differently. It's more idiomatically .NET-like.
EDIT: You could then change the code to return Task<string> instead:
public Task<string> GetAuthorizationUrl()
{
var tcs = new TaskCompletionSource<string>();
_dropNetClient.GetTokenAsync(
login => tcs.SetResult(_dropNetClient.BuildAuthorizeUrl
_authorizationCallback.ToString()),
exception => tcs.SetException(exception));
return tcs.Task;
}
Now the exception would be propagated within the task itself - if you await a task which faults, the exception is thrown at that point. No need for extra properties, etc - it's much closer to how you'd write the corresponding synchronous code.