Data structure to use when gathering response from async calls - c#

I am running this piece of code in my application.
public Task<BulkResponse<JObject>> GetRelatedObjectsAsync(IEnumerable<PrimaryObjectInfo> primaryObjectInfos)
{
var allSecondaries = new List<Tuple<int, List<JObject>>>();
var exceptionsDict = new ConcurrentDictionary<int, Exception>();
var relatedObjectsTasks = primaryObjectInfos.Select(async primaryObjectInfo =>
{
try
{
var secondaryObject = await objectManager.GetRelatedObjectsAsync(primaryObjectInfo);
allSecondaries.Add(Tuple.Create(primaryObjectInfo.Index, secondaryObject.ToList()));
}
catch (Exception ex)
{
exceptionsDict.TryAdd(primaryObjectInfo.Index, ex);
}
});
await Task.WhenAll(relatedObjectsTasks);
return ConvertToBulkResponse(allSecondaries, exceptionsDict);
}
When I run this code allSecondaries object sometimes returns valid list of results and sometimes the code ends up catching exceptions for the parallel threads I have for each primaryObjectInfo.
Async method objectManager.GetRelatedObjectsAsync() internally call 4-5 async functions and there are functions where parameters are passed by reference. (ref keyword)
Question:
Am I using the right data structure to consolidate the result from all parallel threads ?
If yes, what could be the reason I am getting different result every time?

You'd better split the execution from the results gathering:
public Task<BulkResponse<JObject>> GetRelatedObjectsAsync(
IEnumerable<PrimaryObjectInfo> primaryObjectInfos)
{
var relatedObjectsTasks = primaryObjectInfos
.Select(
async primaryObjectInfo =>
(primaryObjectInfo.Index,
await objectManager.GetRelatedObjectsAsync(primaryObjectInfo)))
.ToList();
try
{
await Task.WhenAll(relatedObjectsTasks);
}
catch
{
// observe each task's, exception
}
var allSecondaries = new List<(int index, List<JObject> related)>();
var exceptionsDict = new Dictionary<int, Exception>();
foreach (var relatedObjectsTask in relatedObjectsTasks)
{
try
{
allSecondaries.Add(relatedObjectsTask.Result);
}
catch (Exception ex)
{
exceptionsDict.Add(relatedObjectsTask.index, ex);
}
}
return ConvertToBulkResponse(allSecondaries, exceptionsDict);
}
You could look into each task's IsFaulted/Exception and IsCancelled properties instead of causing exceptions to be thrown:
foreach (var relatedObjectsTask in relatedObjectsTasks)
{
if (relatedObjectsTask.IsCancelled)
{
exceptionsDict.Add(
relatedObjectsTask.index,
new TaskCancelledException(relatedObjectsTask));
}
else if (relatedObjectsTask.IsFaulted)
{
exceptionsDict.TryAdd(relatedObjectsTask.index, ex);
}
else
{
allSecondaries.Add(relatedObjectsTask.Result);
}
}

Related

System.InvalidCastException: 'System.Linq.Expressions.PropertyExpression' to 'System.Linq.Expressions.MethodCallExpression'

I am trying to perform unit tests of a client that returns a task with a list of elements and iterates them, but what I have tried is not working for me.
IEnumerable<ExpenseNote> pendingExpenses = null;
try
{
pendingExpenses = await _expenseNoteService.GetPendingExpenseNotes().ConfigureAwait(false);
}
catch (Exception ex)
{
executionContext.Error($"Step 1-Final: Error when call \"getPendingExpenses()\". Detail: {ex.Message}", true);
throw;
}
if (!pendingExpenses.Any())
{
executionContext.Checkpoint("Step 1-Final: There are not pending expenses");
return;
}
int numApproveExpenses = 0;
executionContext.Checkpoint($"Step 2: Processing {pendingExpenses.Count()} pending expenses");
foreach (var expenseNote in pendingExpenses)
{
if (expenseNote.IsApprovable)
{
try
{
await _workflowsRRHHProxy.Approve(expenseNote.Id.ToString());
executionContext.Checkpoint($"Step 2.1: Expense with number {expenseNote.Number} has been approved ");
numApproveExpenses++;
}
catch (Exception ex)
{
executionContext.Error($"Step 2.1: Error when call \"Approve()\" with id: {expenseNote.Id} " +
$". Detail: {ex.Message}", false);
throw;
}
}
}
test example:
// ARRANGE
var mockExpenseNote = _mockFactory.CreateMock<DomainEntities.ExpenseNote>();
mockExpenseNote.Expects.One.Method(x => x.IsApprovable).WillReturn(true);
var expenseNote = mockExpenseNote.MockObject;
expenseNote.Id = Guid.NewGuid();
expenseNote.Number = "EX-APPROVE";
var expenseNotes = new List<DomainEntities.ExpenseNote>
{
expenseNote
};
Task<IEnumerable<DomainEntities.ExpenseNote>> task = Task.FromResult<IEnumerable<DomainEntities.ExpenseNote>>(expenseNotes);
var executionContext = _mockFactory.CreateMock<ITaskExecutionContext>();
executionContext.Expects.Between(1, 100).Method(method => method.Checkpoint("")).WithAnyArguments();
_expenseNoteService.Expects.One.Method(method => method.GetPendingExpenseNotes())
.WithAnyArguments()
.WillReturn(task);
var sut = GetServiceUnderTest();
// ACT
await sut.ProcessPendingExpenseNotes(executionContext.MockObject);
// ASSERT
_mockFactory.VerifyAllExpectationsHaveBeenMet();
but when i execute this test in " .WillReturn(task);" have this error:
System.InvalidCastException: Cannot cast object of type 'System.Linq.Expressions.PropertyExpression' to type 'System.Linq.Expressions.MethodCallExpression'..
Can Anyone Help me?
With mockExpenseNote.Expects.One.Method(x => x.IsApprovable).WillReturn(true); your lambda uses a property but you used the .Method() method so it expects you to use a method here (or call something else than .Method()).

Recursive method is returning first iteration object C#

static void Main(string[] args)
{
token objtoken = new token();
var location = AddLocations();
OutPutResults outPutResultsApi = new OutPutResults();
GCPcall gCPcall = new GCPcall();
OutPutResults finaloutPutResultsApi = new OutPutResults();
var addressdt = new AddressDataDetails();
finaloutPutResultsApi.addressDatas = new List<AddressDataDetails>();
Console.WriteLine("Hello World!");
List<string> placeId = new List<string>();
var baseUrl = "https://maps.googleapis.com/maps/api/place/textsearch/json?";
var apiKey = "&key=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx".Trim();
foreach (var itemlocations in location)
{
var searchtext = "query=" + itemlocations.Trim();
var finalUrl = baseUrl + searchtext + apiKey;
gCPcall.RecursiveApiCall(finalUrl, ref placeId, objtoken.NextToken);
}
var ids = gCPcall.myPalceid;
}
public List<string> RecursiveApiCall(string finalUrl, ref List<string> placeId, string nextToken = null)
{
try
{
var token = "&pagetoken=" + nextToken;
using (var client = new HttpClient())
{
var responseTask = client.GetAsync(finalUrl + token);
responseTask.Wait();
var result = responseTask.Result;
if (result.IsSuccessStatusCode)
{
var readTask = result.Content.ReadAsStringAsync();
readTask.Wait();
var students = readTask.Result;
Rootobject studentsmodel = JsonConvert.DeserializeObject<Rootobject>(students);
nextToken = studentsmodel.next_page_token;
foreach (var item in studentsmodel.results)
{
placeId.Add(item.place_id);
}
}
}
if (nextToken != null)
{
RecursiveApiCall(finalUrl, ref placeId, nextToken);
}
return placeId;
}
catch (Exception ex)
{
throw;
}
}
My recursive method has some issue. Here whenever I am debugging this code it work fine. It goes in recursive call twice.
As debugging result I am getting list place_id with 20 items in first call and next call 9 items total 29 items in place_id object which is correct in static main method.
But if I run without debugging mode I am getting only 20 place_id. next recursive iteration data is missing even if it has next valid token.
I don't have any clue why this is happening. Can someone tell me what is the issue with my code?
Here are my suggestions, which may or may not solve the problem:
// First of all, let's fix the signature : Go async _all the way_
//public List<string> RecursiveApiCall(string finalUrl, ref List<string> placeId, string nextToken = null)
// also reuse the HttpClient!
public async Task ApiCallAsync(HttpClient client, string finalUrl, List<string> placeId, string nextToken = null)
{
// Loop, don't recurse
while(!(nextToken is null)) // C# 9: while(nextToken is not null)
{
try
{
var token = "&pagetoken=" + nextToken;
// again async all the way
var result = await client.GetAsync(finalUrl+token);
if (result.IsSuccessStatusCode)
{
// async all the way!
var students = await result.Content.ReadAsStringAsync();
Rootobject studentsmodel = JsonConvert.DeserializeObject<Rootobject>(students);
nextToken = studentsmodel.next_page_token;
foreach (var item in studentsmodel.results)
{
// Will be reflected in main, so no need to return or `ref` keyword
placeId.Add(item.place_id);
}
}
// NO recursion needed!
// if (nextToken != null)
// {
// RecursiveApiCall(finalUrl, ref placeId, nextToken);
// }
}
catch (Exception ex)
{
// rethrow, only is somewhat useless
// I'd suggest using a logging framework and
// log.Error(ex, "Some useful message");
throw;
// OR remove try/catch here all together and wrap the call to this method
// in try / catch with logging.
}
}
Mind that you'll need to make your main :
async Task Main(string[] args)
and call this as
await ApiCallAsync(client, finalUrl, placeId, nextToken);
Also create an HttpClient in main and reuse that:
"HttpClient is intended to be instantiated once and re-used throughout the life of an application. Instantiating an HttpClient class for every request will exhaust the number of sockets available under heavy loads. This will result in SocketException errors." - Remarks
... which shouldn't do much harm here, but it's a "best practice" anyhow.
Now, as to why you get only 20 instead of expected 29 items, I cannot say if this resolves that issue. I'd highly recommend to introduce a Logging Framework and make log entries accordingly, so you may find the culprid.

Return the results of Tasks - including the exceptions?

The following code run M1, M2, M3, M4 in parallel. Each method may raise exceptions. The method should return the results of the four async methods - either the int returned by methods or the Exceptions.
async Task<string> RunAll()
{
int m1result, m2result, m3result, m4result;
try
{
var m1task = M1();
var m2task = M2();
var m3task = M3();
var m4task = M4();
// await Task.WhenAll(new Task<int>[] { m1task, m2task, m3task, m4task });
m1result = await m1task;
m2result = await m2task;
m3result = await m3task;
m4result = await m4task;
}
catch (Exception ex)
{
// need to return the ex of the failed task. How?
}
// How to implement M1HasException, M2HasException, ... in the following lines?
var m1msg = M1HasException ? M1ExceptionMessage : m1result.ToString();
var m2msg = M2HasException ? M2ExceptionMessage : m2result.ToString();
var m3msg = M3HasException ? M3ExceptionMessage : m3result.ToString();
var m4msg = M4HasException ? M4ExceptionMessage : m4result.ToString();
return $"M1: {m1msg}, M2: {m2msg}, M3: {m3msg}, M4: {m4msg}";
}
How to capture the individual exceptions of the failed task?
For example, if only M2 threw an exception,
"M1: 1, M2: Excpetion...., M3: 3, M4: 4"
Each task has a Status and and Exception property.
You may want to see if it has faulted:
myTask.Status == TaskStatus.Faulted
Or if it has excepted:
if (myTask.Exception != null)
You can use ContinueWhenAll to run all the tasks and then check the status.
See the docs here.
As other answers/comments pointed out, one possible approach is by using ContinueWith or ContinueWhenAll. This is a clever trick because Task has the Exception property:
Gets the AggregateException that caused the Task to end prematurely.
If the Task completed successfully or has not yet thrown any
exceptions, this will return null.
Using ContinueWith whether a task completes successfully or not, it will be passed as the argument to the delegate function. From there you can check if an exception was thrown.
Task<string> GetStringedResult<T>(Task<T> initialTask)
{
return initialTask.ContinueWith(t => {
return t.Exception?.InnerException.Message ?? t.Result.ToString();
});
}
async Task<string> RunAll()
{
string m1result, m2result, m3result, m4result;
var m1task = GetStringedResult(M1());
var m2task = GetStringedResult(M2());
var m3task = GetStringedResult(M3());
var m4task = GetStringedResult(M4());
m1result = await m1task;
m2result = await m2task;
m3result = await m3task;
m4result = await m4task;
return $"M1: {m1result}, M2: {m2result}, M3: {m3result}, M4: {m4result}";
}
You can wrap the tasks inside a WaitAll and catch the AggregateException (docs),
try
{
Task.WaitAll(new[] { task1, task2 }, token);
}
catch (AggregateException ae)
{
foreach (var ex in ae.InnerExceptions)
//Do what ever you want with the ex.
}
Could you wrap each await in try-catch block and capture the exception message if any, seems feasible...
var results = new List<string>();
try { results.Add(await t1); } catch { results.Add("Exception"); };
try { results.Add(await t2); } catch { results.Add("Exception"); };
try { results.Add(await t3); } catch { results.Add("Exception"); };
return string.Join("|", results);
if you want to use WhenAll you could await for it and ignore exceptions and then do the same exercise as shown above to retrieve individual task results...
try { await Task.WhenAll(t1, t2, t3); } catch { };
// ^^^^^^^^^
// then same as ^ above

async processing in c#

I have a service that implements functionality to return data from sentiment analysis APIs. The client can request results from one or all engines and I want to collate all the data together. I want to process these async and wait for them all to complete before returning the result set. I'm new to async programming and I really cant figure out how to arrange the code and how to implement it syntactically. Here's an EXAMPLE of what I'm TRYING to achieve (I know this doesn't work, but you get the idea; hopefully :-) ):
private ISentimentResponse ProcessRequest(ISentimentRequest request, SentimentEngineServices selectedEngines)
{
SentimentResponse response = new SentimentResponse();
List<Task> taskList = new List<Task>();
foreach (SentimentEngineServices engineService in (SentimentEngineServices[])Enum.GetValues(typeof(SentimentEngineServices)))
{
if (((int)engineService & (int)selectedEngines) > 0)
{
ISentimentEngine engine = _engineFactory.GetSentimentEngine(engineService, null);
Task<ISentimentEngineResult> task = new Task<ISentimentEngineResult>(engine.AnalyseSentimentASync(request));
taskList.Add(task);
}
}
if (taskList.Count > 0)
{
ISentimentEngineResult[] results = Task<ISentimentEngineResult>.WaitAll(taskList);
foreach (result in results)
response.Add(results);
}
return response;
}
The engine has the following code implementation of engine.AnalyseSentimentASync:
public ISentimentEngineResult AnalyseSentiment(ISentimentRequest request)
{
try
{
MultiLanguageBatchInput sentimentList = SentimentRequestToMicrosoftBatchInput(request, Properties.Settings.Default.DefaultLanguage);
SentimentBatchResult sentiment = _client.Sentiment(sentimentList);
KeyPhraseBatchResult keyPhrases = _client.KeyPhrases(sentimentList);
return MicrosoftBatchResultsToSentimentEngineResult(sentiment, keyPhrases);
}
catch (Exception ex)
{
_logger.LogMessage(ex,$"{EngineName} threw an unknown exception: ", LoggingLevel.Error);
throw;
}
}
public async Task<ISentimentEngineResult> AnalyseSentimentAsync(ISentimentRequest request)
{
return AnalyseSentiment(request);
}
What do I need to do and is there any better way to achieve this?
I've looked everywhere for an example but I cant find one that addresses my implementation requirements, or the whole approach is completely wrong!
Thanks all,
Stu.
This is how you can convert it to async:
public async Task<ISentimentEngineResult> AnalyseSentimentAsync(ISentimentRequest request)
{
try
{
MultiLanguageBatchInput sentimentList = SentimentRequestToMicrosoftBatchInput(request, Properties.Settings.Default.DefaultLanguage);
SentimentBatchResult sentiment = await _client.SentimentAsync(sentimentList);
KeyPhraseBatchResult keyPhrases = await _client.KeyPhrasesAsync(sentimentList);
return MicrosoftBatchResultsToSentimentEngineResult(sentiment, keyPhrases);
}
catch (Exception ex)
{
_logger.LogMessage(ex,$"{EngineName} threw an unknown exception: ", LoggingLevel.Error);
throw;
}
}
private async Task<ISentimentResponse> ProcessRequestAsync(ISentimentRequest request, SentimentEngineServices selectedEngines)
{
SentimentResponse response = new SentimentResponse();
List<Task<ISentimentEngineResult>> taskList = new List<Task<ISentimentEngineResult>>();
foreach (SentimentEngineServices engineService in (SentimentEngineServices[])Enum.GetValues(typeof(SentimentEngineServices)))
{
if (((int)engineService & (int)selectedEngines) > 0)
{
ISentimentEngine engine = _engineFactory.GetSentimentEngine(engineService, null);
Task<ISentimentEngineResult> task = engine.AnalyseSentimentASync(request);
taskList.Add(task);
}
}
if (taskList.Count > 0)
{
ISentimentEngineResult[] results = await Task.WhenAll(taskList);
foreach (result in results)
response.Add(results);
}
return response;
}
Remember that you have to call it from some kind of event handler. I don't know what framework you are using (wpf, asp.net, windows service, webapi).
Ok, so here it is:
public ISentimentResponse AnalyseSentiment(ISentimentRequest request, SentimentEngineServices selectedEngines)
{
if (selectedEngines == SentimentEngineServices.None) throw new ArgumentException(nameof(selectedEngines));
ValidateRequest(request);
return ProcessRequestAsync(request, selectedEngines).Result;
}
private async Task<ISentimentResponse> ProcessRequestAsync(ISentimentRequest request, SentimentEngineServices selectedEngines)
{
SentimentResponse response = new SentimentResponse();
List<Task<ISentimentEngineResult>> taskList = new List<Task<ISentimentEngineResult>>();
foreach (SentimentEngineServices engineService in (SentimentEngineServices[])Enum.GetValues(typeof(SentimentEngineServices)))
{
if (((int)engineService & (int)selectedEngines) > 0)
{
ISentimentEngine engine = _engineFactory.GetSentimentEngine(engineService, null);
Task<ISentimentEngineResult> task = engine.AnalyseSentimentASync(request);
taskList.Add(task);
}
}
if (taskList.Count > 0)
{
ISentimentEngineResult[] results = await Task.WhenAll(taskList);
foreach (var result in results)
response.Add(result);
}
return response;
}
And the sentiment interface implementation:
public async Task<ISentimentEngineResult> AnalyseSentiment(ISentimentRequest request)
{
try
{
MultiLanguageBatchInput sentimentList = SentimentRequestToMicrosoftBatchInput(request, Properties.Settings.Default.DefaultLanguage);
SentimentBatchResult sentiment = await _client.SentimentAsync(sentimentList);
KeyPhraseBatchResult keyPhrases = await _client.KeyPhrasesAsync(sentimentList);
return MicrosoftBatchResultsToSentimentEngineResult(sentiment, keyPhrases);
}
catch (Exception ex)
{
_logger.LogMessage(ex,$"{EngineName} threw an unknown exception: ", LoggingLevel.Error);
throw;
}
}
Thanks FCin.

Is it possible that async code takes more time than sync?

I have a piece of code that execute HTTP requests (inside a loop) from some currency API and do some work on the data which received.
Because it was slow, I changed it to asynchronously function using: Task.Run()
However, when I measure calculation time (using Stopwatch..) of the function
with the async method, it takes more time (or even) than the sync one.
How can it possible? Am I do it wrong?
[HttpPost,Route("GetCurrenciesData")]
public IHttpActionResult GetCurrenciesData(InputCurrenciesData cls)
{
try
{
var watch = Stopwatch.StartNew();
var data2 = GetBL.mySyncMethod(cls.currenciesList);
watch.Stop();
string first = watch.Elapsed.ToString(); // --> this faster
watch = Stopwatch.StartNew();
var data = GetBL.myAsyncMethod(cls.currenciesList);
string second = watch.Elapsed.ToString(); // --> this slower
return Ok(data);
}
catch (Exception ex)
{
getGeneric.WriteError(ex.Message, ex.StackTrace);
return BadRequest();
}
}
Example code of the sync function:
public GenericClass mySyncMethod(string[] currencies)
{
try
{
foreach (string currency in currencies)
{
getDataFromApi(currency);
}
return getDataChart;
}
catch (Exception ex)
{
WriteError(ex.Message, ex.StackTrace);
return null;
}
}
Example of the async :
public GenericClass myAsyncMethod(string[] currencies)
{
try
{
List<Task> TaskList = new List<Task>();
foreach (string currency in currenies)
{
var myTask = new Task(() =>
{
getDataFromApi(currency);
});
myTask.Start();
TaskList.Add(myTask);
}
Task.WaitAll(TaskList.ToArray());
return getDataChart;
}
catch (Exception ex)
{
WriteError(ex.Message, ex.StackTrace);
return null;
}
}
The code of the function getDataFromApi:
private void getDataFromApi(string currency)
{
string currencyCode = getCurrenciesDictionary[currency];
//Get api's URL from Web.config and concatenation the wanted currency code
string URL = string.Format(GetConfig("Api"), currencyCode);
/*HTTP request to retrieve data from Api*/
using (HttpClientHandler handler = new HttpClientHandler())
{
handler.UseDefaultCredentials = true;
using (HttpClient httpClient = new HttpClient(handler))
{
var ResultAsync = httpClient.GetAsync(URL).Result;
if (ResultAsync.IsSuccessStatusCode)
{
var content = ResultAsync.Content;
string Data = content.ReadAsStringAsync().Result;
try
{
var ret = JsonConvert.DeserializeObject<RootObject>(Data);
/*add new class to currencyRate list - class which represent the currency and its rates*/
getDataChart.CurrencyRate.Add(new CurrencyRate
{
Currency = currency,
Rates = ret.dataset.data.AsEnumerable().
Select(date => Double.Parse(date[1].ToString())).ToList()
});
}
catch (Exception ex)
{
WriteError(ex.Message + " currency: " + currency + "/n/ URL: " + URL, ex.StackTrace);
}
}
}
}
try and read some of this guy:
http://blog.stephencleary.com/2012/02/async-and-await.html
http://blog.stephencleary.com/2016/12/eliding-async-await.html
he's telling in a few words how it works. The main point is that you get some overhead using async/await, however, you're freeing resources. So any task that will run, will run on the synchronized context. but you can handle much more request.
Besides the fact that you are not really using async functionality with the .result kills actually the async part.
In a short line, doing it async doesn't necessarily speed it up. If you have more CPU-bound tasks, that can run simultaneously you will major performance, however, in your case, you will not performance, but resources like Kevin tells in his comments.
hope it helps!

Categories