We Keep Coding

Issue With HttpClient Bulk Parallel Request in .Net Core C#

So I have been struggling with this issue for like 3 weeks. Here's what I want to do.
So I have like 2000 stock options. I want to fetch 5 of them at a time and process but it all has to be parallel. I'll write them in steps to make it more clear.
Get 5 stock symbols from an array
Send it to fetch its data and process. Don't wait for a response keep on processing.
wait 2.6 seconds (as we are limited to 120 API requests per minute so this delay helps in keeping it throttled to 115 per minute)
Goto 1
All the steps above have to be parallel. I have written the code for it and it all seems to be working fine but randomly it crashes saying
"A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond".
And sometimes it'll never happen and everything works like a charm.
This error is very random. It could show up on maybe 57th stock or maybe at 1829th stock. I have used HttpClient for it. I have tested this same scenario using Angular and creating custom requests and it never crashes there so it's not third-party server's fault.
What I have already done:
Changed HttpClient class usage from new instances every time to a single instance for the whole project.
Increases Service point manager Connection limit to a different number. (Default for .net core is 2)
Instead of HttpClient Queuing I have used SemaphoreSlim for queue and short-circuiting.
Forced ConnectionLeaseTimeout to 40 seconds to detect DNS changes if any.
Changed Async Tasks to threading.
Tried almost everything from the internet.
My doubts:
I doubt that it has something to do with the HttpClient class. I have read a lot of bad things about its misleading documentation etc.
My friend's doubt:
He said it could be because of concurrent tasks and I should change it to threads.
Here's the code:
// Inside Class Constructor
private readonly HttpClient HttpClient = new HttpClient();
SetMaxConcurrency(ApiBaseUrl, maxConcurrentRequests);
// SetMaxConcurrency function
private void SetMaxConcurrency(string url, int maxConcurrentRequests)
{
ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLimit = maxConcurrentRequests;
ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLeaseTimeout = 40*1000;
}
// code for looping through chunks of symbol each chunk has 5 symbols/stocks in it
foreach(var chunkedSymbol in chunkedSymbols)
{
//getting o auth token
string AuthToken = await OAuth();
if(String.IsNullOrEmpty(AuthToken))
{
throw new ArgumentNullException("Access Token is null!");
}
processingSymbols += chunkSize;
OptionChainReq.symbol = chunkedSymbol.ToArray();
async Task func()
{
//function that makes request
var response = await GetOptionChain(AuthToken, ClientId, OptionChainReq);
// concat the result in main list
appResponses = appResponses.Concat(response).ToList();
}
// if request reaches 115 process the remaning requests first
if(processingSymbols >= 115)
{
await Task.WhenAll(tasks);
processingSymbols = 0;
}
tasks.Add(func());
// 2600 millisecond delay to wait for all the data to process
await Task.Delay(delay);
}
//once the loop is completed process the remaining requests
await Task.WhenAll(tasks);
// This code processes every symbol. this code is inside GetOptionChain()
try{
var tasks = new List<Task>();
foreach (string symbol in OptionChainReq.symbol)
{
List<KeyValuePair<string, string>> Params = new List<KeyValuePair<string, string>>();
string requestParams = string.Empty;
// Converting Request Params to Key Value Pair.
Params.Add(new KeyValuePair<string, string>("apikey" , ClientId));
// URL Request Query parameters.
requestParams = new FormUrlEncodedContent(Params).ReadAsStringAsync().Result;
string endpoint = ApiBaseUrl + "/marketdata/chains?";
HttpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", OAuthToken);
Uri tosUri = new Uri(endpoint + requestParams, UriKind.Absolute);
async Task func()
{
try{
string responseString = await GetTosData(tosUri);
OptionChainResponse OptionChainRes = JsonConvert.DeserializeObject<OptionChainResponse>(responseString);
var mappedOptionAppRes = MapOptionsAppRes( OptionChainRes );
if(mappedOptionAppRes != null)
{
OptionsData.Add( mappedOptionAppRes );
}
}
catch(Exception ex)
{
throw new Exception("Crashed");
}
}
// asyncronusly processing each request
tasks.Add(func());
}
//making sure all 5 requests are processed
await Task.WhenAll(tasks);
}
catch (Exception ex)
{
failedSymbols += " "+ string.Join(",", OptionChainReq.symbol);
}
// The code below is for individual request
public async Task<string> GetTosData(Uri url)
{
try
{
await semaphore.WaitAsync();
if (IsTripped())
{
return UNAVAILABLE;
}
var response = await HttpClient.GetAsync(url);
if(response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
{
string OAuthToken = await OAuth();
HttpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", OAuthToken);
return await GetTosData(url);
}
else if(response.StatusCode != HttpStatusCode.OK)
{
TripCircuit(reason: $"Status not OK. Status={response.StatusCode}");
return UNAVAILABLE;
}
return await response.Content.ReadAsStringAsync();
}
catch(Exception ex) when (ex is OperationCanceledException || ex is TaskCanceledException)
{
Console.WriteLine("Timed out");
TripCircuit(reason: $"Timed out");
return UNAVAILABLE;
}
finally
{
semaphore.Release();
}
}

ContinueWith not waiting for task to complete

I have a function (below) that I retrieve data from an API. If I set a breakpoint at the line that deserializes it, then I can see that it is populated with data which is great.
When I continue on, it goes into the second function (below) and it throws an error. The error says next to it Not yet computed, and therefore throwing an exception.
When I do it with a small list it works just fine (I presume its cos it's a small set of data).
How is this possible when I'm using ContinueWith (waiting for the task to complete)?
public static async Task<Data> GetAllCardsInSet(string setName)
{
setName = WebUtility.UrlEncode(setName);
var correctUri = Path.Combine(ApiConstants.YugiohGetAllCardsInSet, setName);
Console.WriteLine();
using (var httpClient = new HttpClient())
{
var response =
await httpClient.GetAsync(correctUri);
var result = await response.Content.ReadAsStringAsync();
var cardData = JsonConvert.DeserializeObject<CardSetCards>(result);
for (int i = 0; i < cardData.Data.Cards.Count; i++)
{
cardData.Data.Cards[i] = FormatWords(cardData.Data.Cards[i]);
}
return cardData.Data;
}
}
private void GetYugiohCardsAndNavigate(string name)
{
var cardSetData = YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name).ContinueWith((result) =>
{
//var cards = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
try
{
this.mainPage.NavigateToYugiohCardListPage(result.Result);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App", "Sorry, we could not fetch this set");
}
});
}

Your GetAllCardsInSet method no need to change.
But using of this method can be refactored.
Method GetAllCardsInSet return Task and you not observed the completion of the this Task.
You need to check is Task completes succesfully, easiest approach to use await keyword. Awaiting task will unwrapp returned value or throw exception if task completed with exception.
For using async/await in the GetYugiohCardsAndNavigate change method signature to aynchronous and returning Task
private async Task GetYugiohCardsAndNavigate(string name)
{
try
{
var cardSetData = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
this.mainPage.NavigateToYugiohCardListPage(cardSetData);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App",
"Sorry, we could not fetch this set");
}
}

you called an async method in a sync method without Wait. It should have been done like:
YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name).ContinueWith((result) =>
{
//var cards = await YugiohRequester.GetAllCardsInSet(_selectedCardSet.Name);
try
{
this.mainPage.NavigateToYugiohCardListPage(result.Result);
}
catch (Exception e)
{
HelperFunctions.ShowToastNotification("Trading Card App", "Sorry, we could not fetch this set");
}
}).Wait();

Execute blocked while waiting for an asynchronous operation to complete

I am trying to make multithreaded my program but I am always getting the above error. I have read some posts and I saw that MVC 4 is not completely support multhreading . The little part what am I tried to change as multithreaded in controller you can see here :
public async Task<ActionResult> Status(bool? checkPage)
{
try
{
bool control = checkPage.GetValueOrDefault(false);
var result = Task.Factory.StartNew(() => dashboardMngr.getComputerInfo());
var resultDisabled = Task.Factory.StartNew(() => dashboardMngr.getStatus(dashboardMngr.getComputerInfo().Result));
Task.WaitAll(result, resultDisabled);
ViewData["Info"] = result.Result.Result;
ViewData["Status"] = resultDisabled.Result.Result;
}
catch (Exception e)
{
System.Diagnostics.Debug.WriteLine("Exception Raised in Stat ---> " + e.Message);
}
return View();
}

await Task.WhenAll(tasks) Exception Handling, log all exceptions from the tasks

I am trying to figure out how to report all exceptions thrown by a list of tasks from the code below.
The basic idea of this code snippet is: The user sends a request to the handler, the handler creates the messages tasks and send them to a class that send them to the external system. I included the methods involved below.
I must have something off because I was debugging the exception handler and the tasks Exception are always null because it seems their status is Waiting for Activiation unless I stay in the breakpoint long enough.
// Handle the user request
public async void Handle(WriteScanToSys settings)
{
_successfulScanIds = new List<int>();
// create the messages as tasks
var tasks = _factory.CreateMessage(settings).Select(msg => SendScans(msg));
try
{
// wait for all of them to complete
await Task.WhenAll(tasks); // I had ConfigureAwait(false) here, but took it off
}
catch (Exception)
{
foreach (var task in tasks.Where(t => t.Exception != null))
{
// ELMAH
var errorLog = ErrorLog.GetDefault(null);
errorLog.Log(new Error(task.Exception));
}
}
// save to repository
}
// the task to perform
private async Task<IDictionary<string, object>> SendScans(IDictionary<string, object> data)
{
object sysMsg = null;
var response = await _msgCenter.SendMessage(data);
response.TryGetValue("SystemMessage", out sysMsg);
_successfulScanIds.Add(Convert.ToInt32(data["Id"]));
return response;
}
// the communication with the external system (The message center class)
private async Task<IDictionary<string, object>> SendMessage(IDictionary<string, object> msg)
{
var response = new Dictionary<string, object>();
var response = await _client.sendAsync(
new BodyOfRequest(
// Compose Object
));
if (response.ScreenMessage != "SUCCESSFUL")
throw new CustomException("The transaction for job " + job + " failed with msg: " + body.ScreenMessage);
response.Add("SystemMessage", body.ScreenMessage);
return response;
}

You've fallen foul of lazy evaluation - the result of Select will create a new set of tasks each time you iterate over it. You can fix this just by calling ToList():
var tasks = _factory.CreateMessage(settings)
.Select(msg => SendScans(msg))
.ToList();
That way the set of tasks that you're awaiting will be the same set of tasks checked with your foreach loop.

Instead of iterating over all tasks, you can get the Exceptions (if any) from the Task.WhenAll-Task:
var taskResult = Task.WhenAll(tasks);
try
{
await taskResult;
}
catch (Exception e)
{
if (taskResult.IsCanceled)
{
// Cancellation is most likely due to a shared cancellation token. Handle as needed, possibly check if ((TaskCanceledException)e).CancellationToken == token etc.
}
else if (taskResult.IsFaulted)
{
// use taskResult.Exception which is an AggregateException - which you can iterate over (it's a tree! .Flatten() might help)
// caught exception is only the first observed exception
}
else
{
// Well, this should not really happen because it would mean: Exception thrown, not faulted nor cancelled but completed
}
}

Combining a while loop with Task.Run() in C#

I'm pretty new to multithread applications in C# and I'm trying to edit my code below so that it runs on multiple threads. Right now it operates synchronously and it takes up very little cpu power. I need it to run much faster on multiple threads. My thought was starting a task for each core and then when a task finishes, allow another to take its place or something like that if it is possible.
static void Main(string[] args)
{
string connectionString = CloudConfigurationManager.GetSetting("Microsoft.ServiceBus.ConnectionString");
QueueClient Client = QueueClient.CreateFromConnectionString(connectionString, "OoplesQueue");
try
{
while (true)
{
Task.Run(() => processCalculations(Client));
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.StackTrace);
}
}
public static ConnectionMultiplexer connection;
public static IDatabase cache;
public static async Task processCalculations(QueueClient client)
{
try
{
BrokeredMessage message = await client.ReceiveAsync();
if (message != null)
{
if (connection == null || !connection.IsConnected)
{
connection = await ConnectionMultiplexer.ConnectAsync("connection,SyncTimeout=10000,ConnectTimeout=10000");
//connection = ConnectionMultiplexer.Connect("connection,SyncTimeout=10000,ConnectTimeout=10000");
}
cache = connection.GetDatabase();
string sandpKey = message.Properties["sandp"].ToString();
string dateKey = message.Properties["date"].ToString();
string symbolclassKey = message.Properties["symbolclass"].ToString();
string stockdataKey = message.Properties["stockdata"].ToString();
string stockcomparedataKey = message.Properties["stockcomparedata"].ToString();
List<StockData> sandp = cache.Get<List<StockData>>(sandpKey);
DateTime date = cache.Get<DateTime>(dateKey);
SymbolInfo symbolinfo = cache.Get<SymbolInfo>(symbolclassKey);
List<StockData> stockdata = cache.Get<List<StockData>>(stockdataKey);
List<StockMarketCompare> stockcomparedata = cache.Get<List<StockMarketCompare>>(stockcomparedataKey);
StockRating rating = performCalculations(symbolinfo, date, sandp, stockdata, stockcomparedata);
if (rating != null)
{
saveToTable(rating);
if (message.LockedUntilUtc.Minute <= 1)
{
await message.RenewLockAsync();
}
await message.CompleteAsync();
}
else
{
Console.WriteLine("Message " + message.MessageId + " Completed!");
await message.CompleteAsync();
}
}
}
catch (TimeoutException time)
{
Console.WriteLine(time.Message);
}
catch (MessageLockLostException locks)
{
Console.WriteLine(locks.Message);
}
catch (RedisConnectionException redis)
{
Console.WriteLine("Start the redis server service!");
}
catch (MessagingCommunicationException communication)
{
Console.WriteLine(communication.Message);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
Console.WriteLine(ex.StackTrace);
}
}

This looks like a classic producer-consumer pattern.
In this case, where you need concurrency combined with async IO bound operations (such as retrieving data from a Redis cache) and CPU bound operations (such as doing compute bound calculations), i'd leverage TPL Dataflow for the job.
You can use a ActionBlock<T> which is responsible for processing of a single action you pass to it. Behind the scenes, it takes care of concurrency, while you can limit it as you want by passing it an ExecutionDataflowBlockOptions.
You start off by creating the ActionBlock<BrokeredMessage>:
private static void Main(string[] args)
{
var actionBlock = new ActionBlock<BrokeredMessage>(async message =>
await ProcessCalculationsAsync(message),
new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = Environment.ProcessorCount
});
var produceMessagesTask = Task.Run(async () => await
ProduceBrokeredMessagesAsync(client,
actionBlock));
produceMessagesTask.Wait();
}
Now lets look what ProduceBrokeredMessageAsync. It simply receives your QueueClient and the ActionBlock to the the following:
private async Task ProduceBrokeredMessagesAsync(QueueClient client,
ActionBlock<BrokeredMessage> actionBlock)
{
BrokeredMessage message;
while ((message = await client.ReceiveAsync()) != null)
{
await actionBlock.SendAsync(message);
}
actionBlock.Complete();
await actionBlock.Completion;
}
What this does is while you receive messages from your QueueClient, it will asynchronously post the message to the ActionBlock, which will process those message concurrently.

Right now it operates synchronously and it takes up very little cpu power. I need it to run much faster on multiple threads.
"Multiple threads" doesn't necessarily mean "faster". That is only true if you have multiple calculations to perform that are independent of each other, and they are CPU-bound (meaning they mainly involve CPU operations, not IO operations).
Additionally, async doesn't necessarily mean multiple threads. It just means your operation is not blocking a process thread while in progress. If you're starting another thread and blocking it, then that looks like async but it really isn't. Check out this Channel 9 video: Async Library Methods Shouldn't Lie
Most of your operations in processCalculations look like they are dependent on each other; however, this part might be a potential improvement point:
List<StockData> sandp = cache.Get<List<StockData>>(sandpKey);
DateTime date = cache.Get<DateTime>(dateKey);
SymbolInfo symbolinfo = cache.Get<SymbolInfo>(symbolclassKey);
List<StockData> stockdata = cache.Get<List<StockData>>(stockdataKey);
List<StockMarketCompare> stockcomparedata = cache.Get<List<StockMarketCompare>>(stockcomparedataKey);
StockRating rating = performCalculations(symbolinfo, date, sandp, stockdata, stockcomparedata);
I'm not familiar with the API you're using but IF it includes an async equivalent of the Get method you might be able to do those IO operations asynchronously in parallel, e.g.:
var sandpTask = List<StockData> sandp = cache.GetAsync<List<StockData>>(sandpKey);
var dateTask = cache.GetAsync<DateTime>(dateKey);
var symbolinfoTask = cache.GetAsync<SymbolInfo>(symbolclassKey);
var stockdataTask = cache.GetAsync<List<StockData>>(stockdataKey);
var stockcomparedataTask = cache.GetAsync<List<StockMarketCompare>>(stockcomparedataKey);
await Task.WhenAll(sandpTask, dateTask,symbolinfoTask,
stockdataTask, stockcomparedataTask);
List<StockData> sandp = sandpTask.Result;
DateTime date = dateTask.Result;
SymbolInfo symbolinfo = symbolinfoTask.Result;
List<StockData> stockdata = stockdataTask.Result;
List<StockMarketCompare> stockcomparedata = stockcomparedataTask.Result;
StockRating rating = performCalculations(symbolinfo, date, sandp, stockdata, stockcomparedata);
Also, note that you don't need to wrap the processCalculations call in another Task since it already returns a task:
// instead of Task.Run(() => processCalculations(message));
processCalculations(message);

You need two parts:
Part 1 waits for an incoming message: ConnectAsync() this runs in a simple loop. Whenever something is received an instance of Part2 is started to process the incoming message.
Part2 runs in another thread / in the background and processes a single incoming message.
That way several instances of Part2 may run in parallel.
So your structure is like this:
while (true)
{
connection = await ConnectionMultiplexer.ConnectAsync(...);
StartProcessCalculationsInBackground(connection, ...); // return immediately
}