I've been attempting to figure this out for a bit now but I can't seem to wrap my head around it. I have the following async task which calls other async tasks from my ClientFunctions object named "c".
public async Task RunAsync(Dictionary<int, Robots> botList)
{
this.botList = botList;
Task[] tasks = new Task[botList.Count]; //5 tasks for each bot, 5 bots total in a list
for (int i = 0; i < botList.Count; i++)
{
tasks[i] = botList[i].c.StartClient();
await tasks[i];
tasks[i] = botList[i].c.setConnection();
}
await Task.WhenAll(tasks);
Form1.Log("All done");
}
I await after StartClient() because it writes data to a shared file, and setConnection() reads the data from that file. I do this for all 5 bots.
The StartClient() function returns a Process, and I want to store that Process in each bot's class in a variable called "proc".
How would I go about storing the result while still being able to use a task array to wait for all 5 to complete?
Thanks.
Here's one possible implementation, assuming you want to StartClient on all bots sequentially and then call setConnection and await them all to finish.
public async Task RunAsync(Dictionary<int, Robots> botList)
{
this.botList = botList;
var tasks = new List<Task>();
foreach(var botKvp in botList)
{
var bot = botKvp.Value;
bot.proc = await bot.c.StartClient();
tasks.Add(bot.c.setConnection());
}
await Task.WhenAll(tasks);
Form1.Log("All done");
}
Task comes in two varieties: Task and Task<T>. You have an array of Task which does not define a return value. If you want to return a value you need to await a Task<T>. For instance if setConnection() should return a bool then it's signature should declare that as public Task<bool> setConnection(...)
Task[] tasks = new Task<Process>[botList.Count]
should be
Task<Process>[] tasks = new Task<Process>[botList.Count]
This works
bot.proc = await bot.c.StartClient();
Becuase StartClient() returns Task<Process> and await awaits that task and assigns the process to proc. As counter example this will fail:
Task procTask = bot.c.StartClient();
bot.proc = await procTask
When you await the task you get the result, so:
public async Task RunAsync(Dictionary<int, Robots> botList)
{
this.botList = botList;
Task[] tasks = new Task[botList.Count]; //5 tasks for each bot, 5 bots total in a list
for (int i = 0; i < botList.Count; i++)
{
tasks[i] = botList[i].c.StartClient();
botList[i].proc = await tasks[i];
tasks[i] = botList[i].c.setConnection();
}
await Task.WhenAll(tasks);
Form1.Log("All done");
}
If it were the setConnection() method that returned the item, the result of await Task.WhenAll(tasks) would hold a collection of items.
Related
I'm building a solution to find a desired value from an API call inside a for loop.
I basically need to pass to an API the index of a for loop, one of those index will return a desired output, in that moment I want the for loop to break, but I need to efficient this process. I thought making it asynchronous without the await, so that when some API returns the desired output it breaks the loop.
Each API call takes around 10sec, so if I make this async or multithread I would reduce the execution time considerably.
I haven't fount any good orientation of how making async / not await HTTP request.
Any suggestions?
for (int i = 0; i < 60000; i += 256)
{
Console.WriteLine("Incrementing_value: " + i);
string response = await client.GetStringAsync(
"http://localhost:7075/api/Function1?index=" + i.ToString());
Console.WriteLine(response);
if (response != "null")//
{
//found the desired output
break;
}
}
You can run requests in parallel, and cancel them once you have found your desired output:
public class Program {
public static async Task Main() {
var cts = new CancellationTokenSource();
var client = new HttpClient();
var tasks = new List<Task>();
// This is the number of requests that you want to run in parallel.
const int batchSize = 10;
int requestId = 0;
int batchRequestCount = 0;
while (requestId < 60000) {
if (batchRequestCount == batchSize) {
// Batch size reached, wait for current requests to finish.
await Task.WhenAll(tasks);
tasks.Clear();
batchRequestCount = 0;
}
tasks.Add(MakeRequestAsync(client, requestId, cts));
requestId += 256;
batchRequestCount++;
}
if (tasks.Count > 0) {
// Await any remaining tasks
await Task.WhenAll(tasks);
}
}
private static async Task MakeRequestAsync(HttpClient client, int index, CancellationTokenSource cts) {
if (cts.IsCancellationRequested) {
// The desired output was already found, no need for any more requests.
return;
}
string response;
try {
response = await client.GetStringAsync(
"http://localhost:7075/api/Function1?index=" + index.ToString(), cts.Token);
}
catch (TaskCanceledException) {
// Operation was cancelled.
return;
}
if (response != "null") {
// Cancel all current connections
cts.Cancel();
// Do something with the output ...
}
}
}
Note that this solution uses a simple mechanism to limit the amount of concurrent requests, a more advanced solution would make use of semaphores (as mentioned in some of the comments).
There are multiple ways to solve this problem. My personal favorite is to use an ActionBlock<T> from the TPL Dataflow library as a processing engine. This component invokes a provided Action<T> delegate for every data element received, and can also be provided with an asynchronous delegate (Func<T, Task>). It has many useful features, including (among others) configurable degree of parallelism/concurrency, and cancellation via a CancellationToken. Here is an implementation that takes advantage of those features:
async Task<string> GetStuffAsync()
{
var client = new HttpClient();
var cts = new CancellationTokenSource();
string output = null;
// Define the dataflow block
var block = new ActionBlock<string>(async url =>
{
string response = await client.GetStringAsync(url, cts.Token);
Console.WriteLine($"{url} => {response}");
if (response != "null")
{
// Found the desired output
output = response;
cts.Cancel();
}
}, new ExecutionDataflowBlockOptions()
{
CancellationToken = cts.Token,
MaxDegreeOfParallelism = 10 // Configure this to a desirable value
});
// Feed the block with URLs
for (int i = 0; i < 60000; i += 256)
{
block.Post("http://localhost:7075/api/Function1?index=" + i.ToString());
}
block.Complete();
// Wait for the completion of the block
try { await block.Completion; }
catch (OperationCanceledException) { } // Ignore cancellation errors
return output;
}
The TPL Dataflow library is built-in the .NET Core / .NET 5. and it is available as a package for .NET Framework.
The upcoming .NET 6 will feature a new API Parallel.ForEachAsync, that could also be used to solve this problem in a similar fashion.
Edit:
here is my main method
static void Main(string[] args)
{
LogNewOrders();
DataTable initialData = ControllerSqlAgent.SelectQuery(Resources.Controller, Resources.qryGetInitalData);
Console.WriteLine($"|There are {initialData.Rows.Count} orders to check|");
Task.WhenAll(UpdateItemCountsField(initialData));
}
I have a method called UpdateItemCountsField(datatable)
The purpose of this method is to get both:
total cancelled items
total shipped items
private static async Task UpdateItemCountsField(DataTable initialData)
{
try
{
foreach (DataRow row in initialData.Rows)
{
string narvarId = row["NarvarID"].ToString();
int orderedItemCount = (int)row["ItemsOrdered"];
int totalShippedItems = (int)row["ItemsShipped"]; ;
int totalCancelledItems = (int)row["ItemsCancelled"];
string locateConstraint = GetLocateInConstraint(row["OrderNumber"].ToString(), row["CompanyNumber"].ToString());
Task<int> totalShippedItemsTask = CheckShipmentCountsAsync(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalShippedItems, narvarId);
Task<int> totalCancelledItemsTask = CheckCancellationCountsAsync(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalCancelledItems, narvarId); ;
int[] result = await Task.WhenAll(totalShippedItemsTask, totalCancelledItemsTask);
//totalShippedItems = CheckShipmentCounts(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalShippedItems, narvarId);
//totalCancelledItems = CheckCancellationCounts(locateConstraint, row["OrderNumber"].ToString(), row["CompanyNumber"].ToString(), totalCancelledItems, narvarId);
Console.WriteLine($"|ID:{narvarId}|Ordered: {orderedItemCount}|Shipped: {result[0]}|Cancelled: {result[1]}|");
Console.WriteLine("|___________________________________________|");
}
}
catch
{
throw;
}
}
I have two Task objects that I obtain from async methods:
Task<int> totalShippedItemsTask = CheckShipmentCountsAsync(params);
Task<int> totalCancelledItemsTask = CheckCancellationCountsAsync(params);
Then i get my results like so (this is where the program crashes)
int[] result = await Task.WhenAll(totalShippedItemsTask,totalCancelledItemsTask);
within CheckShipmentCoutnsAsync(params) and CheckCancellationCountsAsync(params) are two more Task<int> objects . This is due to there being two data sources that i have to pull from
private static async Task<int> CheckShipmentCountsAsync(string locateConstraint, string orderNumber, string companyNumber, int currentShippedItemCount, string narvarId)
{
Task<int> wmsShippedCountTask = WmsSqlAgent.GetCountOfShippedItemsAsync(orderNumber, companyNumber);
Task<int> locateShippedCountTask = LocateSqlAgent.GetCountOfShippedItemsAsync(locateConstraint);
int[] result = await Task.WhenAll(wmsShippedCountTask, locateShippedCountTask);
int newShippedItemCount = result.Sum();
if (newShippedItemCount > currentShippedItemCount)
ControllerSqlAgent.UpdateShippedItemCount(narvarId, newShippedItemCount);
return newShippedItemCount;
}
The method CheckCancelCountsAsync(PARAMS) is the same as the above but for cancellations.
When my program runs, It hits the task and then crashes without any exception and never reaches the next datarow to perform the asynchronous methods on. Am I missing a piece here? Why would my application crash without an exception. the reason I believe it is crashing due to the fact that it never makes it to the Console.Writeline(); right after i get the result array from Task.WhenAll
And here is the data source method that retrieves the actual count
internal static async Task<int> GetCountOfCancelledItemsAsync(string orderNumber, string companyNumber)
{
int itemCount = 0;
await Task.Run(() =>
{
try
{
using (var connection = new SqlConnection(Resources.WMS))
{
connection.Open();
using (var command = new SqlCommand(Resources.qryWmsGetCancelledItemCount, connection))
{
command.Parameters.AddWithValue("#orderNumber", orderNumber);
command.Parameters.AddWithValue("#companyNumber", companyNumber);
itemCount = int.Parse(command.ExecuteScalar().ToString());
}
connection.Close();
}
}
catch (SqlException E)
{
string message = E.Message;
throw;
}
return itemCount;
});
return itemCount;
While stepping through the code I can see that my program crashes when i get to nt[] result = await Task.WhenAll(totalShippedItemsTask, totalCancelledItemsTask); but still no exception.
You have not awaited the results of Task.WhenAll(UpdateItemCountsField(initialData)) in your main method. This means that the program will exit without waiting for that code to finish executing.
Change your method to this:
static async Task Main(string[] args)
{
LogNewOrders();
DataTable initialData = ControllerSqlAgent.SelectQuery(Resources.Controller, Resources.qryGetInitalData);
Console.WriteLine($"|There are {initialData.Rows.Count} orders to check|");
await UpdateItemCountsField(initialData);
}
Since UpdateItemCountsField returns a single Task, the Task.WhenAll wrapping it was redundant, so I removed that from the code in my answer.
Note that you must be using C# 7.1 or later in order to have an async main method.
Make sure on any method that returns a Task that you await the result.
I am starting 2 channels in the mediaservices azure portal.
Starting a channel takes a long time to complete, about 25-30 seconds per channel. Hence, multithreading :)
However, the following is not clear to me:
I have 2 methods:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
var workerThreads = new List<Thread>();
var results = new List<bool>();
foreach (var azureProgram in _accounts.GetPrograms(programName))
{
var thread = new Thread(() =>
{
var result = StartChannelAsync(azureProgram).Result;
lock (results)
{
results.Add(result);
}
});
workerThreads.Add(thread);
thread.Start();
}
foreach (var thread in workerThreads)
{
thread.Join();
}
return results.All(r => r);
}
and
private async Task<bool> StartChannelAsync(IProgram azureProgram)
{
var state = _channelFactory.ConvertToState(azureProgram.Channel.State);
if (state == State.Running)
{
return true;
}
if (state.IsTransitioning())
{
return false;
}
await azureProgram.Channel.StartAsync();
return true;
}
in the first method I use
var result = StartChannelAsync(azureProgram).Result;
In this case everything works fine. But if I use
var result = await StartChannelAsync(azureProgram);
Executing is not awaited and my results has zero entries.
What am I missing here?
And is this a correct way?
Any comments on the code is appreciated. I am not a multithreading king ;)
Cheers!
Don't span new Thread instances to execute tasks in parallel, instead use Task.WhenAll:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
// Create a task for each program and fire them "at the same time"
Task<bool>[] startingChannels = _accounts.GetPrograms(programName))
.Select(n => StartChannelAsync(n))
.ToArray();
// Create a task that will be completed when all the supplied tasks are done
bool[] results = await Task.WhenAll(startingChannels);
return results.All(r => r);
}
Note: I see that you're passing a CancellationToken to your StartAsync method, but you're not actually using it. Consider passing it as an argument to StartChannelAsync, and then use it when calling azureProgram.Channel.StartAsync
If you love one-liners:
public async Task<bool> StartAsync(string programName, CancellationToken token = default(CancellationToken))
{
return (await Task.WhenAll(_accounts.GetPrograms(programName)
.Select(p => StartChannelAsync(p))
.ToArray())).All(r => r);
}
Hello I'm wondering if there is a cleaner way of writing the async code below. Basically I want to wait on all the tasks, but one of the tasks are optional. It feels needlessly elaborate, thinking if I can do it through some callback but haven't been able to figure it out.
var mobile = true;
var task1 = _service.Async1();
var tasks = new List<Task>
{
task1
};
Task<int> task2 = null;
if (!mobile)
{
task2 = _service.Async2();
tasks.Add(task2);
}
await Task.WhenAll(tasks);
var result1 = task1.Result;
if (!mobile)
{
result2 = task2.Result;
// Do stuff
}
There is no need to create a list and await all the results at once. Why not await it when you need it? If task2 runs much longer than task1, you can at least start processing it, long before task2 is done.
Something like this:
var task1 = _service.Async1();
Task<int> task2 = null;
if (!mobile)
{
task2 = _service.Async2();
}
var result1 = await task1;
if (!mobile)
{
var result2 = await task2;
// Do stuff
}
I saw couple of posts on this... but most people are manually creating tasks rather than receiving it from an API.
I have this code
Task[] tasks = new Task[companyCount];
// start all threads
for(int i = 0; i < companyCount; i++) {
string input = GetNextCompanyRecord(i);
string token = GetUserToken();
HttpClient client = GetHttpClient();
StringContent content = CreateStringContent(input);
var x = client.PostAsync(companyUrlString, content).ConfigureAwait(continueOnCapturedContext: false);
tasks[i] = x;
}
Task.WaitAll(tasks);
the objective is that I start the task for each record in a for loop and then when all tasks are started ... doing a single waitall.
problem is that I cannot assign the x variable to the task array.
The PostAsync method is not returning me the Task. instead it is returning ConfiguredTaskAwaitable. But my hope was that I will get Task and I will be able to add that to my array and then do a waitall.
Just do it like this:
Task<HttpResponseMessage>[] tasks = new Task<HttpResponseMessage>[companyCount];
for(int i = 0; i < companyCount; i++)
{
string input = GetNextCompanyRecord(i);
string token = GetUserToken();
HttpClient client = GetHttpClient();
StringContent content = CreateStringContent(input);
var x = client.PostAsync(companyUrlString, content);
tasks[i] = x;
}
Task.WaitAll(tasks);
But, perhaps a cleaner way would be to do this:
var tasks =
from i in Enumerable.Range(0, companyCount)
let input = GetNextCompanyRecord(i)
let token = GetUserToken()
let client = GetHttpClient()
let content = CreateStringContent(input)
select client.PostAsync(companyUrlString, content);
Task.WaitAll(tasks.ToArray());
Or even better yet:
using (var client = GetHttpClient())
{
var tasks =
from i in Enumerable.Range(0, companyCount)
let input = GetNextCompanyRecord(i)
let token = GetUserToken()
let content = CreateStringContent(input)
select client.PostAsync(companyUrlString, content);
Task.WaitAll(tasks.ToArray());
}
PostAsync is returning a Task (a Task<HttpResponseMessage>, to be precise), but then you're calling ConfigureAwait on it, which returns a ConfiguredTaskAwaitable.
Try getting a reference to the task first, and then calling ConfigureAwait.
var task = client.PostAsync(companyUrlString, content);
ConfiguredTaskAwaitable awaitable = task.ConfigureAwait(continueOnCapturedContext: false);
tasks[i] = task;
Also, consider:
removing the call to ConfigureAwait, since you don't seem to need it anyway.
changing your array from Task[] to Task<HttpResponseMessage>[], if you want to retrieve the tasks' results when they're done.