We Keep Coding

How do you set up some function to run in a background thread but keep the normal return type instead of the Task<int>?

I have a function from the one service to that will get the count of all files inside a directory. And another service will get that int number to do some stuff with it.
public int GetNumberOfAvatarsInFile()
{
try
{
var path = GetAvatarsFilePath();
var numberOfAvatars = Directory.GetFiles(path, "*.*", SearchOption.TopDirectoryOnly).Length;
return numberOfAvatars;
}
catch (Exception exception)
{
var message = $"Error While Getting Total Numbers Of Avatars at {DateTime.Now}\n\t" +
$"Error: {JsonConvert.SerializeObject(exception)}";
sentryClient.CaptureMessage(message, SentryLevel.Error);
return 1;
}
}
private string GetAvatarsFilePath()
{
var webRootPath = webHostEnvironment.WebRootPath;
var path = Path.Combine(webRootPath, "path");
return path;
}
The other service will use this function like this
private int GetMaximumAvatarId() => avatarService.GetNumberOfAvatarsInFile();
How do I set up so that all these file getting logic and string combine will be separated to a background thread/another thread by either Task.Run or something similar?
When I try to set up the GetNumberOfAvatarsInFile() by implementing await Task.Run(async () => LOGIC+Return int);
I have to return a Task rather than the int from the other service that is calling it as well, which is not desirable since those are other people code and I should not change them. Also as far as I know all the Path.Combine and Directory functions do not employ any awaiter.
Is there a way to implement this?

As mentioned in the comments, the best practice is to provide async methods to the caller and use async all the way (see this article). However there are 2 things that can already be done:
1. Make your I/O method run asynchronously in a separate thread.
2. Have callers call your method asynchronously even if the implementation is synchronous.
The implementations on client side and on service side are independent. Here is a commented example that I hope shows how to do this. Most of the code below is unnecessary and is there only to illustrate what happens when multiple callers call your method and what is executed when. You may change the Thread.Sleep() values to simulate different execution time.
I also added a side note regarding the value you return in the Exception, that does not look ok to me.
public class Program
{
public static void Main()
{
// These simulate 3 callers calling your service at different times.
var t1 = Task.Run(() => GetMaximumAvatarId(1));
Thread.Sleep(100);
var t2 = Task.Run(() => GetMaximumAvatarId(2));
Thread.Sleep(2000);
var t3 = Task.Run(() => GetMaximumAvatarId(3));
// Example purposes.
Task.WaitAll(t1, t2, t3);
Console.WriteLine("MAIN: Done.");
Console.ReadKey();
}
// This is a synchronous call on the client side. This could very well be implemented
// as an asynchronous call, even if the service method is synchronous, by using a
// Task and having the caller await for it (GetMaximumAvatarIdAsync).
public static int GetMaximumAvatarId(int callerId)
{
Console.WriteLine($"CALLER {callerId}: Calling...");
var i = GetNumberOfAvatarsInFile(callerId);
Console.WriteLine($"CALLER {callerId}: Done -> there are {i} files.");
return i;
}
// This method has the same signature as yours. It's synchronous in the sense that it
// does not return an awaitable. However it now uses `Task.Run` in order to execute
// `Directory.GetFiles` in a threadpool thread, which allows to run other code in
// parallel (in this example `Sleep` calls, in real life useful code). It finally
// blocks waiting for the result of the task, then returns it to the caller as an int.
// The "callerId" is for the example only, you may remove this everywhere.
public static int GetNumberOfAvatarsInFile(int callerId)
{
Console.WriteLine($" SERVICE: Called by {callerId}...");
var path = GetAvatarsFilePath();
var t = Task.Run(() => Directory.GetFiles(path, "*.*", SearchOption.TopDirectoryOnly).Length);
// Simulate long work for a caller, showing the caller.
Console.WriteLine($" SERVICE: Working for {callerId}...");
Thread.Sleep(500);
Console.WriteLine($" SERVICE: Working for {callerId}...");
Thread.Sleep(500);
Console.WriteLine($" SERVICE: Working for {callerId}...");
Thread.Sleep(500);
Console.WriteLine($" SERVICE: Blocking for {callerId} until task completes.");
return t.Result; // Returns an int.
// --------------------------------------------------------
// Side note: you should return `-1` in the `Exception`.
// Otherwise it is impossible for the caller to know if there was an error or
// if there is 1 avatar in the file.
// --------------------------------------------------------
}
// Unchanged.
private string GetAvatarsFilePath()
{
var webRootPath = webHostEnvironment.WebRootPath;
var path = Path.Combine(webRootPath, "path");
return path;
}
}

Run parallel task and return the first completed task and run other in background to save results

I have a WebApi in .NET CORE 3.1 in which I'm trying to get results from a service (other third party). I have created multiple requests in my API for the same service but some parameters of every request are different, the results return from service will be different for every request but structure of result will be same.
As all requests are independent of each other I want to run all that in parallel. And I want to return the first result as soon as received from the service from my API, but I also want to run all other requests in background and save there results in REDIS.
I tried to create a sample code to check if possible:
[HttpPost]
[Route("Test")]
public async Task<SearchResponse> Test(SearchRequest req)
{
List<Task<SearchResponse>> TaskList = new List<Task<SearchResponse>>();
for (int i = 0; i < 10; i++)
{
SearchRequest copyReq = Util.Copy(req); // my util function to copy the request
copyReq.ChangedParameter = i; // This is an example, many param can changed
TaskList.Add(Task.Run(() => DoSomething(copyReq)));
}
var finishedTask = await Task.WhenAny(TaskList);
return await finishedTask;
}
private async Task<SearchResponse> DoSomething(SearchRequest req)
{
// Here calling the third party service
SearchResponse resp = await service.GetResultAsync(req);
// Saving the result in REDIS
RedisManager.Save("KEY",resp);
return resp;
}
Now I'm wondering if this is correct way to dealing with this problem or not. If there is any better way please guide me to that.
EDIT
Use Case scenario
I have created a web app which will fetch results from my webapi and will display the results.
The WebApp searches for list of products (can be anything) by sending a request to my api. Now my api creates different requests as the source (Let's say Site1 and Site2) for results can be different.
Now the third party handles all requests to different sources(Site1 and Site2) and convert there results into my result structure. I have just to provide the parameter from which site i want to get results and then call the service at my end.
Now I want to send the results to my WebApp as soon as any source(site1 or site2) gives me the result, and in background I want to save the result of other source in redis. So that I can fetch that too from my webapp on other request hit.

The code looks pretty good; there's only one adjustment I'd recommend: don't use Task.Run. Task.Run causes a thread switch, which is totally unnecessary here.
[HttpPost]
[Route("Test")]
public async Task<SearchResponse> Test(SearchRequest req)
{
var TaskList = new List<Task<SearchResponse>>();
for (int i = 0; i < 10; i++)
{
SearchRequest copyReq = Util.Copy(req); // my util function to copy the request
copyReq.ChangedParameter = i; // This is an example, many param can changed
TaskList.Add(DoSomething(copyReq));
}
return await await Task.WhenAny(TaskList);
}
private async Task<SearchResponse> DoSomething(SearchRequest req)
{
// Here calling the third party service
SearchResponse resp = await service.GetResultAsync(req);
// Saving the result in REDIS
RedisManager.Save("KEY",resp);
return resp;
}
Note that this is using fire-and-forget. In the general sense, fire-and-forget is dangerous, since it means you don't care if the code fails or if it even completes. In this case, since the code is only updating a cache, fire-and-forget is acceptable.

C# Add to a List Asynchronously in API

I have an API which needs to be run in a loop for Mass processing.
Current single API is:
public async Task<ActionResult<CombinedAddressResponse>> GetCombinedAddress(AddressRequestDto request)
We are not allowed to touch/modify the original single API. However can be run in bulk, using foreach statement. What is the best way to run this asychronously without locks?
Current Solution below is just providing a list, would this be it?
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>();
foreach(AddressRequestDto request in requests)
{
var newCombinedAddress = (await GetCombinedAddress(request)).Value;
combinedAddressResponses.Add(newCombinedAddress);
}
return combinedAddressResponses;
}
Update:
In debugger, it has to go to combinedAddressResponse.Result.Value
combinedAddressResponse.Value = null
and Also strangely, writing combinedAddressResponse.Result.Value gives error below "Action Result does not contain a definition for for 'Value' and no accessible extension method

I'm writing this code off the top of my head without an IDE or sleep, so please comment if I'm missing something or there's a better way.
But effectively I think you want to run all your requests at once (not sequentially) doing something like this:
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>(requests.Count);
var tasks = new List<Task<ActionResult<CombinedAddressResponse>>(requests.Count);
foreach (var request in requests)
{
tasks.Add(Task.Run(async () => await GetCombinedAddress(request));
}
//This waits for all the tasks to complete
await tasks.WhenAll(tasks.ToArray());
combinedAddressResponses.AddRange(tasks.Select(x => x.Result.Value));
return combinedAddressResponses;
}

looking for a way to speed things up and run in parallel thanks
What you need is "asynchronous concurrency". I use the term "concurrency" to mean "doing more than one thing at a time", and "parallel" to mean "doing more than one thing at a time using threads". Since you're on ASP.NET, you don't want to use additional threads; you'd want to use a form of concurrency that works asynchronously (which uses fewer threads). So, Parallel and Task.Run should not be parts of your solution.
The way to do asynchronous concurrency is to build a collection of tasks, and then use await Task.WhenAll. E.g.:
public async Task<ActionResult<IReadOnlyList<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
// Build the collection of tasks by doing an asynchronous operation for each request.
var tasks = requests.Select(async request =>
{
var combinedAddressResponse = await GetCombinedAdress(request);
return combinedAddressResponse.Value;
}).ToList();
// Wait for all the tasks to complete and get the results.
var results = await Task.WhenAll(tasks);
return results;
}

How to properly execute a List of Tasks async in C#

I have a list of objects that I need to run a long running process on and I would like to kick them off asynchronously, then when they are all finished return them as a list to the calling method. I've been trying different methods that I have found, however it appears that the processes are still running synchronously in the order that they are in the list. So I am sure that I am missing something in the process of how to execute a list of tasks.
Here is my code:
public async Task<List<ShipmentOverview>> GetShipmentByStatus(ShipmentFilterModel filter)
{
if (string.IsNullOrEmpty(filter.Status))
{
throw new InvalidShipmentStatusException(filter.Status);
}
var lookups = GetLookups(false, Brownells.ConsolidatedShipping.Constants.ShipmentStatusType);
var lookup = lookups.SingleOrDefault(sd => sd.Name.ToLower() == filter.Status.ToLower());
if (lookup != null)
{
filter.StatusId = lookup.Id;
var shipments = Shipments.GetShipments(filter);
var tasks = shipments.Select(async model => await GetOverview(model)).ToList();
ShipmentOverview[] finishedTask = await Task.WhenAll(tasks);
return finishedTask.ToList();
}
else
{
throw new InvalidShipmentStatusException(filter.Status);
}
}
private async Task<ShipmentOverview> GetOverview(ShipmentModel model)
{
String version;
var user = AuthContext.GetUserSecurityModel(Identity.Token, out version) as UserSecurityModel;
var profile = AuthContext.GetProfileSecurityModel(user.Profiles.First());
var overview = new ShipmentOverview
{
Id = model.Id,
CanView = true,
CanClose = profile.HasFeatureAction("Shipments", "Close", "POST"),
CanClear = profile.HasFeatureAction("Shipments", "Clear", "POST"),
CanEdit = profile.HasFeatureAction("Shipments", "Get", "PUT"),
ShipmentNumber = model.ShipmentNumber.ToString(),
ShipmentName = model.Name,
};
var parcels = Shipments.GetParcelsInShipment(model.Id);
overview.NumberParcels = parcels.Count;
var orders = parcels.Select(s => WareHouseClient.GetOrderNumberFromParcelId(s.ParcelNumber)).ToList();
overview.NumberOrders = orders.Distinct().Count();
//check validations
var vals = Shipments.GetShipmentValidations(model.Id);
if (model.ValidationTypeId == Constants.OrderValidationType)
{
if (vals.Count > 0)
{
overview.NumberOrdersTotal = vals.Count();
overview.NumberParcelsTotal = vals.Sum(s => WareHouseClient.GetParcelsPerOrder(s.ValidateReference));
}
}
return overview;
}

It looks like you're using asynchronous methods while you really want threads.
Asynchronous methods yield control back to the calling method when an async method is called, then wait until the methods has completed on the await. You can see how it works here.
Basically, the only usefulness of async/await methods is not to lock the UI, so that it stays responsive.
If you want to fire multiple processings in parallel, you will want to use threads, like such:
using System.Threading.Tasks;
public void MainMethod() {
// Parallel.ForEach will automagically run the "right" number of threads in parallel
Parallel.ForEach(shipments, shipment => ProcessShipment(shipment));
// do something when all shipments have been processed
}
public void ProcessShipment(Shipment shipment) { ... }

Marking the method as async doesn't auto-magically make it execute in parallel. Since you're not using await at all, it will in fact execute completely synchronously as if it wasn't async. You might have read somewhere that async makes functions execute asynchronously, but this simply isn't true - forget it. The only thing it does is build a state machine to handle task continuations for you when you use await and actually build all the code to manage those tasks and their error handling.
If your code is mostly I/O bound, use the asynchronous APIs with await to make sure the methods actually execute in parallel. If they are CPU bound, a Task.Run (or Parallel.ForEach) will work best.
Also, there's no point in doing .Select(async model => await GetOverview(model). It's almost equivalent to .Select(model => GetOverview(model). In any case, since the method actually doesn't return an asynchronous task, it will be executed while doing the Select, long before you get to the Task.WhenAll.
Given this, even the GetShipmentByStatus's async is pretty much useless - you only use await to await the Task.WhenAll, but since all the tasks are already completed by that point, it will simply complete synchronously.

If your tasks are CPU bound and not I/O bound, then here is the pattern I believe you're looking for:
static void Main(string[] args) {
Task firstStepTask = Task.Run(() => firstStep());
Task secondStepTask = Task.Run(() => secondStep());
//...
Task finalStepTask = Task.Factory.ContinueWhenAll(
new Task[] { step1Task, step2Task }, //more if more than two steps...
(previousTasks) => finalStep());
finalStepTask.Wait();
}

What is the best way to do multithreading or asynchronous task in .NET with return value?

What would be the best way to do multithreading or asynchronous task in the following situation in C#?
The simplified situation:
A http request needs to make 5 or more
web service calls. Upon completion
each web service call will receive and
return a string list as a result. The
caller (of 5 web service calls) need
to merge the 5 results into a single
string list and return it to the http
caller.
Because each thread needs to return a value in the end so I am wondering if Asynchronous Delegates is the way to go. Because I am not so experienced in this area so I am asking this questions and/or suggestions.
Thank you!

You should have a look at QueueUserWorkItem. This would allow you to do each call on a separate thread and get the string value based on the particular call e.g.
ManualResetEvent[] calls = new ManualResetEvent[5];
string[] results = new string[5];
calls[0] = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(t =>
{
results[0] = // do webservice call
calls[0].Set();
});
calls[1] = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(t =>
{
results[1] = // do webservice call
calls[1].Set();
});
....
// wait for all calls to complete
WaitHandle.WaitAll(calls);
// merge the results into a comma delimited string
string resultStr = String.Join(", ", results);

Here's a short piece og code using .Net 4.0 which utilizes the new System.Collections.Concurrent which encapsulates the concurrency code:
class StackOverflowParalell
{
public void Execute()
{
List<int> codeParam = new List<int>(){1,2,3};
ConcurrentBag<string> result = new ConcurrentBag<string>();
Parallel.For(0, codeParam.Count, i => DoSometing(i).ForEach( result.Add ));
// return result here as List, Array....
}
List<string> DoSometing(int value)
{
return new List<string>(){"1","2","3","4"};
}
}

You need to use a callback to accomplish what you're asking.
The easiest way to do this if you're not using the 4.0 framework is as follows:
var strings = new List<string>();
for (var i = 0; i < 5; i++)
{
ThreadPool.QueueUserWorkItem((x) => strings.AddRange(SomeWebServiceCall(i)));
}
This requires some synchronization after the loop to ensure the threads complete before you move on. In the 4.0 framework, the Task Parallel Library does that work for you:
Parallel.For(0, 5, i => strings.AddRange(someWebServiceCall(i)));
Asynchronous delegates were useful when the 1.0 framework came out, but are a lot of extra framework to deal with since the addition of anonymous delegates in 2.0. In 4.0, the TPL makes them even more irrelevant. IAsyncResult relies on an explicit callback, and you can now use implicit callbacks (as demonstrated above).