We Keep Coding

C# Add to a List Asynchronously in API

I have an API which needs to be run in a loop for Mass processing.
Current single API is:
public async Task<ActionResult<CombinedAddressResponse>> GetCombinedAddress(AddressRequestDto request)
We are not allowed to touch/modify the original single API. However can be run in bulk, using foreach statement. What is the best way to run this asychronously without locks?
Current Solution below is just providing a list, would this be it?
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>();
foreach(AddressRequestDto request in requests)
{
var newCombinedAddress = (await GetCombinedAddress(request)).Value;
combinedAddressResponses.Add(newCombinedAddress);
}
return combinedAddressResponses;
}
Update:
In debugger, it has to go to combinedAddressResponse.Result.Value
combinedAddressResponse.Value = null
and Also strangely, writing combinedAddressResponse.Result.Value gives error below "Action Result does not contain a definition for for 'Value' and no accessible extension method

I'm writing this code off the top of my head without an IDE or sleep, so please comment if I'm missing something or there's a better way.
But effectively I think you want to run all your requests at once (not sequentially) doing something like this:
public async Task<ActionResult<List<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
var combinedAddressResponses = new List<CombinedAddressResponse>(requests.Count);
var tasks = new List<Task<ActionResult<CombinedAddressResponse>>(requests.Count);
foreach (var request in requests)
{
tasks.Add(Task.Run(async () => await GetCombinedAddress(request));
}
//This waits for all the tasks to complete
await tasks.WhenAll(tasks.ToArray());
combinedAddressResponses.AddRange(tasks.Select(x => x.Result.Value));
return combinedAddressResponses;
}

looking for a way to speed things up and run in parallel thanks
What you need is "asynchronous concurrency". I use the term "concurrency" to mean "doing more than one thing at a time", and "parallel" to mean "doing more than one thing at a time using threads". Since you're on ASP.NET, you don't want to use additional threads; you'd want to use a form of concurrency that works asynchronously (which uses fewer threads). So, Parallel and Task.Run should not be parts of your solution.
The way to do asynchronous concurrency is to build a collection of tasks, and then use await Task.WhenAll. E.g.:
public async Task<ActionResult<IReadOnlyList<CombinedAddressResponse>>> GetCombinedAddress(List<AddressRequestDto> requests)
{
// Build the collection of tasks by doing an asynchronous operation for each request.
var tasks = requests.Select(async request =>
{
var combinedAddressResponse = await GetCombinedAdress(request);
return combinedAddressResponse.Value;
}).ToList();
// Wait for all the tasks to complete and get the results.
var results = await Task.WhenAll(tasks);
return results;
}

Async expressions C#

I'm trying to generate a repository with some methods and including Async versions for those ones, like this:
//Method 1
public static List<MyEntity> GetMyEntityByDate(MyContextType context)
{
var tmpMov = context.MyEntity
.AsNoTracking()
.ToList();
return tmpMov;
}
//Method 2 V1
public async static List<MyEntity> GetMyEntityByDateAsync(MyContextType context)
{
var tmpMov = await context.MyEntity
.AsNoTracking()
.ToListAsync();
return tmpMov;
}
//Method 2 V2
public async static List<MyEntity> GetMyEntityByDateAsync(MyContextType context)
{
List<MyEntity> tmpMov = null;
await Task.Factory.StartNew(() => {
tmpMov = GetMyEntityByDate(context);
});
return tmpMov;
}
Now I have some questions:
What are the pros and cons of using Method 2 V1 vs Method2 V2, in terms of performance and fluency?
Before someone downvote this question, YES I'm implementing a repository pattern and want to write less code, thats why I'm considering V2, that way I have only one query to maintain.
But I have poor experience on async and obiously my target is get The best possible performance.

Using context.MyEntity.ToList() keeps the current thread busy until the Database finishes the query execution.
Using await context.MyEntity.ToListAsync(); frees the current thread until the Database finishes the query execution.
Using await Task.Factory.StartNew(() => { }); frees the current thread, but creates a new thread, and keeps it busy until the DB finishes the execution.

What are the pros and cons of using Method 2 V1 vs Method2 V2, in terms of performance and fluency?
Method 2 V1 is a proper asynchronous method. Method 2 V2 is a fake-asynchronous method that executes blocking work on TaskScheduler.Current (which would be the thread pool if there is no current task scheduler).
As such, V2 runs afoul of the best practice not to expose asynchronous wrappers for synchronous methods.
Before someone downvote this question, YES I'm implementing a repository pattern and want to write less code, thats why I'm considering V2, that way I have only one query to maintain.
I'd say that the ideal scenario is to actually only expose Method 2 V1, and get rid of Method 1 completely. Querying a database is inherently I/O-based, so the API is naturally asynchronous.
If you really need both synchronous and asynchronous APIs, then I recommend using the "bool argument hack" as described in my MSDN article on Brownfield Async Development.
This could look like:
private async static Task<List<MyEntity>> DoGetMyEntityByDateAsync(MyContextType context, bool sync)
{
var query = context.MyEntity
.AsNoTracking();
return sync ?
query.ToList() :
await query.ToListAsync();
}
public static Task<List<MyEntity>> GetMyEntityByDateAsync(MyContextType context)
{
return DoGetMyEntityByDateAsync(context, sync: false);
}
public static List<MyEntity> GetMyEntityByDate(MyContextType context)
{
return DoGetMyEntityByDateAsync(context, sync: true).GetAwaiter().GetResult();
}

Best way to call many web services?

I have 30 sub companies and every one has implemented their web service (with different technologies).
I need to implement a web service to aggregate them, for example, all the sub company web services have a web method with name GetUserPoint(int nationalCode) and I need to implement my web service that will call all of them and collect all of the responses (for example sum of points).
This is my base class:
public abstract class BaseClass
{ // all same attributes and methods
public long GetPoint(int nationalCode);
}
For each of sub companies web services, I implement a class that inherits this base class and define its own GetPoint method.
public class Company1
{
//implement own GetPoint method (call a web service).
}
to
public class CompanyN
{
//implement own GetPoint method (call a web service).
}
so, this is my web method:
[WebMethod]
public long MyCollector(string nationalCode)
{
BaseClass[] Clients = new BaseClass[] { new Company1(),//... ,new Company1()}
long Result = 0;
foreach (var item in Clients)
{
long ResultTemp = item.GetPoint(nationalCode);
Result += ResultTemp;
}
return Result;
}
OK, it works but it's so slow, because every sub companys web service is hosted on different servers (on the internet).
I can use parallel programing like this:(is this called parallel programing!?)
foreach (var item in Clients)
{
Tasks.Add(Task.Run(() =>
{
Result.AddRange(item.GetPoint(MasterLogId, mobileNumber));
}
}
I think parallel programing (and threading) isn't good for this solution, because my solution is IO bound (not CPU intensive)!
Call every external web service is so slow, am i right? Many thread that are pending to get response!
I think async programming is the best way but I am new to async programming and parallel programing.
What is the best way? (parallel.foreach - async TAP - async APM - async EAP -threading)
Please write for me an example.

It's refreshing to see someone who has done their homework.
First things first, as of .NET 4 (and this is still very much the case today) TAP is the preferred technology for async workflow in .NET. Tasks are easily composable, and for you to parallelise your web service calls is a breeze if they provide true Task<T>-returning APIs. For now you have "faked" it with Task.Run, and for the time being this may very well suffice for your purposes. Sure, your thread pool threads will spend a lot of time blocking, but if the server load isn't very high you could very well get away with it even if it's not the ideal thing to do.
You just need to fix a potential race condition in your code (more on that towards the end).
If you want to follow the best practices though, you go with true TAP. If your APIs provide Task-returning methods out of the box, that's easy. If not, it's not game over as APM and EAP can easily be converted to TAP. MSDN reference: https://msdn.microsoft.com/en-us/library/hh873178(v=vs.110).aspx
I'll also include some conversion examples here.
APM (taken from another SO question):
MessageQueue does not provide a ReceiveAsync method, but we can get it to play ball via Task.Factory.FromAsync:
public static Task<Message> ReceiveAsync(this MessageQueue messageQueue)
{
return Task.Factory.FromAsync(messageQueue.BeginReceive(), messageQueue.EndPeek);
}
...
Message message = await messageQueue.ReceiveAsync().ConfigureAwait(false);
If your web service proxies have BeginXXX/EndXXX methods, this is the way to go.
EAP
Assume you have an old web service proxy derived from SoapHttpClientProtocol, with only event-based async methods. You can convert them to TAP as follows:
public Task<long> GetPointAsyncTask(this PointWebService webService, int nationalCode)
{
TaskCompletionSource<long> tcs = new TaskCompletionSource<long>();
webService.GetPointAsyncCompleted += (s, e) =>
{
if (e.Cancelled)
{
tcs.SetCanceled();
}
else if (e.Error != null)
{
tcs.SetException(e.Error);
}
else
{
tcs.SetResult(e.Result);
}
};
webService.GetPointAsync(nationalCode);
return tcs.Task;
}
...
using (PointWebService service = new PointWebService())
{
long point = await service.GetPointAsyncTask(123).ConfigureAwait(false);
}
Avoiding races when aggregating results
With regards to aggregating parallel results, your TAP loop code is almost right, but you need to avoid mutating shared state inside your Task bodies as they will likely execute in parallel. Shared state being Result in your case - which is some kind of collection. If this collection is not thread-safe (i.e. if it's a simple List<long>), then you have a race condition and you may get exceptions and/or dropped results on Add (I'm assuming AddRange in your code was a typo, but if not - the above still applies).
A simple async-friendly rewrite that fixes your race would look like this:
List<Task<long>> tasks = new List<Task<long>>();
foreach (BaseClass item in Clients) {
tasks.Add(item.GetPointAsync(MasterLogId, mobileNumber));
}
long[] results = await Task.WhenAll(tasks).ConfigureAwait(false);
If you decide to be lazy and stick with the Task.Run solution for now, the corrected version will look like this:
List<Task<long>> tasks = new List<Task<long>>();
foreach (BaseClass item in Clients)
{
Task<long> dodgyThreadPoolTask = Task.Run(
() => item.GetPoint(MasterLogId, mobileNumber)
);
tasks.Add(dodgyThreadPoolTask);
}
long[] results = await Task.WhenAll(tasks).ConfigureAwait(false);

You can create an async version of the GetPoint:
public abstract class BaseClass
{ // all same attributes and methods
public abstract long GetPoint(int nationalCode);
public async Task<long> GetPointAsync(int nationalCode)
{
return await GetPoint(nationalCode);
}
}
Then, collect the tasks for each client call. After that, execute all tasks using Task.WhenAll. This will execute them all in parallell. Also, as pointed out by Kirill, you can await the results of each task:
var tasks = Clients.Select(x => x.GetPointAsync(nationalCode));
long[] results = await Task.WhenAll(tasks);
If you do not want to make the aggregating method async, you can collect the results by calling .Result instead of awaiting, like so:
long[] results = Task.WhenAll(tasks).Result;

Asynchronous begin/end in same method

(I'm using .Net 4.0)
I want to call a WCF service asynchronously, from my service layer. This service layer is used by and MVC.Net controller. I've read that it's good practice to call a WCF service asynchronously. So I'm using begin/end (apm). I want to double check if I'm doing it richt:
public byte[] GetSomeData()
{
IAsyncResult result = myServiceClient.BeginDoSomething(someInputValue, null, null);
var data = _pdfCreatieService.EndCreateForPreview(result);
return data;
}
I'm not totally sure about the code above, because I've seen constructions like the code below, which seem a bit more complex and unnecessary in my case:
public byte[] GetSomeData()
{
var myState = new MyState();
IAsyncResult result = _myServiceClient.BeginDoSomething(someInputValue, CreateForPreviewCallback, myState);
result.AsyncWaitHandle.WaitOne();
return myState.Bytes;
}
private void DoSomethingCallback(IAsyncResult result)
{
var myState = (MyState)result.AsyncState;
myState.Bytes = _myServiceClient.EndDoSomething(result);
}
Thanks Avner Shahar-Kashtan, Ned Stoyanov and Noseratio. Your answers are really insightfull!

What your code will do is, in effect, take an asynchronous method and call it synchronously. When you call the EndDoSomething method, you are effectively blocking your thread until the asynchronous method has completed, which is exactly the opposite of an async call.
Of course, your second code block also calls the code synchronously, by blocking executing explicitly, using the waithandle.
What you want to do is, instead of returning the byte[] from your initial method, have your DoSomethingCallback do something active with the bytes - either store them in some class member that can be checked by the controller, raise an event, or do something else. If you're waiting for your async call, you're not getting any benefit.
What you can also do if you're using .NET 4.5 or higher (or .NET 4.0 in VS2012, using the BCL Async Package) is to use async/await, which is a nice wrapper which will allow you to consolidate the calling method and the callback method into a single, more coherent method.
But regardless of the syntax or libraries you choose, your first step is to understand that async programming necessarily breaks your code's control flow into the invocation and the result callback, or continuation of the asynchronous operation.

In ASP.NET MVC, you only benefit from asynchronous calls if your controller is asynchronous too. Apparently, this is not the case with your code, because you're blocking with WaitOne inside your controller's method.
Implementing asynchronous controllers is really easy with .NET 4.5, check "Using Asynchronous Methods in ASP.NET MVC 4" for more info.
With .NET 4.0, it's a bit more tedious, check "Using an Asynchronous Controller in ASP.NET MVC". Your controller should derive from AsyncController and use AsyncManager to notify the ASP.NET stack about the pending asynchronous operations.
Here's an example from there for .NET 4. 0, adapted for your case(untested). Note the use of Task.Factory.FromAsync and Task.ContinueWith:
public static class WcfExt
{
public static Task<byte[]> DoSomethingAsync(
this IMyService service,
string someInputValue)
{
return Task.Factory.FromAsync(
(asyncCallback, asyncState) =>
service.BeginDoSomething(someInputValue, asyncCallback, asyncState),
(asyncResult) =>
service.EndDoSomething(asyncResult);
}
}
public class PortalController : AsyncController
{
public void NewsAsync(string someInputValue) {
AsyncManager.OutstandingOperations.Increment();
var myService = new MyService();
myService.DoSomethingAsync(someInputValue).ContinueWith((task) =>
{
AsyncManager.Parameters["data"] = task.Result;
AsyncManager.OutstandingOperations.Decrement();
}, TaskScheduler.FromCurrentSynchronizationContext());
}
public ActionResult NewsCompleted(byte[] data)
{
return View("News", new ViewStringModel
{
NewsData = data
});
}
}

Both of your approaches are doing what is called sync over async. That is executing an asynchronous method in synchronous fashion. A better approach would be to build your own asynchronous method to rerieve the data with TaskCompletionSource. I haven't tested this, but you should be able to do something like this:
public Task<byte[]> GetSomeDataAsync()
{
var tcs = new TaskCompletionSource();
IAsyncResult result = myServiceClient.BeginDoSomething(someInputValue, x =>
{
try
{
var data = _pdfCreatieService.EndCreateForPreview(result);
tcs.SetResult(data);
}
catch(Exception ex)
{
tcs.SetException(ex);
}
}, null);
return tcs.Task;
}
Then to use it simply do this
GetSomeDataAsync().ContinueWith(t => /*<Use retrieved data from t.Result>*/);
This code will return straight away and once the asynchronous operation is complete the ContinueWith part will execute.

How to return type of Task<mytype> without running async

In my data access layer I'm attempting to create methods with return types of Task.
I cannot get the return type from the entity framework to return Task<List<MYtype>>
public static Task<List<ILeaf>> GetLeafs(int doorID)
{
using (var db = new StorefrontSystemEntities())
{
return db.proc_GetDoorLeafs(doorID).ToList<ILeaf>();
};
}
Is the only way to make this run correctly is to format the code like so:
public async static Task<List<ILeaf>> GetLeafs(int doorID)
{
return await Task.Run(() =>
{
using (var db = new StorefrontSystemEntities())
{
return db.proc_GetDoorLeafs(doorID).ToList<ILeaf>();
};
});
}
The reason I was asking is because I would like to give the option to run async, or am I not understanding this correctly?
If I can just return a Task then on the calling end I could give the option to await if I wanted to run async, but if i wanted to run synchronously I would just call the method as normal.
If I'm returning a Task do I always have to include the async keyword in the method signature?
Am I thinking about this the wrong way? If I've got a return type of Task then the method has the option of being called async or synchronously?
But, if I have async and Task in the method signature then the method runs async no matter what?
Thanks!

So to answer the literal question asked, you can just do this:
public static Task<List<ILeaf>> GetLeafs(int doorID)
{
return Task.Run(() =>
{
using (var db = new StorefrontSystemEntities())
{
return db.proc_GetDoorLeafs(doorID).ToList<ILeaf>();
};
});
}
That said, note that this isn't a particularly useful method. The idea of leveraging asynchronous programming is that you don't want to have a thread pool thread sitting there doing nothing but waiting on this IO operation. Ideally you'd leverage IO that is inherently asynchronous in nature; a method that itself naturally returns a task.
You aren't really providing value to consumers of your code by wrapping the blocking IO in a call to Task.Run. If they need that operation to be run in a background thread they can do so on their own, and then they'll know more precisely that it's not a naively asynchronous operation.
See Should I expose asynchronous wrappers for synchronous methods? for more information on the subject.

public static async Task<List<ILeaf>> GetLeafs(int doorID)
{
using (var db = new StorefrontSystemEntities())
{
var result = db.proc_GetDoorLeafs(doorID).ToList<ILeaf>();
return await Task.FromResult(result);
}
}