We Keep Coding

Async LINQ - not lazy? Multithreaded?

I have the following code:
var things = await GetDataFromApi(cancellationToken);
var builder = new StringBuilder(JsonSerializer.Serialize(things));
await things
.GroupBy(x => x.Category)
.ToAsyncEnumerable()
.SelectManyAwaitWithCancellation(async (category, ct) =>
{
var thingsWithColors = await _colorsApiClient.GetColorsFor(category.Select(thing => thing.Name).ToList(), ct);
return category
.Select(thing => ChooseBestColor(thingsWithColors))
.ToAsyncEnumerable();
})
.ForEachAsync(thingAndColor =>
{
Console.WriteLine(Thread.CurrentThread.ManagedThreadId); // prints different IDs
builder.Replace(thingAndColor.Thing, $"{thingAndColor.Color} {thingAndColor.Thing}");
}, cancellationToken);
It uses System.Linq.Async and I find it difficult to understand.
In "classic"/synchronous LINQ, the whole thing would get executed only when I call ToList() or ToArray() on it. In the example above, there is no such call, but the lambdas get executed anyway. How does it work?
The other concern I have is about multi-threading. I heard many times that async != multithreading. Then, how is that possible that the Console.WriteLine(Thread.CurrentThread.ManagedThreadId); prints various IDs? Some of the IDs get printed multiple times, but overall there are about 5 thread IDs in the output. None of my code creates any threads explicitly. It's all async-await.
The StringBuilder does not support multi-threading, and I'd like to understand if the implementation above is valid.
Please ignore the algorithm of my code, it does not really matter, it's just an example. What matters is the usage of System.Async.Linq.

ForEachAsync would have a similar effect as ToList/ToArray since it forces evaluation of the entire list.
By default, anything after an await continues on the same execution context, meaning if the code runs on the UI thread, it will continue running on the UI thread. If it runs on a background thread, it will continue to run on a background thread, but not necessarily the same one.
However, none of your code should run in parallel. That does not necessarily mean it is thread safe, there probably need to be some memory barriers to ensure data is flushed correctly, but I would assume these barriers are issued by the framework code itself.

The System.Async.Linq, as well as the whole dotnet/reactive repository, is currently a semi-abandoned project. The issues on GitHub are piling up, and nobody answers them officially for almost a year. There is no documentation published, apart from the XML documentation in the source code on top of each method. You can't really use this library without studying the source code, which is generally easy to do because the code is short, readable, and honestly doesn't do too much. The functionality offered by this library is similar with the functionality found in the System.Linq, with the main difference being that the input is IAsyncEnumerable<T> instead of IEnumerable<T>, and the delegates can return values wrapped in ValueTask<T>s.
With the exception of a few operators like the Merge (and only one of its overloads), the System.Async.Linq doesn't introduce concurrency. The asynchronous operations are invoked one at a time, and then they are awaited before invoking the next operation. The SelectManyAwaitWithCancellation operator is not one of the exceptions. The selector is invoked sequentially for each element, and the resulting IAsyncEnumerable<TResult> is enumerated sequentially, and its values yielded the one after the other. So it's unlikely to create thread-safety issues.
The ForEachAsync operator is just a substitute of doing a standard await foreach loop, and was included in the library at a time when the C# language support for await foreach was non existent (before C# 8). I would recommend against using this operator, because its resemblance with the new Parallel.ForEachAsync API could create confusion. Here is what is written inside the source code of the ForEachAsync operator:
// REVIEW: Once we have C# 8.0 language support, we may want to do away with these
// methods. An open question is how to provide support for cancellation,
// which could be offered through WithCancellation on the source. If we still
// want to keep these methods, they may be a candidate for
// System.Interactive.Async if we consider them to be non-standard
// (i.e. IEnumerable<T> doesn't have a ForEach extension method either).

Async search request

I didn't coded for a year now and I need some help with LinqPad. I need to run multiple search query to get data from my database.
Let's say I need to run
void DeepSearch(string input)
{
Orders.Where(y => y.OrderReference.Contains(input)).Dump();
Invoice.Where(y => y.InvoiceReference.Contains(input)).Dump();
Clients.Where(y => y.Name.Contains(input)).Dump();
}
To speed up performance I would like to launch these 3 queries together and dump as soon I get the result. I don't care about the order.
Can I symply add async ?
async void DeepSearch(string input)
{
Orders.Where(y => y.OrderReference.Contains(input)).Dump();
Invoice.Where(y => y.InvoiceReference.Contains(input)).Dump();
Clients.Where(y => y.Name.Contains(input)).Dump();
}

No you cannot; "async" is not the same as "parallel", and even if it were: data-sources need to be designed for (separately) both async and/or parallel. I'm guessing that Orders, Invoices and Clients here are LINQ providers over a shared database connection (probably using EF), in which case: database connections in ADO.NET aren't designed for concurrency/parallelism. Additionally, LINQ doesn't directly expose async operations, but: EF does make some concessions to async execution - if you use methods like Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.ToListAsync(), which just means: "add using Microsoft.EntityFrameworkCore; to the top of the file, and use await source.Where(...).ToListAsync()".
However:
going async without going parallel doesn't improve your direct observed performance; it just means that you're not tying up the thread (a limited resource) while waiting on the DB, allowing the thread to be released to go and do more interesting things than waiting
going concurrent is complicated, and requires either separate isolated connections (etc), or connections that are designed for concurrency

You can certainly run queries in parallel if you instantiate a separate data context per parallel operation. (This is required because data contexts are not threadsafe, whether LINQ-to-SQL or EF Core).
LINQPad's automatically generated data context is called TypedDataContext, so your code in LINQPad will look like this:
Task.Run(() => new TypedDataContext().Orders.Where(y => y.OrderReference.Contains(input)Dump());
Task.Run(() => new TypedDataContext().Invoices.Where(y => y.InvoiceReference.Contains(input)).Dump());
Task.Run(() => new TypedDataContext().Clients.Where(y => y.Name.Contains(input).Dump());
If you assign the tasks to variables, you can await them (or use Task.WhenAll to await them all).
Depending on whether you're using LINQ-to-SQL or EF Core, there's also a cost associated with instantiating data contexts, so you might want to re-use the same TypedDataContext for subsequent operations on the same thread.

Async vs Parallel.Invoke vs Task.WhenAll for EntityF6 query that gets data, in ASP MVC 5 web app

I'm trying to figure what is the best approach, apart of synchronous programming, for doing some EF6 queries that retrieve data. I'll post here all 5 methods(these take place in a Controller Action ):
//would it be better to not "async" the ActionResult?
public async Task<ActionResult> Index{
// I depend on this so I don't even know if it's ok to make it async or not -> what do you think?
var userinfo = _dataservice.getUserInfo("John");
// C1: synchronous way
var watch1 = System.Diagnostics.Stopwatch.StartNew();
var info1 = _getInfoService.GetSomeInfo1(userinfo);
var info2 = _getInfoService.GetSomeInfo2(userinfo);
watch1.Stop();
var t1 = watch.EllapsedMilliSeconds; // this takes about 3200
// C2: asynchronous way
var watch2 = System.Diagnostics.Stopwatch.StartNew();
var infoA1 = await _getInfoService.GetSomeInfoAsync1(userinfo).ConfigureAwait(false);
var infoA2 = await _getInfoService.GetSomeInfoAsync2(userinfo).ConfigureAwait(false);
watch2.Stop();
var t2 = watch2.EllapsedMilliSeconds; // this takes about 3020
// C2.1: asynchronous way launch then await
var watch21 = System.Diagnostics.Stopwatch.StartNew();
var infoA21 = _getInfoService.GetSomeInfoAsync1(userinfo).ConfigureAwait(false);
var infoA22 = _getInfoService.GetSomeInfoAsync2(userinfo).ConfigureAwait(false);
// I tought if I launch them first then await, it would run faster...but not
var a = await infoA21;
var b = await infoA22;
watch21.Stop();
var t21 = watch21.EllapsedMilliSeconds; // this takes about the same 30201
// C3: asynchronous with Task.Run() and await.WhenAll()
var watch1 = System.Diagnostics.Stopwatch.StartNew();
var infoT1 = TaskRun(() => _getInfoService.GetSomeInfo1(userinfo));
var infoT2 = TaskRun(() => _getInfoService.GetSomeInfo2(userinfo));
await Task.WhenAll(infoT1,infoT2)
watch3.Stop();
var t3 = watch3.EllapsedMilliSeconds; // this takes about 2010
// C4: Parallel way
MyType var1; MyType2 var2;
var watch4 = System.Diagnostics.Stopwatch.StartNew();
Parallel.Invoke(
() => var1 = _getInfoService.GetSomeInfoAsync1(userinfo).GetAwaiter().GetResult(),// also using just _getInfoService.GetSomeInfo1(userinfo) - but sometimes throws an Entity error on F10 debugging
() => var2 = _getInfoService.GetSomeInfoAsync2(userinfo).GetAwaiter().GetResult()// also using just _getInfoService.GetSomeInfo2(userinfo)- but sometimes throws an Entity error on F10 debugging
);
watch4.Stop();
var t4 = watch4.EllapsedMilliSeconds; // this takes about 2012
}
Methods implementation:
public MyType1 GetSomeInfo1(SomeOtherType param){
// result = some LINQ queries here
Thread.Sleep(1000);
return result;
}
public MyType2 GetSomeInfo2(SomeOtherType param){
// result = some LINQ queries here
Thread.Sleep(2000);
return result;
}
public Task<MyType1> GetSomeInfoAsync1(SomeOtherType param){
// result = some LINQ queries here
Thread.Sleep(1000);
return Task.FromResult(result);
}
public Task<MyType2> GetSomeInfoAsync2(SomeOtherType param){
// result = some LINQ queries here
Thread.Sleep(2000);
return Task.FromResult(result);
}
If I understood correctly, await for 2 tasks(like in C2 and C2.1) does not make them run in parallel(not even in C.1 example where I launch them first then await), it just frees the current thread and gives them to another 2 different threads that will deal with those tasks
Task.Run() will in fact do just as Invoke.Parallel does, spreading the work on 2 different CPU's for making them run in parallel
Launching them first and then awaiting (C.1 example) shouldn't make them run a some sort of parallel way?
Would it be better not using async or parallel at all?
Please make me understand on these examples how can I have async and also better performance, also if there are any implications with EntityF that I must consider. I'm reading for a few days already and I only get confused, so please don't give me another links to read :)

async code can be mixed with parallelism by calling without await, then awaiting a Task.WaitAll(). However, the main consideration when looking at parallelism is ensuring the code called is thread -safe. DbContexts are not thread-safe, so to run parallel operations you need separate DbContext instances for each method. This means that code that normally relies on dependency injection to receive a DbContext/Unit of Work and would get a reference that is lifetime scoped to something like the web request cannot be used in parallelized calls. Calls that are parallelized will need to have a DbContext that is scoped for just that call.
When dealing with parallelized methods working with EF Entities that also means that you need to ensure that any entity references are treated as detached entities. They cannot safely be associated with one another as if they had been returned by different DbContexts in different parallel tasks.
For example, using normal async & await:
var order = await Repository.GetOrderById(orderId);
var orderLine = await Repository.CreateOrderLineForProduct(productId, quantity);
order.OrderLines.Add(orderLine);
await Repository.SaveChanges();
As a very basic example where the repository class gets a DbContext injected. The CreateOrderLine method would be using the DbContext to load the Product and possibly other details to make an OrderLine. When awaited, the async variants ensure only one thread is accessing the DbContext at a time so the same single DbContext instance can be used by the Repository. The Order, new OrderLine, Product, etc. are all tracked by the same DbContext instance so a SaveChanges call issued by the repository against that single instance will work as expected.
If we tried to parallelize it like:
var orderTask = Repository.GetOrderById(orderId);
var orderLineTask = Repository.CreateOrderLineForProduct(productId, quantity);
await Task.WhenAll(orderTask, orderLineTask);
var order = orderTask.Result;
var orderLine = orderLineTask.Result;
order.OrderLines.Add(orderLine);
await Repository.SaveChanges();
This would likely result in exceptions from EF that the DbContext is being accessed across threads as both the GetOrderById, and calls within CreateOrderLine. What's worse is that EF won't detect that it is being called by multiple threads until those threads both try to access the DbSets etc. at the same time. So this can sometimes result in an intermittent error that might not appear during testing or appear reliably when not under load (queries all finish quite quickly and don't trip on each other) but grind to a halt with exceptions when running under load. To address this, the DbContext reference in the Repository needs to be scoped for each method. This means rather than using an injected DbContext, it needs to look more like:
public Order GetOrderById(int orderId)
{
using(var context = new AppDbContext())
{
return context.Orders
.Include(x=>x.OrderLines)
.AsNoTracking()
.Single(x => x.OrderId == orderId);
}
}
We could still use dependency injection to inject something like a DbContext Factory class to create the DbContext which can be mocked out. The key thing is that the scope of the DbContext must be moved to within the parallelized method. AsNoTracking() is important because we cannot leave this order "tracked" by this DbContext; When we want to save the order and any other associated entities, we will have to associate this order with a new DbContext instance. (this one is being disposed) If the Entity still thinks it's tracked, that will result in an error. This also means that the repository Save has to change to something more like:
Repository.Save(order);
to pass in an entity, associate it and all referenced entities with a DbContext, and then calling SaveChanges.
Needless to say this starts getting messy, and it hasn't even touched on things like exception handling. You also lose aspects like change tracking because of the need to work with detached entities. To avoid potential issues between tracked and untracked entities and such I would recommend that parallelized code should always deal with POCO view models or more complete "operations" with entities rather than doing things like returning detached entities. We want to avoid confusion between code that might be called via an Order that is tracked (using synchronous or async calls) vs. an Order that is not tracked because it is the result of a parallelized call. That said, it can have its uses, but I would highly recommend keeping it's use to a minimum.
async/await can be an excellent pattern to adopt for longer, individual operations where that web request can expect to wait a few seconds such as a search or report. This frees up the web request handling thread to start responding to other requests while the user waits. Hence it's use to boost server responsiveness, not to be confused with making calls faster. For short and snappy operations it ends up adding a bit of extra overhead, so these should just be left as synchronous calls. async is not something I would ever argue needs to be an "all or nothing" decision in an application.
So that above example, loading an Order by ID and creating an Orderline would be something that I would normally leave synchronous, not asynchronous. Loading an entity graph by ID is typically quite fast. A better example where I would leverage async would be something like:
var query = Repository.GetOrders()
.Where(x => x.OrderStatus.OrerStatusId == OrderStatus.New
&& x.DispatchDate <= DateTime.Today());
if (searchCriteria.Any())
query = query.Where(buildCriteria(searchCriteria));
var pendingOrders = await query.Skip(pageNumber * pageSize)
.Take(PageSize)
.ProjectTo<OrderSearchResultViewModel>()
.ToListAsync();
Where in this example I have a search operation which is expected to run across a potentially large number of orders and possibly include less efficient user defined search criteria before fetching a page of results. It might take less than a second, or several seconds to run, and there could be a number of calls, including other searches, to be processing from other users at the time.
Parallelization is more geared towards situations where there are a mix of long and short-running operations that need to be completed as a unit so one doesn't need to wait for the other to complete before it starts. Much more care needs to be taken in this model when it comes to operations with EF Entities, so it's definitely not a pattern I would design as the "default" for in a system.
So to summarize:
Synchronous - Quick hits to the database or in-memory cache such as pulling rows by ID or in general queries expected to execute in 250ms or less. (Basically, the default)
Asynchronous - Bigger queries across larger sets with potentially slower execution time such as dynamic searches, or shorter operations that are expected to be called extremely frequently.
Parallel - Expensive operations that will launching several queries to complete where the queries can be "stripped" for the necessary data and run completely independently and in the background. I.e. reports or building exports, etc.

Task.Run using custom thread pool

I have to address a temporary situation that requires me to do a non-ideal thing: I have to call an async method from inside a sync one.
Let me just say here that I know all about the problems I'm getting myself into and I understand reasons why this is not advised.
That said, I'm dealing with a large codebase, which is completely sync from top to bottom and there is no way I can rewrite everything to use async await in a reasonable amount of time. But I do need to rewrite a number of small parts of this codebase to use the new async API that I'be been slowly developing over the last year or so, because it has a lot of new features that the old codebase would benefit from as well, but can't get them for legacy reasons. And since all that code isn't going away any time soon, I'm facing a problem.
TL;DR: A large sync codebase cannot be easily rewritten to support async but now requires calls into another large codebase, which is completely async.
I'm currently doing the simplest thing that works in the sync codebase: wrapping each async call into a Task.Run and waiting for the Result in a sync way.
The problem with this approach is, that it becomes slow whenever sync codebase does this in a tight loop. I understand the reasons and I sort of know what I can do - I'd have to make sure that all async calls are started on the same thread instead of borrowing a new one each time from the thread pool (which is what Task.Run does). This borrowing and returning incurs a lot of switching which can slow things down considerably if done a lot.
What are my options, short of writing my own scheduler that would prefer to reuse a single dedicated thread?
UPDATE: To better illustrate what I'm dealing with, I offer an example of one of the simplest transformations I need to do (there are more complex ones as well).
It's basically simple LINQ query that uses a custom LINQ provider under the hood. There's no EF or anything similar underneath.
[Old code]
var result = (from c in syncCtx.Query("Components")
where c.Property("Id") == id
select c).SingleOrDefault();
[New code]
var result = Task.Run(async () =>
{
Dictionary<string, object> data;
using (AuthorizationManager.Instance.AuthorizeAsInternal())
{
var uow = UnitOfWork.Current;
var source = await uow.Query("Components")
.Where("Id = #id", new { id })
.PrepareAsync();
var single = await source.SingleOrDefaultAsync();
data = single.ToDictionary();
}
return data;
}).Result;
As mentioned, this is one of the less complicated examples and it already contains 2 async calls.
UPDATE 2: I tried removing the Task.Run and invoking .Result directly on the result of a wrapper async method, as suggested by #Evk and #Manu. Unfortunately, while testing this in my staging environment, I quickly ran into a deadlock. I'm still trying to understand what exactly transpired, but it's obvious that Task.Run cannot simply be removed in my case. There are additional complications to be resolved, first...

I don't think you are on the right track. Wrapping every async call in a Task.Run seems horrible to me, it always starts an additional tasks which you don't need. But I understand that introducing async/await in a large codebase can be problematic.
I see a possible solution: Extract all async calls into separate, async methods. This way, your project will have a pretty nice transition from sync to async, since you can change methods one by one without affecting other parts of the code.
Something like this:
private Dictionary<string, object> GetSomeData(string id)
{
var syncCtx = GetContext();
var result = (from c in syncCtx.Query("Components")
where c.Property("Id") == id
select c).SingleOrDefault();
DoSomethingSyncWithResult(result);
return result;
}
would become something like this:
private Dictionary<string, object> GetSomeData(string id)
{
var result = FetchComponentAsync(id).Result;
DoSomethingSyncWithResult(result);
return result;
}
private async Task<Dictionary<string, object>> FetchComponentAsync(int id)
{
using (AuthorizationManager.Instance.AuthorizeAsInternal())
{
var uow = UnitOfWork.Current;
var source = await uow.Query("Components")
.Where("Id = #id", new { id })
.PrepareAsync();
var single = await source.SingleOrDefaultAsync();
return single.ToDictionary();
}
}
Since you are in a Asp.Net environment, mixing sync with async is a very bad idea. I'm surprised that your Task.Run solution works for you. The more you incorporate the new async codebase into the old sync codebase, the more you will run into problems and there is no easy fix for that, except rewriting everything in an async way.
I strongly suggest you to not mix your async parts into the sync codebase. Instead, work from "bottom to top", change everything from sync to async where you need to await an async call. It may seem like a lot of work, but the benefits are much higher than if you search for some "hacks" now and don't fix the underlining problems.

Easiest way to make controller asynchronous

I inherited a large web application that uses MVC5 and C#. Some of our controllers make several slow database calls and I want to make them asynchronous in an effort to allow the worker threads to service other requests while waiting for the database calls to complete. I want to do this with the least amount of refactoring. Say I have the following controller
public string JsonData()
{
var a = this.servicelayer.getA();
var b = this.servicelayer.getB();
return SerializeObject(new {a, b});
}
I have made the two expensive calls a, b asynchronous by leaving the service layer unchanged and rewriting the controller as
public async Task<string> JsonData()
{
var task1 = Task<something>.Run(() => this.servicelayer.getA());
var task2 = Task<somethingelse>.Run(() => this.servicelayer.getB());
await Task.WhenAll(task1, task2);
var a = await task1;
var b = await task2;
return SerializeObject(new {a, b});
}
The above code runs without any issues but I can't tell using Visual Studio if the worker threads are now available to service other requests or if using Task.Run() in a asp.net controller doesn't do what I think it does. Can anyone comment on the correctness of my code and if it can be improved in any way? Also, I read that using async in a controller has additional overhead and should be used only for long running code. What is the minimum criteria that I can use to decide if the controller needs async? I understand that every use case is different but wondering if there is a baseline that I can use as a starting point. 2 database calls? anything over 2 seconds to return?

The guideline is that you should use async whenever you have I/O. I.e., a database. The overhead is miniscule compared to any kind of I/O.
That said, blocking a thread pool thread via Task.Run is what I call "fake asynchrony". It's exactly what you don't want to do on ASP.NET.
Instead, start at your "lowest-level" code and make that truly asynchronous. E.g., EF6 supports asynchronous database queries. Then let the async code grow naturally from there towards your controller.

The only improvement the new code has is it runs both A and B concurrently and not one at a time. There's actually no real asynchrony in this code.
When you use Task.Run you are offloading work to be done on another thread, so basically you start 2 threads and release the current thread while awaiting both tasks (each of them running completely synchronously)
That means that the operation will finish faster (because of the parallelism) but will be using twice the threads and so will be less scalable.
What you do want to do is make sure all your operations are truly asynchronous. That will mean having a servicelayer.getAAsync() and servicelayer.getBAsync() so you could truly release the threads while IO is being processed:
public async Task<string> JsonData()
{
return SerializeObject(new {await servicelayer.getAAsync(), await servicelayer.getBAsync()});
}
If you can't make sure your actual IO operations are truly async, it would be better to keep the old code.
More on why to avoid Task.Run: Task.Run Etiquette Examples: Don't Use Task.Run in the Implementation