We Keep Coding

Parallelize without Task.Run()

Is it possible to let something run in background (in example a DB query) without spawining new thread with Task.Run()?
Example Db query
public async Task Query(long objectToDelete)
{
using ( var ctx= new Context())
{
Car car = new Car { Id = objectToDelete};
ctx.Entry(employer).State = EntityState.Deleted;
await ctx.SaveChangesAsync();
}
}
From what I understand the Query is runned synchronously up until the first "await", then the control is returned to the caller,
I wonder if I can leave a caller like that:
public async Task Caller( long id)
{
var runningQuery = Query( id);
/* do something else
*/
// await runningQuery; // commented out so "Caller" can complete early
}

You can do exactly this. The query is running as soon as SaveChangesAsync returns. await pauses execution until the task is done. So don't use await (you did this correctly).
Note, that fire and forget work is very difficult. You need to make sure to log errors. Also, you often cannot rely on the work to ever complete. There might be an error (like a network blip or a deadlock or a timeout) or your whole process might exit before the query is complete. This is especially relevant in ASP.NET. Consider fire and forget work optional if you do not wait for it at any point.

The await keyword will halt execution within the method that it is called and return execution to the method that called that method. So if you wanted to return from "Caller" early, you would just leave the
await runningQuery;
in place.
This Microsoft Page explains it. If you look at the diagram under the heading What Happens in an Async Method , you get a good visual reference of the logic flow.

Calling async methods from non-async code

I'm in the process of updating a library that has an API surface that was built in .NET 3.5. As a result, all methods are synchronous. I can't change the API (i.e., convert return values to Task) because that would require that all callers change. So I'm left with how to best call async methods in a synchronous way. This is in the context of ASP.NET 4, ASP.NET Core, and .NET/.NET Core console applications.
I may not have been clear enough - the situation is that I have existing code that is not async aware, and I want to use new libraries such as System.Net.Http and the AWS SDK that support only async methods. So I need to bridge the gap, and be able to have code that can be called synchronously but then can call async methods elsewhere.
I've done a lot of reading, and there are a number of times this has been asked and answered.
Calling async method from non async method
Synchronously waiting for an async operation, and why does Wait() freeze the program here
Calling an async method from a synchronous method
How would I run an async Task<T> method synchronously?
Calling async method synchronously
How to call asynchronous method from synchronous method in C#?
The problem is that most of the answers are different! The most common approach I've seen is use .Result, but this can deadlock. I've tried all the following, and they work, but I'm not sure which is the best approach to avoid deadlocks, have good performance, and plays nicely with the runtime (in terms of honoring task schedulers, task creation options, etc). Is there a definitive answer? What is the best approach?
private static T taskSyncRunner<T>(Func<Task<T>> task)
{
T result;
// approach 1
result = Task.Run(async () => await task()).ConfigureAwait(false).GetAwaiter().GetResult();
// approach 2
result = Task.Run(task).ConfigureAwait(false).GetAwaiter().GetResult();
// approach 3
result = task().ConfigureAwait(false).GetAwaiter().GetResult();
// approach 4
result = Task.Run(task).Result;
// approach 5
result = Task.Run(task).GetAwaiter().GetResult();
// approach 6
var t = task();
t.RunSynchronously();
result = t.Result;
// approach 7
var t1 = task();
Task.WaitAll(t1);
result = t1.Result;
// approach 8?
return result;
}

So I'm left with how to best call async methods in a synchronous way.
First, this is an OK thing to do. I'm stating this because it is common on Stack Overflow to point this out as a deed of the devil as a blanket statement without regard for the concrete case.
It is not required to be async all the way for correctness. Blocking on something async to make it sync has a performance cost that might matter or might be totally irrelevant. It depends on the concrete case.
Deadlocks come from two threads trying to enter the same single-threaded synchronization context at the same time. Any technique that avoids this reliably avoids deadlocks caused by blocking.
In your code snippet, all calls to .ConfigureAwait(false) are pointless because the return value is not awaited. ConfigureAwait returns a struct that, when awaited, exhibits the behavior that you requested. If that struct is simply dropped, it does nothing.
RunSynchronously is invalid to use because not all tasks can be processed that way. This method is meant for CPU-based tasks, and it can fail to work under certain circumstances.
.GetAwaiter().GetResult() is different from Result/Wait() in that it mimics the await exception propagation behavior. You need to decide if you want that or not. (So research what that behavior is; no need to repeat it here.) If your task contains a single exception then the await error behavior is usually convenient and has little downside. If there are multiple exceptions, for example from a failed Parallel loop where multiple tasks failed, then await will drop all exceptions but the first one. That makes debugging harder.
All these approaches have similar performance. They will allocate an OS event one way or another and block on it. That's the expensive part. The other machinery is rather cheap compared to that. I don't know which approach is absolutely cheapest.
In case an exception is being thrown, that is going to be the most expensive part. On .NET 5, exceptions are processed at a rate of at most 200,000 per second on a fast CPU. Deep stacks are slower, and the task machinery tends to rethrow exceptions multiplying their cost. There are ways of blocking on a task without the exception being rethrown, for example task.ContinueWith(_ => { }, TaskContinuationOptions.ExecuteSynchronously).Wait();.
I personally like the Task.Run(() => DoSomethingAsync()).Wait(); pattern because it avoids deadlocks categorically, it is simple and it does not hide some exceptions that GetResult() might hide. But you can use GetResult() as well with this.

I'm in the process of updating a library that has an API surface that was built in .NET 3.5. As a result, all methods are synchronous. I can't change the API (i.e., convert return values to Task) because that would require that all callers change. So I'm left with how to best call async methods in a synchronous way.
There is no universal "best" way to perform the sync-over-async anti-pattern. Only a variety of hacks that each have their own drawbacks.
What I recommend is that you keep the old synchronous APIs and then introduce asynchronous APIs alongside them. You can do this using the "boolean argument hack" as described in my MSDN article on Brownfield Async.
First, a brief explanation of the problems with each approach in your example:
ConfigureAwait only makes sense when there is an await; otherwise, it does nothing.
Result will wrap exceptions in an AggregateException; if you must block, use GetAwaiter().GetResult() instead.
Task.Run will execute its code on a thread pool thread (obviously). This is fine only if the code can run on a thread pool thread.
RunSynchronously is an advanced API used in extremely rare situations when doing dynamic task-based parallelism. You're not in that scenario at all.
Task.WaitAll with a single task is the same as just Wait().
async () => await x is just a less-efficient way of saying () => x.
Blocking on a task started from the current thread can cause deadlocks.
Here's the breakdown:
// Problems (1), (3), (6)
result = Task.Run(async () => await task()).ConfigureAwait(false).GetAwaiter().GetResult();
// Problems (1), (3)
result = Task.Run(task).ConfigureAwait(false).GetAwaiter().GetResult();
// Problems (1), (7)
result = task().ConfigureAwait(false).GetAwaiter().GetResult();
// Problems (2), (3)
result = Task.Run(task).Result;
// Problems (3)
result = Task.Run(task).GetAwaiter().GetResult();
// Problems (2), (4)
var t = task();
t.RunSynchronously();
result = t.Result;
// Problems (2), (5)
var t1 = task();
Task.WaitAll(t1);
result = t1.Result;
Instead of any of these approaches, since you have existing, working synchronous code, you should use it alongside the newer naturally-asynchronous code. For example, if your existing code used WebClient:
public string Get()
{
using (var client = new WebClient())
return client.DownloadString(...);
}
and you want to add an async API, then I would do it like this:
private async Task<string> GetCoreAsync(bool sync)
{
using (var client = new WebClient())
{
return sync ?
client.DownloadString(...) :
await client.DownloadStringTaskAsync(...);
}
}
public string Get() => GetCoreAsync(sync: true).GetAwaiter().GetResult();
public Task<string> GetAsync() => GetCoreAsync(sync: false);
or, if you must use HttpClient for some reason:
private string GetCoreSync()
{
using (var client = new WebClient())
return client.DownloadString(...);
}
private static HttpClient HttpClient { get; } = ...;
private async Task<string> GetCoreAsync(bool sync)
{
return sync ?
GetCoreSync() :
await HttpClient.GetString(...);
}
public string Get() => GetCoreAsync(sync: true).GetAwaiter().GetResult();
public Task<string> GetAsync() => GetCoreAsync(sync: false);
With this approach, your logic would go into the Core methods, which may be run synchronously or asynchronously (as determined by the sync parameter). If sync is true, then the core methods must return an already-completed task. For implemenation, use synchronous APIs to run synchronously, and use asynchronous APIs to run asynchronously.
Eventually, I recommend deprecating the synchronous APIs.

I just went thru this very thing with the AWS S3 SDK. Used to be sync, and I built a bunch of code on that, but now it's async. And that's fine: they changed it, nothing to be gained by moaning about it, move on.So I need to update my app, and my options are to either refactor a large part of my app to be async, or to "hack" the S3 async API to behave like sync.I'll eventually get around to the larger async refactoring - there are many benefits - but for today I have bigger fish to fry so I chose to fake the sync.Original sync code was ListObjectsResponse response = api.ListObjects(request); and a really simple async equivalent that works for me is Task<ListObjectsV2Response> task = api.ListObjectsV2Async(rq2);ListObjectsV2Response rsp2 = task.GetAwaiter().GetResult();
While I get it that purists might pillory me for this, the reality is that this is just one of many pressing issues and I have finite time so I need to make tradeoffs. Perfect? No. Works? Yes.

You Can Call Async Method From non-async method .Check below Code .
public ActionResult Test()
{
TestClass result = Task.Run(async () => await GetNumbers()).GetAwaiter().GetResult();
return PartialView(result);
}
public async Task<TestClass> GetNumbers()
{
TestClass obj = new TestClass();
HttpResponseMessage response = await APICallHelper.GetData(Functions.API_Call_Url.GetCommonNumbers);
if (response.IsSuccessStatusCode)
{
var result = response.Content.ReadAsStringAsync().Result;
obj = JsonConvert.DeserializeObject<TestClass>(result);
}
return obj;
}

async function never executed why in c# with quartz task backgrounder in c# [duplicate]

I have a multi-tier .Net 4.5 application calling a method using C#'s new async and await keywords that just hangs and I can't see why.
At the bottom I have an async method that extents our database utility OurDBConn (basically a wrapper for the underlying DBConnection and DBCommand objects):
public static async Task<T> ExecuteAsync<T>(this OurDBConn dataSource, Func<OurDBConn, T> function)
{
string connectionString = dataSource.ConnectionString;
// Start the SQL and pass back to the caller until finished
T result = await Task.Run(
() =>
{
// Copy the SQL connection so that we don't get two commands running at the same time on the same open connection
using (var ds = new OurDBConn(connectionString))
{
return function(ds);
}
});
return result;
}
Then I have a mid level async method that calls this to get some slow running totals:
public static async Task<ResultClass> GetTotalAsync( ... )
{
var result = await this.DBConnection.ExecuteAsync<ResultClass>(
ds => ds.Execute("select slow running data into result"));
return result;
}
Finally I have a UI method (an MVC action) that runs synchronously:
Task<ResultClass> asyncTask = midLevelClass.GetTotalAsync(...);
// do other stuff that takes a few seconds
ResultClass slowTotal = asyncTask.Result;
The problem is that it hangs on that last line forever. It does the same thing if I call asyncTask.Wait(). If I run the slow SQL method directly it takes about 4 seconds.
The behaviour I'm expecting is that when it gets to asyncTask.Result, if it's not finished it should wait until it is, and once it is it should return the result.
If I step through with a debugger the SQL statement completes and the lambda function finishes, but the return result; line of GetTotalAsync is never reached.
Any idea what I'm doing wrong?
Any suggestions to where I need to investigate in order to fix this?
Could this be a deadlock somewhere, and if so is there any direct way to find it?

Yep, that's a deadlock all right. And a common mistake with the TPL, so don't feel bad.
When you write await foo, the runtime, by default, schedules the continuation of the function on the same SynchronizationContext that the method started on. In English, let's say you called your ExecuteAsync from the UI thread. Your query runs on the threadpool thread (because you called Task.Run), but you then await the result. This means that the runtime will schedule your "return result;" line to run back on the UI thread, rather than scheduling it back to the threadpool.
So how does this deadlock? Imagine you just have this code:
var task = dataSource.ExecuteAsync(_ => 42);
var result = task.Result;
So the first line kicks off the asynchronous work. The second line then blocks the UI thread. So when the runtime wants to run the "return result" line back on the UI thread, it can't do that until the Result completes. But of course, the Result can't be given until the return happens. Deadlock.
This illustrates a key rule of using the TPL: when you use .Result on a UI thread (or some other fancy sync context), you must be careful to ensure that nothing that Task is dependent upon is scheduled to the UI thread. Or else evilness happens.
So what do you do? Option #1 is use await everywhere, but as you said that's already not an option. Second option which is available for you is to simply stop using await. You can rewrite your two functions to:
public static Task<T> ExecuteAsync<T>(this OurDBConn dataSource, Func<OurDBConn, T> function)
{
string connectionString = dataSource.ConnectionString;
// Start the SQL and pass back to the caller until finished
return Task.Run(
() =>
{
// Copy the SQL connection so that we don't get two commands running at the same time on the same open connection
using (var ds = new OurDBConn(connectionString))
{
return function(ds);
}
});
}
public static Task<ResultClass> GetTotalAsync( ... )
{
return this.DBConnection.ExecuteAsync<ResultClass>(
ds => ds.Execute("select slow running data into result"));
}
What's the difference? There's now no awaiting anywhere, so nothing being implicitly scheduled to the UI thread. For simple methods like these that have a single return, there's no point in doing an "var result = await...; return result" pattern; just remove the async modifier and pass the task object around directly. It's less overhead, if nothing else.
Option #3 is to specify that you don't want your awaits to schedule back to the UI thread, but just schedule to the thread pool. You do this with the ConfigureAwait method, like so:
public static async Task<ResultClass> GetTotalAsync( ... )
{
var resultTask = this.DBConnection.ExecuteAsync<ResultClass>(
ds => return ds.Execute("select slow running data into result");
return await resultTask.ConfigureAwait(false);
}
Awaiting a task normally would schedule to the UI thread if you're on it; awaiting the result of ContinueAwait will ignore whatever context you are on, and always schedule to the threadpool. The downside of this is you have to sprinkle this everywhere in all functions your .Result depends on, because any missed .ConfigureAwait might be the cause of another deadlock.

This is the classic mixed-async deadlock scenario, as I describe on my blog. Jason described it well: by default, a "context" is saved at every await and used to continue the async method. This "context" is the current SynchronizationContext unless it it null, in which case it is the current TaskScheduler. When the async method attempts to continue, it first re-enters the captured "context" (in this case, an ASP.NET SynchronizationContext). The ASP.NET SynchronizationContext only permits one thread in the context at a time, and there is already a thread in the context - the thread blocked on Task.Result.
There are two guidelines that will avoid this deadlock:
Use async all the way down. You mention that you "can't" do this, but I'm not sure why not. ASP.NET MVC on .NET 4.5 can certainly support async actions, and it's not a difficult change to make.
Use ConfigureAwait(continueOnCapturedContext: false) as much as possible. This overrides the default behavior of resuming on the captured context.

I was in the same deadlock situation but in my case calling an async method from a sync method, what works for me was:
private static SiteMetadataCacheItem GetCachedItem()
{
TenantService TS = new TenantService(); // my service datacontext
var CachedItem = Task.Run(async ()=>
await TS.GetTenantDataAsync(TenantIdValue)
).Result; // dont deadlock anymore
}
is this a good approach, any idea?

Just to add to the accepted answer (not enough rep to comment), I had this issue arise when blocking using task.Result, event though every await below it had ConfigureAwait(false), as in this example:
public Foo GetFooSynchronous()
{
var foo = new Foo();
foo.Info = GetInfoAsync.Result; // often deadlocks in ASP.NET
return foo;
}
private async Task<string> GetInfoAsync()
{
return await ExternalLibraryStringAsync().ConfigureAwait(false);
}
The issue actually lay with the external library code. The async library method tried to continue in the calling sync context, no matter how I configured the await, leading to deadlock.
Thus, the answer was to roll my own version of the external library code ExternalLibraryStringAsync, so that it would have the desired continuation properties.
wrong answer for historical purposes
After much pain and anguish, I found the solution buried in this blog post (Ctrl-f for 'deadlock'). It revolves around using task.ContinueWith, instead of the bare task.Result.
Previously deadlocking example:
public Foo GetFooSynchronous()
{
var foo = new Foo();
foo.Info = GetInfoAsync.Result; // often deadlocks in ASP.NET
return foo;
}
private async Task<string> GetInfoAsync()
{
return await ExternalLibraryStringAsync().ConfigureAwait(false);
}
Avoid the deadlock like this:
public Foo GetFooSynchronous
{
var foo = new Foo();
GetInfoAsync() // ContinueWith doesn't run until the task is complete
.ContinueWith(task => foo.Info = task.Result);
return foo;
}
private async Task<string> GetInfoAsync
{
return await ExternalLibraryStringAsync().ConfigureAwait(false);
}

quick answer :
change this line
ResultClass slowTotal = asyncTask.Result;
to
ResultClass slowTotal = await asyncTask;
why? you should not use .result to get the result of tasks inside most applications except console applications if you do so your program will hang when it gets there
you can also try the below code if you want to use .Result
ResultClass slowTotal = Task.Run(async ()=>await asyncTask).Result;

How to convert this Parallel.ForEach code to async/await

I'm having some trouble getting my head around async/await. I'm helping with an existing code base that has the following code (simplified, for brevity):
List<BuyerContext> buyerContexts = GetBuyers();
var results = new List<Result>();
Parallel.ForEach(buyerContexts, buyerContext =>
{
//The following call creates a connection to a remote web server that
//can take up to 15 seconds to respond
var result = Bid(buyerContext);
if (result != null)
results.Add(result);
}
foreach (var result in results)
{
// do some work here that is predicated on the
// Parallel.ForEach having completed all of its calls
}
How can i convert this code to asynchronous code instead of parallel using async/await? I'm suffering from some pretty severe performance issues that I believe are a result of using a parallel approach to multiple network I/O operations.
I've tried several approaches myself but I'm getting warnings from Visual Studio that my code will execute synchronously or that I can't use await keywords outside of an async method so I'm sure I'm just missing something simple.
EDIT #1: I'm open to alternatives to async/await as well. That just seems to be the proper approach based on my reading so far.
EDIT #2: This application is a Windows Service. It calls out to several "buyers" to ask them to bid on a particular piece of data. I need ALL of the bids back before processing can continue.

The key to "making things async" is to start at the leaves. In this case, start in your network code (not shown), and change whatever synchronous call you have (e.g., WebClient.DownloadString) to the corresponding asynchronous call (e.g., HttpClient.GetStringAsync). Then await that call.
Using await will force the calling method to be async, and change its return type from T to Task<T>. It is also a good idea at this point to add the Async suffix so you're following the well-known convention. Then take all of that method's callers and change them to use await as well, which will then require them to be async, etc. Repeat until you have a BidAsync method to use.
Then you should look at replacing your parallel loop; this is pretty easy to do with Task.WhenAll:
List<BuyerContext> buyerContexts = GetBuyers();
var tasks = buyerContexts.Select(buyerContext => BidAsync(buyerContext));
var results = await Task.WhenAll(tasks);
foreach (var result in results)
{
...
}

Basically, to make use of async-await, the Bid method should have this signature instead of the current one:
public async Task<Result> BidAsync(BuyerContext buyerContext);
This will allow you to use await in this method. Now, every time you make a network call, you basically need to await it. For example, here's how to modify the call and signature of a synchronous method to an asynchronous one.
Before
//Signature
public string ReceiveStringFromClient();
//Call
string messageFromClient = ReceiveStringFromClient();
After
//Signature
public Task<string> ReceiveStringFromClientAsync();
//Call
string messageFromClient = await ReceiveStringFromClientAsync();
If you still need to be able to make synchronous calls to these methods, I would recommend creating new ones suffixed with "Async".
Now you need to do this on every level until you reach your network calls, at which point you'll be able to await .Net's async methods. They normally have the same name as their synchronous version, suffixed with "Async".
Once you've done all that, you can make use of this in your main code. I would do something along these lines:
List<BuyerContext> buyerContexts = GetBuyers();
var results = new List<Result>();
List<Task> tasks = new List<Task>();
//There really is no need for Parallel.ForEach unless you have hundreds of thousands of requests to make.
//If that's the case, I hope you have a good network interface!
foreach (var buyerContext in buyerContexts)
{
var task = Task.Run(async () =>
{
var result = await BidAsync(buyerContext);
if (result != null)
results.Add(result);
});
tasks.Add(task);
}
//Block the current thread until all the calls are completed
Task.WaitAll(tasks);
foreach (var result in results)
{
// do some work here that is predicated on the
// Parallel.ForEach having completed all of its calls
}

async library calls relying on each other and how to handle?

this is a follow on from a previous question I posted Calling an async method using a Task.Run seems wrong?
I thought the code written by the contractor was wrong but following on from the answers provided I'm now wondering if it's the libraries fault. This library exposes two methods that I need to use. One returns a "template" and one consumes part of this template (it does in my implementation anyway). But both are async methods returning Tasks.
To explain my library has methods:
public Task<TemplateInfo> TemplateInfo(TemplateInfoRequest request);
public Task<List<EmailResult>> SendMessageTemplate(SendMessageTemplateRequest request);
I need to call these thus:
public bool SendMessage()
{
var template = TemplateInfo(...);
var message = //use template to create message
var sendResponse = SendMessageTemplate(message);
return sendResponse.Count > 0;
}
So the second call relies on the first. This is where the async doesn't make sense to me. I can't and don't want to run these in parallel. I want to run them in sequence. I want my method to by synchonous.
An answer in my previous question also states:
Since you're blocking on an async method (which you shouldn't do)
there is a chance you'll deadlock.
So how do I access these async methods in such a way that they are processed in turn and return a synconhonous result but do not cause deadlocks?

Async doesn't (necessarily) means parallel.
You can call both async method sequentially without them running in parallel. Simply call the first and await the returned task and then do the same with the second.
var template = await TemplateInfo(...);
var message = //use template to create message
var sendResponse = await SendMessageTemplate(message);
This is still useful compared to synchronous code because while the asynchronous operation is "running" there's no thread needed and so your threads can go work on other parts of your application.
If you would call both and only then await the returned tasks you will (maybe) run in parallel:
var templateTask = TemplateInfo(...);
var sendResponseTask = SendMessageTemplate(message);
await templateTask;
await sendResponseTask;