I'm trying to get my head around this code:
[TestFixture]
public class ExampleTest
{
[Test]
public void Example()
{
AwaitEmptyTask().Wait();
}
public async Task AwaitEmptyTask()
{
await new Task(() => { });
}
}
The method Example never ends and blocks forever. Why??
The fix (from Stubbing Task returning method in async unit test) is to replace await new Task( () => {}) with return Task.FromResult<object>(null); but again, why is this necessary?
I know there are a bunch of questions similar to this one, but none that I've seen seem to explain why this is happening:
Await method that returns Task - spins forever?: I included the await keyword AFAIK correctly
Stubbing Task returning method in async unit test: Doesn't explain why.
Why will an empty .NET Task not complete if started and waited for from a static constructor?: I'm not running in a static context and as far as I can tell the context is consistent
Async methods return null: also gives the solution, but doesn't explain why
async await return Task: discusses Task.FromResult but not why await new Task(() => {}) doesn't work
You're creating a task and never starting it, so it never finishes.
You should be using Task.Run to create a task that you want to start executing immediately, rather than using the Task constructor.
You need to call Start() on your task.
Otherwise, it will never finish.
Even if you changed it to
await Task.Run(() => { });
it may still never complete. When you hit this line, the program creates a continuation task that will continue when the task completes. The fact that the task is empty is irrelevant. At that point, control flow goes back to
AwaitEmptyTask().Wait();
which waits for the task to complete and blocks the current thread.
When the task does complete, it attempts to continue on the current thread. However, it can't, because the thread is blocked by the call to Wait().
Change the awaited line to
await Task.Run(() => { }).ConfigureAwait(false);
This is a common deadlocking problem. Don't mix Wait() and await. See http://msdn.microsoft.com/en-us/magazine/jj991977.aspx
Related
Ex, the following code manually instantiates a Task and passes to a Task.WhenAll in a List<T>
public async Task Do3()
{
var task1 = new Task(async () => { await Task.Delay(2000); Console.WriteLine("########## task1"); });
var taskList = new List<Task>() { task1};
taskList[0].Start();
var taskDone = Task.WhenAll(taskList);
await taskDone;
}
without starting the Task it doesn't work, it hangs forever calling from a console app, but the below works just fine without starting it
public async Task Do3()
{
//var task1 = new Task(async () => { await Task.Delay(2000); Console.WriteLine("########## task1"); });
var taskList = new List<Task>() { SubDo1() };
//taskList[0].Start();
var taskDone = Task.WhenAll(taskList);
await taskDone;
}
public async Task SubDo1()
{
await Task.Delay(2000);
Console.WriteLine("########## task1");
}
Task is used in two completely different ways here; when you call an async method: you are starting it yourself; at this point, two things can happen:
it can run to completion (eventually) without ever reaching a truly asynchronous state, and return a completed (or faulted) task to the caller
it can reach an incomplete awaitable (in this case await Task.Delay), at which point it creates a state machine that represents the current position, schedules a completion operation on that incomplete awaitable (to do whatever comes next), and then returns an incomplete task to the caller
It is not "not started"; to return anything to the caller: we have started it. However, unlike Task.Start(), we start that work on our current thread - not an external worker thread - with other threads only getting involved based on how that incomplete awaitable schedules the completion callbacks that the compiler gives it.
This is very different to the new Task(...) scenario, where nothing is initially started. That's why they behave differently. Note also the Remarks section of the Task constructor here - it is a very niche API, and honestly: not hugely recommended.
Additionally: when you don't immediately await an async method, you're essentially going into concurrent territory (assuming the awaitable won't always complete synchronously). In some cases, this matters, and may cause threading problems re race-conditions. It shouldn't matter much in this case, though.
** I've summarised this question at the bottom with an edit **
This has been asked before but I think my circumstances are different.
I am processing multiple requests simultaneously.
This is my code to do that, it runs in a loop. I've removed a bit of code that handles the taskAllocated variable for brevity.
while (!taskAllocated)
{
lock (_lock)
{
// Find an empty slot in the task queue to insert this task
for (i = 0; i < MaxNumTasks; i++)
{
if (_taskQueue[i] == null)
{
_taskQueue[i] = Task.Run(() => Process());
_taskQueue[i].ContinueWith(ProcessCompleted);
break;
}
}
}
}
Process is a typical async Task Process() { CpuIntensiveStuff(); } method.
I've been running the above code, and it has been working fine. It multithreads nicely. Whenever an item comes in, it will find an empty slot in the task queue, and kick it off. When the task completes, the ProcessCompleted method runs, and frees up the slot.
But then I thought, shouldn't I be using await inside my Task.Run? Something like:
_taskQueue[i] = Task.Run(async () => await Process());
After thinking about it, I'm not sure. ContinueWith triggers correctly, when the task has completed, so perhaps it's not necessary.
I ask because I wanted to monitor and log how long each task takes to complete.
So Instead of Process(), I would make another method like:
async Task DoProcess()
{
var sw = Stopwatch.StartNew();
Process();
sw.Stop();
Log(sw.ElapsedMilliseconds);
}
And it occurred to me that if I did that, I wasn't sure if I'd need to await Process(); or not, in addition to not knowing if I should await inside the Task.Run()
I'm in a bit of a tizz about this. Can anyone offer guidance?
Edit:
To summarise:
If Somemethod is:
void SomeMethod() { }
Then
Task.Run(() => SomeMethod()); is great, calls SomeMethod on a new 'thread' (not technically, but you know what I mean).
However, my SomeMethod is actually:
async Task SomeMethod() { }
Do you need to do anything special with Task.Run()?
My code, I am not, I am just straight up ignoring that it's an async Task, and that seems to work:
Task.Run(() => SomeMethod()); // SomeMethod is async Task but I am ignoring that
But I'm not convinced that it a) should work or b) is a good idea. The alternative could be to do:
Task.Run(async() => await SomeMethod());
But is there any point? And this is compounded by the fact I want to really do:
Task.Run(() =>
{
someCode();
var x = startTimer();
SomeMethod();
var y = stopTimer();
someMoreCode()
});
but without await I'm not sure it will wait for somemethod to finish and the timer will be wrong.
Things become more clear if you do not use anonymous methods. For example,
Task.Run(() => Process())
is equivalent to this:
Task.Run(DoSomething);
Task DoSomething() {
return Process();
}
Whereas
Task.Run(async () => await Process())
is equivalent to this:
Task.Run(DoSomething);
async Task DoSomething() {
await Process();
}
In most cases, there is no functional difference between return SomethingThatReturnsATask() and return await SomethingThatReturnsATask(), and you usually want to return the Task directly and not use await (for reasons described here). When used inside Task.Run, things could easily go bad if the .NET team didn't have your back.
It is important to note that asynchronous methods start running on the same thread just like any other method. The magic happens at the first await that acts on an incomplete Task. At that point, await returns its own incomplete Task. That's important - it returns, with a promise to do the rest later.
This could have meant that the Task returned from Task.Run would complete whenever Process() returns a Task. And since Process() returns a Task at the first await, that would happen when it has not yet totally completed.
The .NET team has your back
That is not the case however, because Task.Run has a specific overload for when you give it a method returning a Task. And if you look at the code, it returns a Task *that is tied to the Task you return.
That means that the Task returned from Task.Run(() => Process()) will not complete until the Task returned from Process() has completed.
So your code is fine the way it is.
I'm struggling to understand what's happening in this simple program.
In the example below I have a task factory that uses the LimitedConcurrencyLevelTaskScheduler from ParallelExtensionsExtras with maxDegreeOfParallelism set to 2.
I then start 2 tasks that each call an async method (e.g. an async Http request), then gets the awaiter and the result of the completed task.
The problem seem to be that Task.Delay(2000) never completes. If I set maxDegreeOfParallelism to 3 (or greater) it completes. But with maxDegreeOfParallelism = 2 (or less) my guess is that there is no thread available to complete the task. Why is that?
It seems to be related to async/await since if I remove it and simply do Task.Delay(2000).GetAwaiter().GetResult() in DoWork it works perfectly. Does async/await somehow use the parent task's task scheduler, or how is it connected?
using System;
using System.Linq;
using System.Threading.Tasks;
using System.Threading.Tasks.Schedulers;
namespace LimitedConcurrency
{
class Program
{
static void Main(string[] args)
{
var test = new TaskSchedulerTest();
test.Run();
}
}
class TaskSchedulerTest
{
public void Run()
{
var scheduler = new LimitedConcurrencyLevelTaskScheduler(2);
var taskFactory = new TaskFactory(scheduler);
var tasks = Enumerable.Range(1, 2).Select(id => taskFactory.StartNew(() => DoWork(id)));
Task.WaitAll(tasks.ToArray());
}
private void DoWork(int id)
{
Console.WriteLine($"Starting Work {id}");
HttpClientGetAsync().GetAwaiter().GetResult();
Console.WriteLine($"Finished Work {id}");
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}
}
Thanks in advance for any help
await by default captures the current context and uses that to resume the async method. This context is SynchronizationContext.Current, unless it is null, in which case it is TaskScheduler.Current.
In this case, await is capturing the LimitedConcurrencyLevelTaskScheduler used to execute DoWork. So, after starting the Task.Delay both times, both of those threads are blocked (due to the GetAwaiter().GetResult()). When the Task.Delay completes, the await schedules the remainder of the HttpClientGetAsync method to its context. However, the context will not run it since it already has 2 threads.
So you end up with threads blocked in the context until their async methods complete, but the async methods cannot complete until there is a free thread in the context; thus a deadlock. Very similar to the standard "don't block on async code" style of deadlock, just with n threads instead of one.
Clarifications:
The problem seem to be that Task.Delay(2000) never completes.
Task.Delay is completing, but the await cannot continue executing the async method.
If I set maxDegreeOfParallelism to 3 (or greater) it completes. But with maxDegreeOfParallelism = 2 (or less) my guess is that there is no thread available to complete the task. Why is that?
There are plenty of threads available. But the LimitedConcurrencyTaskScheduler only allows 2 threads at a time to run in its context.
It seems to be related to async/await since if I remove it and simply do Task.Delay(2000).GetAwaiter().GetResult() in DoWork it works perfectly.
Yes; it's the await that is capturing the context. Task.Delay does not capture a context internally, so it can complete without needing to enter the LimitedConcurrencyTaskScheduler.
Solution:
Task schedulers in general do not work very well with asynchronous code. This is because task schedulers were designed for Parallel Tasks rather than asynchronous tasks. So they only apply when code is running (or blocked). In this case, LimitedConcurrencyLevelTaskScheduler only "counts" code that's running; if you have a method that's doing an await, it won't "count" against that concurrency limit.
So, your code has ended up in a situation where it has the sync-over-async antipattern, probably because someone was trying to avoid the problem of await not working as expected with limited concurrency task schedulers. This sync-over-async antipattern has then caused the deadlock problem.
Now, you could add in more hacks by using ConfigureAwait(false) everywhere and continue blocking on asynchronous code, or you could fix it better.
A more proper fix would be to do asynchronous throttling. Toss out the LimitedConcurrencyLevelTaskScheduler completely; concurrency-limiting task schedulers only work with synchronous code, and your code is asynchronous. You can do asynchronous throttling using SemaphoreSlim, as such:
class TaskSchedulerTest
{
private readonly SemaphoreSlim _mutex = new SemaphoreSlim(2);
public async Task RunAsync()
{
var tasks = Enumerable.Range(1, 2).Select(id => DoWorkAsync(id));
await Task.WhenAll(tasks);
}
private async Task DoWorkAsync(int id)
{
await _mutex.WaitAsync();
try
{
Console.WriteLine($"Starting Work {id}");
await HttpClientGetAsync();
Console.WriteLine($"Finished Work {id}");
}
finally
{
_mutex.Release();
}
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}
I think you are encountering a sync deadlock. You are waiting for a thread to complete that is waiting for your thread to complete. Never going to happen. If you make your DoWork method async so you can await the HttpClientGetAsync() call, and you'll avoid the deadlock.
using MassTransit.Util;
using System;
using System.Linq;
using System.Threading.Tasks;
//using System.Threading.Tasks.Schedulers;
namespace LimitedConcurrency
{
class Program
{
static void Main(string[] args)
{
var test = new TaskSchedulerTest();
test.Run();
}
}
class TaskSchedulerTest
{
public void Run()
{
var scheduler = new LimitedConcurrencyLevelTaskScheduler(2);
var taskFactory = new TaskFactory(scheduler);
var tasks = Enumerable.Range(1, 2).Select(id => taskFactory.StartNew(() => DoWork(id)));
Task.WaitAll(tasks.ToArray());
}
private async Task DoWork(int id)
{
Console.WriteLine($"Starting Work {id}");
await HttpClientGetAsync();
Console.WriteLine($"Finished Work {id}");
}
async Task HttpClientGetAsync()
{
await Task.Delay(2000);
}
}
}
https://medium.com/rubrikkgroup/understanding-async-avoiding-deadlocks-e41f8f2c6f5d
TLDR never call .result, which I'm sure .GetResult(); was doing
I want to call an asynchronous method multiple times in a xUnit test and wait for all calls to complete before I continue execution. I read that I can use Task.WhenAll() and Task.WaitAll() for precisely this scenario. For some reason however, the code is deadlocking.
[Fact]
public async Task GetLdapEntries_ReturnsLdapEntries()
{
var ldapEntries = _fixture.CreateMany<LdapEntryDto>(2).ToList();
var creationTasks = new List<Task>();
foreach (var led in ldapEntries)
{
var task = _attributesServiceClient.CreateLdapEntry(led);
task.Start();
creationTasks.Add(task);
}
Task.WaitAll(creationTasks.ToArray()); //<-- deadlock(?) here
//await Task.WhenAll(creationTasks);
var result = await _ldapAccess.GetLdapEntries();
result.Should().BeEquivalentTo(ldapEntries);
}
public async Task<LdapEntryDto> CreateLdapEntry(LdapEntryDto ldapEntryDto)
{
using (var creationResponse = await _httpClient.PostAsJsonAsync<LdapEntryDto>("", ldapEntryDto))
{
if (creationResponse.StatusCode == HttpStatusCode.Created)
{
return await creationResponse.Content.ReadAsAsync<LdapEntryDto>();
}
throw await buildException(creationResponse);
}
}
The system under test is a wrapper around an HttpClient that calls a web service, awaits the response, and possibly awaits reading the response's content that is finally deserialized and returned.
When I change the foreach part in the test to the following (ie, don't use Task.WhenAll() / WaitAll()), the code is running without a deadlock:
foreach (var led in ldapEntries)
{
await _attributesServiceClient.CreateLdapEntry(led);
}
What exactly is happening?
EDIT: While this question has been marked as duplicate, I don't see how the linked question relates to this one. The code examples in the link all use .Result which, as far as I understand, blocks the execution until the task has finished. In contrast, Task.WhenAll() returns a task that can be awaited and that finishes when all tasks have finished. So why is awaiting Task.WhenAll() deadlocking?
The code you posted cannot possibly have the behavior described. The first call to Task.Start would throw an InvalidOperationException, failing the test.
I read that I can use Task.WhenAll() and Task.WaitAll() for precisely this scenario.
No; to asynchronously wait on multiple tasks, you must use Task.WhenAll, not Task.WaitAll.
Example:
[Fact]
public async Task GetLdapEntries_ReturnsLdapEntries()
{
var ldapEntries = new List<int> { 0, 1 };
var creationTasks = new List<Task>();
foreach (var led in ldapEntries)
{
var task = CreateLdapEntry(led);
creationTasks.Add(task);
}
await Task.WhenAll(creationTasks);
}
public async Task<string> CreateLdapEntry(int ldapEntryDto)
{
await Task.Delay(500);
return "";
}
Task.WaitAll() will deadlock simply because it blocks the current thread while the tasks are not finished (and since you are using async/await and not threads, all of your tasks are running on the same thread, and you are not letting your awaited tasks to go back to the calling point because the thread they are running in -the same one where you called Task.WaitAll()-, is blocked).
Not sure why WhenAll is also deadlocking for you here though, it definitely shouldn't.
PS: you don't need to call Start on tasks returned by an async method: they are "hot" (already started) already upon creation
I'm still getting up to speed with async & multi threading. I'm trying to monitor when the Task I Start is still running (to show in a UI). However it's indicating that it is RanToCompletion earlier than I want, when it hits an await, even when I consider its Status as still Running.
Here is the sample I'm doing. It all seems to be centred around the await's. When it hits an await, it is then marked as RanToCompletion.
I want to keep track of the main Task which starts it all, in a way which indicates to me that it is still running all the way to the end and only RanToCompletion when it is all done, including the repo call and the WhenAll.
How can I change this to get the feedback I want about the tskProdSeeding task status?
My Console application Main method calls this:
Task tskProdSeeding;
tskProdSeeding = Task.Factory.StartNew(SeedingProd, _cts.Token);
Which the runs this:
private async void SeedingProd(object state)
{
var token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
await Task.WhenAll(Task.Run(async () =>
{
foreach (var code in codesToSeed)
{
if (!token.IsCancellationRequested)
{
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
}
}, token));
Thread.Sleep(30000);
}
}
If you use async void the outer task can't tell when the task is finished, you need to use async Task instead.
Second, once you do switch to async Task, Task.Factory.StartNew can't handle functions that return a Task, you need to switch to Task.Run(
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
Once you do both of those changes you will be able to await or do a .Wait() on tskProdSeeding and it will properly wait till all the work is done before continuing.
Please read "Async/Await - Best Practices in Asynchronous Programming" to learn more about not doing async void.
Please read "StartNew is Dangerous" to learn more about why you should not be using StartNew the way you are using it.
P.S. In SeedingProd you should switch it to use await Task.Delay(30000); insetad of Thread.Sleep(30000);, you will then not tie up a thread while it waits. If you do this you likely could drop the
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
and just make it
tskProdSeeding = SeedingProd(_cts.Token);
because the function no-longer has a blocking call inside of it.
I'm not convinced that you need a second thread (Task.Run or StartNew) at all. It looks like the bulk of the work is I/O-bound and if you're doing it asynchronously and using Task.Delay instead of Thread.Sleep, then there is no thread consumed by those operations and your UI shouldn't freeze. The first thing anyone new to async needs to understand is that it's not the same thing as multithreading. The latter is all about consuming more threads, the former is all about consuming fewer. Focus on eliminating the blocking and you shouldn't need a second thread.
As others have noted, SeedingProd needs to return a Task, not void, so you can observe its completion. I believe your method can be reduced to this:
private async Task SeedingProd(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
foreach (var code in codesToSeed)
{
if (token.IsCancellationRequested)
return;
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
await Task.Dealy(30000);
}
}
Then simply call the method, without awaiting it, and you'll have your task.
Task mainTask = SeedingProd(token);
When you specify async on a method, it compiles into a state machine with a Task, so SeedingProd does not run synchronously, but acts as a Task even if returns void. So when you call Task.Factory.StartNew(SeedingProd) you start a task that kick off another task - that's why the first one finishes immediately before the second one. All you have to do is add the Task return parameter instead of void:
private async Task SeedingProdAsync(CancellationToken ct)
{
...
}
and call it as simply as this:
Task tskProdSeeding = SeedingProdAsync(_cts.Token);