We Keep Coding

WebAPI HTTP request not completing until queued work kicks off on background task

In my .Net 6 WebPI service, I am queueing work to a background task queue, very closely based on the example here, but I could post parts of my implementation if that would help:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-6.0&tabs=visual-studio#queued-background-tasks
I am running into unexpected behavior where control is not returned to the caller, even after the return Ok(..) completes in the controller. Instead the request only completes after the await Task.Delay(1000); line is reached on the queued work item. The request returns to the client as soon as this line is reached, and does not need to wait for the Delay to finish.
I'm guessing this is because of the await either starting a new async context, or someone un-sticking the async context of the original request. My intention is for the request to complete immediately after queuing the work item.
Any insight into what is happening here would be greatly appreciated.
Controller:
public async Task<ActionResult> Test()
{
var resultMessage = await _testQueue.DoIt();
return Ok(new { Message = resultMessage });
}
Queueing Work:
public TestAdapter(ITaskQueue taskQueue)
{
_taskQueue = taskQueue;
}
public async Task<string> DoIt()
{
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) =>
{
await Task.Delay(1000);
var y = 12;
});
return "cool";
}
IoC:
services.AddHostedService<QueueHostedService>();
services.AddSingleton<ITTaskQueue>(ctx =>
{
return new TaskQueue(MaxQueueCount);
});
TaskQueue:
private readonly Channel<BackgroundTaskContext> _queue;
public TaskQueue(int capacity)
{
var options = new BoundedChannelOptions(capacity)
{
FullMode = BoundedChannelFullMode.Wait
};
_queue = Channel.CreateBounded<BackgroundTaskContext>(options);
}
public async ValueTask QueueBackgroundWorkItemAsync(
Func<CancellationToken, ValueTask> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
await _queue.Writer.WriteAsync(new BackgroundTaskContext(workItem, ...));
}

Not sure what you expect here. I'm assuming you want the async method to return the cool in the api response. That's fine but because your also awaiting the async call with in DoIt(), then it pauses until QueueBackgroundWorkItemAsync finishes. You could remove the await and it will run and return as you expect.
I can't say that I'm a big fan of that design as you lose contact with it with exception of the cancellation token. Another thought would be to Send that work off to a console job or function app using message bus or even another http call.
Additional Notes:
Async can be complicated to explain because in reality, it wraps up code and executes on a thread of it's choosing. The await simulates the synchronous behavior.
await Task.Delay(1000); // Runs on it's own thread but still halts code execution for 1 second.
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) // Waits for control to be returned from the code inside.
var resultMessage = await _testQueue.DoIt(); // Always waits for the code inside to complete.
If your wanting something to run without pausing code execution, you can either remove the await or add a Task.Run(() => { }); pattern. Is it a good idea is a whole other question. It also matters whether you need information back from the async method. If you don't await it then you'll get null back as it doesn't wait around for the answer to be computed.

This appears just to have been user error using the debugger. The debugger is switching to the background task thread and hitting breakpoints there before the response fully returns giving the appearance that control was not being returned to the client and being carried into the background task.
Even after adding some synchronous steps in QueueBackgroundWorkItemAsync and putting breakpoints on them, control was not immediately returned to the original http call. Only after I tried adding a long running task like await Task.Delay(1000); did enough time/ticks pass for the http response to return. I had conflated this with just the await somehow freeing up the original http context.

async wait leading to deadlock

I have an async codeblock running on the pageload.
The codes run smoothy until you reach capturevalue method where we create a new task.On executing that code block the await just freezes and then the control doesnt come back seems like the code just went to deadlock
protected void Page_Load(object sender, EventArgs e)
{
try
{
var textvalue = GetTextValueFromTask();
txtbox.Text = textvalue.Result;
string ss = "";
}
catch (Exception ex)
{
Console.WriteLine(ex.ToString());
}
}
private async Task<string> GetTextValueFromTask()
{
string strReturn = await CaptureValue();
return strReturn;
}
private async Task<string> CaptureValue()
{
Thread.Sleep(5000);
Task<string> T = Task.Factory.StartNew<string>(() => "hi");
return await T;
}
Then I made a small change in Capturevalue method.
private async Task<string> CaptureValue()
{
Thread.Sleep(5000);
Task<string> T = Task.Factory.StartNew<string>(() => "hi");
string ss = T.Result;
return await T;
}
Once I made that change it started working normal.What difference did it make on just fetching the result initially. Please help me Iam a newbee to async

The difference is that second time it doesn't happen any "await" because you waited the task yourself, so await doesn't do anything.
I think you missed the await keyword the first time, here:
var textvalue = await GetTextValueFromTask();
Without it your method GetTextValueFromTask runs synchronously, then it comes into CaptureValue method where await occurs. But the default behaviour of the await is that it tries to capture synchronization context where it was called and to continue the rest of the method in that context, in your example it is WPF synchronization context, which does not allow more than one thread to execute at once. But the continuation cannot proceed, because context is already used by await mechanism.
So, one more time. There is one thread (UI thread), that executes your code up to the last await, which is return await T;, then it yields back to the caller - GetTextValueFromTask, and again to the Page_Load when it gets blocked, because initially you called GetTextValueFromTask synchronously (without await). After that, your operation T completes, and your code tries to continue executing using the initial synchronization context, the WPF one. But it can't, because it is already waiting in the Page_Load.
The link in the comments describes the situation in more detail.
Also consider not using Thread.Sleep in async/await scenarios, because it kills all the "asynchronous" nature of the code. Further reading: link.
Another general piece of advice, which is not directly applicable to your source code, is not to use Task.Factory.StartNew, but rather use Task.Run. Explanation here.

Please use Task.Run() instead of Task.Factory.StartNew()
var T = Task.Run(() => "hi");
It's up to Task.Run to decide how to handle this task.
Also please use .ConfigureAwait(false) in your await calls that do not requires the continuation being done in the awaiter thread context.

Task being marked as RanToCompletion at await, when still Running

I'm still getting up to speed with async & multi threading. I'm trying to monitor when the Task I Start is still running (to show in a UI). However it's indicating that it is RanToCompletion earlier than I want, when it hits an await, even when I consider its Status as still Running.
Here is the sample I'm doing. It all seems to be centred around the await's. When it hits an await, it is then marked as RanToCompletion.
I want to keep track of the main Task which starts it all, in a way which indicates to me that it is still running all the way to the end and only RanToCompletion when it is all done, including the repo call and the WhenAll.
How can I change this to get the feedback I want about the tskProdSeeding task status?
My Console application Main method calls this:
Task tskProdSeeding;
tskProdSeeding = Task.Factory.StartNew(SeedingProd, _cts.Token);
Which the runs this:
private async void SeedingProd(object state)
{
var token = (CancellationToken)state;
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
await Task.WhenAll(Task.Run(async () =>
{
foreach (var code in codesToSeed)
{
if (!token.IsCancellationRequested)
{
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
}
}, token));
Thread.Sleep(30000);
}
}

If you use async void the outer task can't tell when the task is finished, you need to use async Task instead.
Second, once you do switch to async Task, Task.Factory.StartNew can't handle functions that return a Task, you need to switch to Task.Run(
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
Once you do both of those changes you will be able to await or do a .Wait() on tskProdSeeding and it will properly wait till all the work is done before continuing.
Please read "Async/Await - Best Practices in Asynchronous Programming" to learn more about not doing async void.
Please read "StartNew is Dangerous" to learn more about why you should not be using StartNew the way you are using it.
P.S. In SeedingProd you should switch it to use await Task.Delay(30000); insetad of Thread.Sleep(30000);, you will then not tie up a thread while it waits. If you do this you likely could drop the
tskProdSeeding = Task.Run(() => SeedingProd(_cts.Token), _cts.Token);
and just make it
tskProdSeeding = SeedingProd(_cts.Token);
because the function no-longer has a blocking call inside of it.

I'm not convinced that you need a second thread (Task.Run or StartNew) at all. It looks like the bulk of the work is I/O-bound and if you're doing it asynchronously and using Task.Delay instead of Thread.Sleep, then there is no thread consumed by those operations and your UI shouldn't freeze. The first thing anyone new to async needs to understand is that it's not the same thing as multithreading. The latter is all about consuming more threads, the former is all about consuming fewer. Focus on eliminating the blocking and you shouldn't need a second thread.
As others have noted, SeedingProd needs to return a Task, not void, so you can observe its completion. I believe your method can be reduced to this:
private async Task SeedingProd(CancellationToken token)
{
while (!token.IsCancellationRequested)
{
int totalSeeded = 0;
var codesToSeed = await _myRepository.All().ToListAsync(token);
foreach (var code in codesToSeed)
{
if (token.IsCancellationRequested)
return;
try
{
int seedCountByCode = await _myManager.SeedDataFromLive(code);
totalSeeded += seedCountByCode;
}
catch (Exception ex)
{
_logger.InfoFormat(ex.ToString());
}
}
await Task.Dealy(30000);
}
}
Then simply call the method, without awaiting it, and you'll have your task.
Task mainTask = SeedingProd(token);

When you specify async on a method, it compiles into a state machine with a Task, so SeedingProd does not run synchronously, but acts as a Task even if returns void. So when you call Task.Factory.StartNew(SeedingProd) you start a task that kick off another task - that's why the first one finishes immediately before the second one. All you have to do is add the Task return parameter instead of void:
private async Task SeedingProdAsync(CancellationToken ct)
{
...
}
and call it as simply as this:
Task tskProdSeeding = SeedingProdAsync(_cts.Token);

Await stops execution of thread and never continues

I have the following:
public async Task<bool> SearchForUpdatesAsync()
{
return await TaskEx.Run(() =>
{
if (!ConnectionChecker.IsConnectionAvailable())
return false;
// Check for SSL and ignore it
ServicePointManager.ServerCertificateValidationCallback += delegate { return (true); };
var configurations = UpdateConfiguration.Download(UpdateConfigurationFileUri, Proxy);
var result = new UpdateResult(configurations, CurrentVersion,
IncludeAlpha, IncludeBeta);
if (!result.UpdatesFound)
return false;
_updateConfigurations = result.NewestConfigurations;
double updatePackageSize = 0;
foreach (var updateConfiguration in _updateConfigurations)
{
var newPackageSize = GetUpdatePackageSize(updateConfiguration.UpdatePackageUri);
if (newPackageSize == null)
throw new SizeCalculationException(_lp.PackageSizeCalculationExceptionText);
updatePackageSize += newPackageSize.Value;
_packageOperations.Add(new UpdateVersion(updateConfiguration.LiteralVersion),
updateConfiguration.Operations);
}
TotalSize = updatePackageSize;
return true;
});
}
As you can see I'm using Microsoft.Bcl.
Now in my other class I wrote this code in a normal void:
TaskEx.Run(async delegate
{
// ...
_updateAvailable = await _updateManager.SearchForUpdatesAsync();
MessageBox.Show("Test");
});
The problem I have is that it executes _updateAvailable = await _updateManager.SearchForUpdatesAsync(); and then it doesn't continue the thread, it just stops as if there is nothing after that call. Visual Studio also tells me this after a while: Thread ... exited with code 259, so something seems to be still alive.
I debugged through it to search for any exceptions that could maybe be swallowed, but nothing, everything works fine and it executes the return-statement.
And that is what I don't understand, I never see the MessageBox and/or no code beyond this line's being executed.
After I talked to some friends, they confirmed that this shouldn't be. Did I make a horrible mistake when implementing async-await?
Thanks in advance, that's actually all I can say about that, I got no more information, I appreciate any tips and help as far as it's possible.

The main issue that you're having is that you're unnecessarily wrapping your method in TaskEx.Run() and you are probably experiencing deadlock somewhere.
Your method signature is currently:
public async Task<bool> SearchForUpdatesAsync()
This means the following:
async --> Doesn't actually do anything, but provides a "heads up" that await might be called within this method and that this method can be used as a runnable Task.
Task --> This method returns a runnable task that can be run asynchronously on the threadpool
<bool> --> This method actually returns bool when awaited.
The await TaskEx.Run() is unnecessarily since this says run this method and then don't return until after a value is available. This is most likely causing a synchronization problem. Removing this construct will make your method work properly, however you'll notice that now you have no reason to even include the async operator or the Task<T> portion since the method is actually synchronous anyway. Usually you're only going to use async identifier on the method signature if you have methods that you are going to call await on them.
Instead you have two options.
Whenever you want to call SearchForUpdates() you can wrap this in a Task<bool>.Run() to run it asynchronously (or the Bcl equivalent)
Since you are using WinForms you might be better off using a BackgroundWorker and just calling this method within it.
Regarding using the async-await pattern I think that this is a great article to use to make sure you're following best practices: https://msdn.microsoft.com/en-us/magazine/jj991977.aspx

The best practice is to have async all the way through your layers, and then call await or less desirably .Wait() / .Result at the final use site.
Also, try to keep your UI calls separate from the backend work, since you can run into synchronicity/thread-context issue.
public class WinFormsCode
{
private async Task WinsFormCodeBehindMethodAsync()
{
var updatesAvailable = await _updateManager.SearchForUpdatesAsync();
MessageBox.Show("Updates Available: " + updatesAvailable.ToString());
}
private void WinsFormCodeBehindMethodSync()
{
var updatesAvailable = _updateManager.SearchForUpdatesAsync().Result;
MessageBox.Show("Updates Available: " + updatesAvailable.ToString());
}
}
public class UpdateManager
{
public async Task<bool> SearchForUpdatesAsync()
{
return true;
}
}

Running several infinite loops with async/await

I am developing android messanger app based on xamarin and .net 5 async/awaits.
In my app i have producer/consumer pattern for processing messages which is made on infinite loops.
for example ReadTcpClientAsync producer:
async Task ReadTcpClientAsync(CancellationToken cancellationToken)
{
cde.Signal();
while (!cancellationToken.IsCancellationRequested)
{
byte[] buffer = await atc.ReadAsync(cancellationToken);
// queue message...
}
}
or SendStatementsAsync consumer which deque messages and awaits WriteAsync
private async Task SendStatementsAsync(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
var nextItem = await _outputStatements.Take();
cancellationToken.ThrowIfCancellationRequested();
// misc ...
await atc.WriteAsync(call.Serialize());
}
}
and some consumers just await on Take calls
var update = await _inputUpdateStatements.Take();
this construction works pretty well on tests, but there is one method where i think i made a huge mistake.
this method intent to run entire client backend, starting 3 pro/con while (true) loops simultaneously.
here it is:
public async Task RunAsync()
{
_isRunning = true;
_progress.ProgressChanged += progress_ProgressChanged;
await InitMTProto(_scheme).ConfigureAwait(false); // init smth...
// various init stuf...
await atc.ConnectAsync().ConfigureAwait(false); // open connection async
// IS IT WRONG?
try
{
await Task.WhenAny(SendStatementsAsync(_cts.Token),
ReadTcpClientAsync(_cts.Token),
ProcessUpdateAsync(_cts.Token, _progress)).ConfigureAwait(false);
}
catch (OperationCanceledException oce)
{
}
catch (Exception ex)
{
}
}
Forget about android for now, think any UI (WinForm, WPF, etc) OnCreate method in UI context to call RunAsync
protected async override void OnCreate(Bundle bundle)
{
// start RA
await client.RunAsync()
// never gets here - BAD, but nonblock UI thread - good
Debug.WriteLine("nevar");
}
so, as you can see there is a problem. I can't do anything after RunAsync await call because it will never returns from Task.WhenAny(...). And i need perform status check there, but i need this pro/cons methods started, because my check wait on ManualResetEvent for it:
if (!cde.Wait(15000))
{
throw new TimeoutException("Init too long");
}
Also, my check is async too, and it works like a charm :)
public async Task<TLCombinatorInstance> PerformRpcCall(string combinatorName, params object[] pars)
{
// wait for init on cde ...
// prepare call ...
// Produce
ProduceOutput(call);
// wait for answer
return await _inputRpcAnswersStatements.Take();
}
I think i should use another approach for starting this infinite loops, but i already have async Task methods all the way - so i really have no idea what to do.
Any help please?

Ok, after a lot of reading (nothing found) and #svick's advice i decided to call this methods without "await" as separate Task.Run's.
Aso i decided to run it in ThreadPool.
My final code is:
try
{
/*await Task.WhenAny(SendStatementsAsync(_cts.Token),
ReadTcpClientAsync(_cts.Token),
ProcessUpdateAsync(_cts.Token, _progress)).ConfigureAwait(false);*/
Task.Run(() => SendStatementsAsync(_cts.Token)).ConfigureAwait(false);
Task.Run(() => ReadTcpClientAsync(_cts.Token)).ConfigureAwait(false);
Task.Run(() => ProcessUpdateAsync(_cts.Token, _progress)).ConfigureAwait(false);
Trace.WriteLineIf(clientSwitch.TraceInfo, "Worker threads started", "[Client.RunAsync]");
}
Everything works fine as expected..
i'm not sure what problems it will cause in exception handling, as i know they will be lost
Of course such calls produce warning
Because this call is not awaited, execution of the current method
continues before the call is completed. Consider applying the 'await'
operator to the result of the call.
which can be easily suppressed this way
// just save task into variable
var send = Task.Run(() => SendStatementsAsync(_cts.Token)).ConfigureAwait(false);
Also, if anyone know better solution i will be grateful to hear it.