In my .Net 6 WebPI service, I am queueing work to a background task queue, very closely based on the example here, but I could post parts of my implementation if that would help:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-6.0&tabs=visual-studio#queued-background-tasks
I am running into unexpected behavior where control is not returned to the caller, even after the return Ok(..) completes in the controller. Instead the request only completes after the await Task.Delay(1000); line is reached on the queued work item. The request returns to the client as soon as this line is reached, and does not need to wait for the Delay to finish.
I'm guessing this is because of the await either starting a new async context, or someone un-sticking the async context of the original request. My intention is for the request to complete immediately after queuing the work item.
Any insight into what is happening here would be greatly appreciated.
Controller:
public async Task<ActionResult> Test()
{
var resultMessage = await _testQueue.DoIt();
return Ok(new { Message = resultMessage });
}
Queueing Work:
public TestAdapter(ITaskQueue taskQueue)
{
_taskQueue = taskQueue;
}
public async Task<string> DoIt()
{
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) =>
{
await Task.Delay(1000);
var y = 12;
});
return "cool";
}
IoC:
services.AddHostedService<QueueHostedService>();
services.AddSingleton<ITTaskQueue>(ctx =>
{
return new TaskQueue(MaxQueueCount);
});
TaskQueue:
private readonly Channel<BackgroundTaskContext> _queue;
public TaskQueue(int capacity)
{
var options = new BoundedChannelOptions(capacity)
{
FullMode = BoundedChannelFullMode.Wait
};
_queue = Channel.CreateBounded<BackgroundTaskContext>(options);
}
public async ValueTask QueueBackgroundWorkItemAsync(
Func<CancellationToken, ValueTask> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
await _queue.Writer.WriteAsync(new BackgroundTaskContext(workItem, ...));
}
Not sure what you expect here. I'm assuming you want the async method to return the cool in the api response. That's fine but because your also awaiting the async call with in DoIt(), then it pauses until QueueBackgroundWorkItemAsync finishes. You could remove the await and it will run and return as you expect.
I can't say that I'm a big fan of that design as you lose contact with it with exception of the cancellation token. Another thought would be to Send that work off to a console job or function app using message bus or even another http call.
Additional Notes:
Async can be complicated to explain because in reality, it wraps up code and executes on a thread of it's choosing. The await simulates the synchronous behavior.
await Task.Delay(1000); // Runs on it's own thread but still halts code execution for 1 second.
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) // Waits for control to be returned from the code inside.
var resultMessage = await _testQueue.DoIt(); // Always waits for the code inside to complete.
If your wanting something to run without pausing code execution, you can either remove the await or add a Task.Run(() => { }); pattern. Is it a good idea is a whole other question. It also matters whether you need information back from the async method. If you don't await it then you'll get null back as it doesn't wait around for the answer to be computed.
This appears just to have been user error using the debugger. The debugger is switching to the background task thread and hitting breakpoints there before the response fully returns giving the appearance that control was not being returned to the client and being carried into the background task.
Even after adding some synchronous steps in QueueBackgroundWorkItemAsync and putting breakpoints on them, control was not immediately returned to the original http call. Only after I tried adding a long running task like await Task.Delay(1000); did enough time/ticks pass for the http response to return. I had conflated this with just the await somehow freeing up the original http context.
I'm newbie in async. And making WPF app for scraping and API calls purposes. WPF's UI is needed only for service monitoring and settings control, so all services will run simultaneously in the background. And most of them is doing similar work.
For this one I need to implement strategy like this:
Start worker on threadpool
Worker must send request and process response from Website
2.1 If response processed and some new data appeared - raise an event
2.2 If request failed - handle an error
2.3 If there are many error percent for last x requests - stop worker
No matter the last request completed/failed/running we must send another request
Another request should be sent not earlier than the set delay but should not exceed the delay too much (as far as possible).
private _workTask;
private List<ScrapeParameters> _scrapeParams = new();
public event EventHandler<ScrapedEventArgs>? NewDataScraped;
//Can I run Worker like this?
public void RunWorker()
{
if(_workTask.IsCompleted)
_workTask = WorkAsync(_token)
}
private async Task WorkAsync(CancellationToken cancelToken)
{
List<Task> processTasks = new();
while(true)
{
if(cancelToken.IsCancellationRequested) return;
//Delay could be from 0.5 second to any value
var delayTask = Task.Delay(WorkerDelay);
var completedTasks = processTasks.Where(t => t.IsCompleted)
var setToHandle = new HashSet<Task>(completedTasks);
foreach(var task in setToHandle)
{
//Theoretical logic to handle errors and completion
if(task.IsFaulted)
HandleFaultedTask(task);
else
CountCompleted();
processTasks.Remove(task);
}
//Theoretical logic to obtain the desired parameters.
var currParameters = GetParameters();
processTasks.Add(ProcessAsync(currParameters, cancelToken));
await delayTask;
}
}
//This method usually takes around 2-4 seconds
private async Task ProcessAsync(ScrapeParameters parameters CancellationToken cancelToken)
{
//Some work with http requests
var response = await Client.GetAsync(parameters.ToUri());
...
//Processing response
...
if(newData != null)
NewDataScraped?.Invoke(new(newData));
}
Does my implementation matches the TAP pattern?
Especially I would like to focus on RunWorker() and setToHandle
I am using the C# client library for SendGrid version 9.27.0.
The only way I can get it to send an email is by adding Wait() to end of my method.
I understand the await operator is a promise to return back to the point in code after the asynchronous method is finished.
But why do I need to add Wait()? Isn't that converting the asynchronous method to synchronous? If so, what's the point in making it async?
Program.cs
static void Main(string[] args) {
//var customerImport = new CustomerImport();
//customerImport.DoImport();
var mailClient = new MailClient();
var recipients = new List<string>();
recipients.Add("test#lobbycentral.com");
//Never sends an email
var response = mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false);
//Will send an email
mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false).Wait();
}
MailClient.cs
public async Task SendMail(string emailFrom, List<string> emailTo, string subject, string body, bool isPlainText) {
try {
var apiKey = Utils.GetConfigValue("sendgridAPIKey");
var emails = new List<EmailAddress>();
foreach (string email in emailTo) {
emails.Add(new EmailAddress(email));
}
var plainTextContent = "";
var htmlContent = "";
if (!isPlainText) {
htmlContent = body;
} else {
plainTextContent = body;
}
var message = MailHelper.CreateSingleEmailToMultipleRecipients(new EmailAddress(emailFrom, "LobbyCentral"), emails, subject, plainTextContent, htmlContent);
//if (metaData != null)
// message.AddCustomArgs(metaData);
foreach (string filename in FileAttachments) {
if (System.IO.File.Exists(filename)) {
using (var filestream = System.IO.File.OpenRead(filename)) {
await message.AddAttachmentAsync(filename, filestream);
}
}
}
foreach (PlainTextAttachmentM plainTextM in PlainTextAttachments) {
byte[] byteData = Encoding.ASCII.GetBytes(plainTextM.Content);
var attachment = new Attachment();
attachment.Content = Convert.ToBase64String(byteData);
attachment.Filename = plainTextM.AttachmentFilename;
attachment.Type = "txt/plain";
attachment.Disposition = "attachment";
message.AddAttachment(attachment);
}
var client = new SendGridClient(apiKey);
var response = await client.SendEmailAsync(message);
if (response.IsSuccessStatusCode) {
if (DeleteAttachmentsAfterSend && FileAttachments.Count > 0) {
foreach (string filename in FileAttachments) {
if (System.IO.File.Exists(filename)) {
System.IO.File.Delete(filename);
}
}
}
} else {
Utils.DebugPrint("error sending email");
}
} catch (Exception ex) {
throw new Exception(string.Format("{0}.{1}: {2} {3}", System.Reflection.MethodBase.GetCurrentMethod().DeclaringType.FullName, System.Reflection.MethodBase.GetCurrentMethod().Name, ex.Message, ex.StackTrace));
}
}
Calling mailClient.SendMail starts a Task, but doesn't wait for its completion.
If that's the last instruction of you program, the program simply ends before the tasks can finish.
You want your last instructions to start and wait for the task's completion. You can do that by using either of the following.
Make your Main async. (That's what I would personally do.)
// Signature changed to async Task instead of void.
static async Task Main(string[] args) {
// (...)
// Added await. Removed Wait.
await mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false);
}
Use Wait like you're doing.
static void Main(string[] args) {
// (...)
var task = mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false);
task.Wait();
}
Question : But why do I need to add Wait()? Isn't that converting the asynchronous method to synchronous? If so, what's the point in making it async?
Answer : It's not mandatory to Wait() a Task. What's the point in making it async? It gives you two great benefits.
You can return Task, which means it's awaitable, and which means again, you can choose either wait it or just forget about it and let the task scheduled and picked up by a threadPool thread instead of synchronously running it in the same thread.
You can use async/await pattern, which means you can gracefully avoid blocking the current thread
Below code obviously will block the current thread.
static void Main(string[] args)
{
.
.
//Will send an email
mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false)
.Wait(); // <- block the current thread until the task completes.
}
If SendMail is something you can fire&forget in your real codebase, you can just get rid of .Wait() and move on without checking the Task state. But your application should be up and running for a certain, enough time to finish the scheduled task. If it's not, you'd better think of using async/await pattern instead of blocking your valuable thread by using .Wait().
static async Task Main(string[] args)
{
.
.
//Will send an email and get to know when the task is done.
await mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false);
}
You need to await it in this console app because the very last thing that the Main() does before it returns (and the app exits) is send the mail. I'd say there is slim to no chance at all that the mail sending task will complete before the app exits; it involves a lot of IO, whereas it will take nanoseconds for the app to quit, taking the incomplete Task with it
Anything you do to make it wait long enough will suffice; turning it synchronous with a call to Wait, making the Main as async Task Main(...), even adding a Console.ReadLine will all mean the mail sending will have time to complete ..
.. though I question why a console app that only does one thing then exits even needs to be async anything - it simply has no other jobs to do while it waits for this IO to complete so there seems little point using any asyncronous facilities
TL;DR
Update your code to use async, await, and Task's throughout the whole call chain of your method, including your Main method, otherwise your program won't wait for the asynchronous Task's to complete and will not run all your code.
Your code should look like this:
static async Task Main(string[] args) {
//var customerImport = new CustomerImport();
//customerImport.DoImport();
var mailClient = new MailClient();
var recipients = new List<string>();
recipients.Add("test#lobbycentral.com");
//Never sends an email
var response = await mailClient.SendMail("noreply#lobbycentral.com", recipients, "Test email", "This is a test of the new SG client", false);
}
In C# .NET the Task class represents an operation that usually is performed asynchronously on another thread. When you use the await keyword or the .Wait() method, the program will wait for the operation to finish and then continue executing the next statements.
The difference between using the await keyword and the .Wait() method is that the await keyword will release the current thread so other work can be done on the current thread, while the .Wait() method will block the current thread, leaving the thread unutilized until the task is complete.
When the task is complete, when using the async keyword, the program will find an available thread (not necessarily the same thread as you started on) and will continue to run the instructions subsequent to your await. The .Wait() blocks the current thread, so it'll keep using the current thread when the task is complete.
To use await you need to make your method async. Sometimes, you don't need to wait on the Task and you can simply return it, in that case you don't need to mark the method as async although it is still recommended for easier debugging.
However, once you start using async and Task's, you need to do so all the way up to the call chain.
This sample using .Wait needs to be converted to using async/await.
public static class Program
{
public static void Main(string[] args)
{
DoSomeWork();
}
public static void DoSomeWork()
{
DoFileOperations();
}
public static void DoFileOperations()
{
File.ReadAllTextAsync("/path/to/file").Wait();
}
}
Making this program asynchronous looks like this:
public static class Program
{
public static async Task Main(string[] args)
{
await DoSomeWork();
}
public static Task DoSomeWork()
{
// you can return a Task without awaiting it, as a result no need for async keyword.
return DoFileOperations();
// But the following is recommended for easier debugging, at cost of a negligible performance hit. (Also need to mark method is async for this)
// return await DoFileOperations();
}
public static async Task DoFileOperations()
{
await File.ReadAllTextAsync("/path/to/file");
}
}
For a console program, you need to update the Main method signature to use async Task or Task for this to work. If you don't, the Task will start running but the program will exit immediately and not wait for the Task to be complete.
(Non-console applications will provide other APIs to do async/await.)
In your application, your Main method does not have the async Task or Task return signature, and you're not awaiting the Task returned from the mailClient.SendMail method.
What likely is happening inside of your mailClient.SendMail method is
The first asynchronous Task is started, created by the message.AddAttachmentAsync
This Task runs on a separate thread, meanwhile the current thread goes back to run the code after mailClient.SendMail in the Main method
The Main method finishes and the console application stops running.
At this point, the Task from message.AddAttachmentAsync is likely finished already, but the subsequent code is never run, including the client.SendEmailAsync method, because the Task from mailClient.SendMail wasn't awaited.
There's a lot more nuance to async, await, and Task's, but those details aren't too relevant to your questions, so if you're interested, here's more info on Asynchronous programming with async and await from the Microsoft docs.
I'd also recommend reading David Fowler's guidance on async/await which lists some pitfalls and best practices.
My MainPage code-behind:
private void Button_Clicked(object sender, EventArgs e)
{
Query().Wait();
App.Current.MainPage = new Categories();
}
public async Task Query()
{
restaurantsClient = (App.Current as App).restaurantsClient;
try
{
var restaurantsNames = await restaurantsClient.GetCatalogsAsync(1);
}
catch (Exception ex)
{
var x = 0;
}
}
I tried this code too but didn't work, happens the same problem:
async static Task GetRequest(String URL)
{
using (HttpClient client = new HttpClient())
{
// As soon as I try to step over (or even into) this next line, it crashes.
using (HttpResponseMessage response = await client.GetAsync(URL))
{
using (HttpContent content = response.Content)
{
string data = await content.ReadAsStringAsync();
Console.WriteLine(data);
}
}
}
}
Rest-API in C#:
var response_ = await client_.SendAsync(request_, System.Net.Http.HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false);
When the project reach this line it just die without showing any error.
If I do it in the browser it works.
The API is running locally in my PC (Im using Conveyor to expose the API).
Ok I make a video to see better what Im talking about:
https://youtu.be/ONKTipPsEXI
As you can see after I click Next Step in response line stop executing the rest of the code.
That's because you're using .Wait() of a Task on the UI thread (Button.Clicked event is handled on the UI thread) causing a deadlock. The task is waiting for the UI thread to give it control and the UI thread is waiting for the task to complete. The solution to this is to add async modifier to your event handler and use await Query() instead of Query().Wait().
private async void Button_Clicked(object sender, EventArgs e)
{
await Query();
App.Current.MainPage = new Categories();
}
I'd also recommend reading this article by Stephen Cleary about this matter. Moreover, he's made a fabulous series (A Tour of Task) about C# tasks in general.
UPDATE:
After OP's question update and discussion in this answer's comments; he thinks that there's a problem because he can reach the end of GetCatalogsAsync(int) before the end of GetCatalogsAsync(int, CancellationToken). That's completely natural and is to be expected. Solution:
public async System.Threading.Tasks.Task<CatalogsInCategory> GetCatalogsAsync(int id)
{
return await GetCatalogsAsync(id, System.Threading.CancellationToken.None);
}
I have a console program which sends async HTTP requests to an external web API. (HttpClient.GetAsync());)
These tasks can take several minutes to complete - during which I'd like to be able to show to the user that the app is still running - for example by sending Console.WriteLine("I ain't dead - yet") every 10 seconds.
I am not sure how to do it right, without the risk of hiding exceptions, introducing deadlocks etc.
I am aware of the IProgress<T>, however I don't know whether I can introduce it in this case. I am await a single async call which does not report progress. (It's essentially an SDK which calls httpClient GetAsync() method
Also:
I cannot set the GUI to 'InProgress', because there is no GUI, its a console app - and it seems to the user as if it stopped working if I don't send an update message every now and then.
Current idea:
try
{
var task = httpClient.GetAsync(uri); //actually this is an SDK method call (which I cannot control and which does not report progress itself)
while (!task.IsCompleted)
{
await Task.Delay(1000 * 10);
this.Logger.Log(Verbosity.Verbose, "Waiting for reply...");
}
onSuccessCallback(task.Result);
}
catch (Exception ex)
{
if (onErrorCallback == null)
{
throw this.Logger.Error(this.GetProperException(ex, caller));
}
this.Logger.Log(Verbosity.Error, $"An error when executing command [{action?.Command}] on {typeof(T).Name}", ex);
onErrorCallback(this.GetProperException(ex, caller));
}
Let me tidy this code up a bit for you
async Task Main()
{
var reporter = new ConsoleProgress();
var result = await WeatherWaxProgressWrapper(() => GetAsync("foo"), reporter);
Console.WriteLine(result);
}
public async Task<int> GetAsync(string uri)
{
await Task.Delay(TimeSpan.FromSeconds(10));
return 1;
}
public async Task<T> WeatherWaxProgressWrapper<T>(Func<Task<T>> method, System.IProgress<string> progress)
{
var task = method();
while(!task.IsCompleted && !task.IsCanceled && !task.IsFaulted)
{
await Task.WhenAny(task, Task.Delay(1000));
progress.Report("I ain't dead");
}
return await task;
}
public class ConsoleProgress : System.IProgress<string>
{
public void Report(string value)
{
Console.WriteLine(value);
}
}
You could have a never-ending Task as a beacon that signals every 10 sec, and cancel it after the completion of the long running I/O operation:
var beaconCts = new CancellationTokenSource();
var beaconTask = Task.Run(async () =>
{
while (true)
{
await Task.Delay(TimeSpan.FromSeconds(10), beaconCts.Token);
Console.WriteLine("Still going...");
}
});
await LongRunningOperationAsync();
beaconCts.Cancel();
You are looking for System.Progress<T>, a wonderful implementation of IProgress.
https://learn.microsoft.com/en-us/dotnet/api/system.progress-1
You create an object of this class on the "UI thread" or the main thread in your case, and it captures the SynchronizationContext for you. Pass it to your worker thread and every call to Report will be executed on the captured thread, you don't have to worry about anything.
Very useful in WPF or WinForms applications.