We Keep Coding

WebAPI HTTP request not completing until queued work kicks off on background task

In my .Net 6 WebPI service, I am queueing work to a background task queue, very closely based on the example here, but I could post parts of my implementation if that would help:
https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-6.0&tabs=visual-studio#queued-background-tasks
I am running into unexpected behavior where control is not returned to the caller, even after the return Ok(..) completes in the controller. Instead the request only completes after the await Task.Delay(1000); line is reached on the queued work item. The request returns to the client as soon as this line is reached, and does not need to wait for the Delay to finish.
I'm guessing this is because of the await either starting a new async context, or someone un-sticking the async context of the original request. My intention is for the request to complete immediately after queuing the work item.
Any insight into what is happening here would be greatly appreciated.
Controller:
public async Task<ActionResult> Test()
{
var resultMessage = await _testQueue.DoIt();
return Ok(new { Message = resultMessage });
}
Queueing Work:
public TestAdapter(ITaskQueue taskQueue)
{
_taskQueue = taskQueue;
}
public async Task<string> DoIt()
{
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) =>
{
await Task.Delay(1000);
var y = 12;
});
return "cool";
}
IoC:
services.AddHostedService<QueueHostedService>();
services.AddSingleton<ITTaskQueue>(ctx =>
{
return new TaskQueue(MaxQueueCount);
});
TaskQueue:
private readonly Channel<BackgroundTaskContext> _queue;
public TaskQueue(int capacity)
{
var options = new BoundedChannelOptions(capacity)
{
FullMode = BoundedChannelFullMode.Wait
};
_queue = Channel.CreateBounded<BackgroundTaskContext>(options);
}
public async ValueTask QueueBackgroundWorkItemAsync(
Func<CancellationToken, ValueTask> workItem)
{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
await _queue.Writer.WriteAsync(new BackgroundTaskContext(workItem, ...));
}

Not sure what you expect here. I'm assuming you want the async method to return the cool in the api response. That's fine but because your also awaiting the async call with in DoIt(), then it pauses until QueueBackgroundWorkItemAsync finishes. You could remove the await and it will run and return as you expect.
I can't say that I'm a big fan of that design as you lose contact with it with exception of the cancellation token. Another thought would be to Send that work off to a console job or function app using message bus or even another http call.
Additional Notes:
Async can be complicated to explain because in reality, it wraps up code and executes on a thread of it's choosing. The await simulates the synchronous behavior.
await Task.Delay(1000); // Runs on it's own thread but still halts code execution for 1 second.
await _taskQueue.QueueBackgroundWorkItemAsync(async (_cancellationToken) // Waits for control to be returned from the code inside.
var resultMessage = await _testQueue.DoIt(); // Always waits for the code inside to complete.
If your wanting something to run without pausing code execution, you can either remove the await or add a Task.Run(() => { }); pattern. Is it a good idea is a whole other question. It also matters whether you need information back from the async method. If you don't await it then you'll get null back as it doesn't wait around for the answer to be computed.

This appears just to have been user error using the debugger. The debugger is switching to the background task thread and hitting breakpoints there before the response fully returns giving the appearance that control was not being returned to the client and being carried into the background task.
Even after adding some synchronous steps in QueueBackgroundWorkItemAsync and putting breakpoints on them, control was not immediately returned to the original http call. Only after I tried adding a long running task like await Task.Delay(1000); did enough time/ticks pass for the http response to return. I had conflated this with just the await somehow freeing up the original http context.

How to await the return response in order to assign to the .Text property of a label in C#

The program asks a person if he wants to check the last closing price of Bitcoin and whenever he pushes the button it should first say "Loading..." and wait until the price is received and assign it to the .Text property of the label. When I run the code in a console app the application closes before actually receiving the needed information (writes only "Loading..." ) but if I add ReadKey() the price information shows up. I guess something similar happens to the windows forms app which tries to assign a missing value to the text property of value, hence the program crashes after displaying "Loading...".
public static async Task<string> ApiCall(string apikey)
{
RestClient client = new RestClient("https://api.polygon.io/v2/aggs/ticker/X:BTCUSD/prev?adjusted=true&apiKey=");//write your api key
RestRequest request = new RestRequest($"?api-key={apikey}", Method.GET);
IRestResponse response = await client.ExecuteAsync(request);
return response.Content;
}
public static async Task<string> apiReceiver(string last_closed)
{
Task<string> apiCallTask = getAPI.ApiCall("[apikey]");
string result = apiCallTask.Result;
dynamic array = JsonConvert.DeserializeObject(result);
last_closed = array.results[0].c;
return last_closed;
}
public static async Task dataWait(Label lab, string last_closed)
{
lab.Text = "Loading info ...";
lab.Text = await apiReceiver(last_closed);
}
private async void button1_Click(object sender, EventArgs e)
{
string last_closed = "";
await getAPI.dataWait(label1, last_closed);
}

Why aren't you awaiting getAPI.ApiCall("[apikey]");? Using .Result before the task is completed, will result in a deadlock. The winforms has a SynchronizationContext set on that thread. Meaning that after the await, you're back on the UI thread and allowed and therefor able to modify UI-controls.
When you use .Result on a task, if it's not finished, it will wait there (block the thread). The problem is that when the task is ready, it will be posted on the UI thread, but never be executed, because the thread still blocked.
The difference between winforms and console. The console hasn't got a SynchronizationContext set, so the rest of the method (after the await) is posted on the threadpool. You're allowed to call Console.Writeline on anythread.
So use await here, so the thread isn't blocked.
public static async Task<string> apiReceiver(string last_closed)
{
string result = await getAPI.ApiCall("[apikey]");
dynamic array = JsonConvert.DeserializeObject(result);
last_closed = array.results[0].c;
return last_closed;
}
Here's some information:
source
However, if that code is run in a UI Application, for example when a button is clicked like in the following example:
Then the application will freeze and stop working, we have a deadlock. Of course, users of our library will complain because it makes the application unresponsive.
More about SynchronizationContext read codeproject.com Understanding-the-SynchronizationContext

Understanding async await instruction during a HttpClient request

Let's say I have a solution composed of 2 projects. An old WinForm classic project. In this old project, I have a login window. On click "OK" of this login window, I start an event that will call a REST API. Both applications start at the same time in debug mode.
Somewhere in my code, I have this code:
public async Task<User> Login(string username, string password)
{
HttpResponseMessage response = await Client.GetAsync($"api/Login?login={username}&password={password}");
if (response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
return new User();
response.EnsureSuccessStatusCode();
var userDto = await response.Content.ReadAsAsync<UserDto>();
var user = userDto.ToUser();
return user;
}
On the first line when I call the Client.GetAsync I call my API. In my API I properly receive the call and I properly return an Ok with my User object or I return another code. It works. My API works. But then nothing. My client never continues. It seems Client.GetSync waits for something. I never go on the next step where I evaluate the StatusCode.
public async Task<User> Login(string username, string password)
{
HttpResponseMessage response = Client.GetAsync($"api/Login?login={username}&password={password}").Result;
if (response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
return new User();
response.EnsureSuccessStatusCode();
var userDto = await response.Content.ReadAsAsync<UserDto>();
var user = userDto.ToUser();
return user;
}
Same code without the await I have no problem. My code run till the next step. Proof my API is not the problem.
This is clear it is an issue related to await/async. I must do something wrong but what? Can you help me? Is it related to the debugger?
For more information here is a picture of my code before
And after I click for the next step. Note my call stack is empty and code is still running.
As requested here is the code where I call the login. I just added the Async word before the Sub and changed the _authService.Login(username, password).Result by await _authService.Login(username, password)
I works now.
Private Async Sub ButLogin_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles butLogin.Click
DataProxies.SetToken()
Dim _authService As IAuthenticationService = New AuthenticationService()
Dim username As String = txtLogin.Text
Dim password As SecureString = New NetworkCredential(String.Empty, txtPwd.Text).SecurePassword
Dim auth As Tuple(Of Boolean, User) = Await _authService.Login(username, password)
If (auth.Item1) Then
Dim user As User = auth.Item2
Name = $"{user.FirstName} {user.LastName}"
ApiInformations.ApiToken = user.SessionToken
End If
End Sub

I just added the Async word before the Sub and changed the _authService.Login(username, password).Result by await _authService.Login(username, password)
The general guidance is "Don't block on async code". This is one of the asynchronous programming best practices.
Blocking on asynchronous code is bad because await works by capturing a "context" by default, and resumes executing the async method in that context. One context is the UI context, which causes the async method to resume executing on the UI thread.
So the deadlock you were seeing was caused by blocking the UI thread. The code was calling the async method and then blocking the UI thread until that async method completed. However, the await in that async method captured the UI context, so it was waiting for the UI thread to be free before it could complete. The UI thread was waiting for the async method and the async method was waiting for the UI thread: deadlock.
The reason your fix worked is that the UI thread is no longer blocked waiting for the async method, so there's no more deadlock.

async / await and Fiddler Demo

Right to the Point
Why is it if I try running the code below in Visual Studio, and have a break point on the “string urlContents = await getStringTask;” line, I don’t see the call to msdn.microsoft.com until after I execute that line.
I would expect the call to msdn.microsoft.com to occur after processing executes line: Task getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
That's the point of an async method, correct? To kick off tasks that might take a while to run (client.GetStringAsync) so independent work (DoIndependentWork();) can continue while the longer task do their thing.
I am confused by my results. Can someone explain why this occurred.
I am thinking it is because I am running everything on my development machine.
public partial class MainWindow : Window
{
// Mark the event handler with async so you can use await in it.
private async void StartButton_Click(object sender, RoutedEventArgs e)
{
int contentLength = await AccessTheWebAsync();
resultsTextBox.Text +=
String.Format("\r\nLength of the downloaded string: {0}.\r\n", contentLength);
}
// Three things to note in the signature:
// - The method has an async modifier.
// - The return type is Task or Task<T>. (See "Return Types" section.)
// Here, it is Task<int> because the return statement returns an integer.
// - The method name ends in "Async."
async Task<int> AccessTheWebAsync()
{
// You need to add a reference to System.Net.Http to declare client.
HttpClient client = new HttpClient();
// GetStringAsync returns a Task<string>. That means that when you await the
// task you'll get a string (urlContents).
Task<string> getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
// You can do work here that doesn't rely on the string from GetStringAsync.
DoIndependentWork();
// The await operator suspends AccessTheWebAsync.
// - AccessTheWebAsync can't continue until getStringTask is complete.
// - Meanwhile, control returns to the caller of AccessTheWebAsync.
// - Control resumes here when getStringTask is complete.
// - The await operator then retrieves the string result from getStringTask.
string urlContents = await getStringTask;
// The return statement specifies an integer result.
// Any methods that are awaiting AccessTheWebAsync retrieve the length value.
return urlContents.Length;
}
void DoIndependentWork()
{
resultsTextBox.Text += "Working . . . . . . .\r\n";
}
More History and Why are I am asking the questions above
My boss and I were discussing the async / await commands after I read: https://msdn.microsoft.com/library/hh191443(vs.110).aspx (which is where I got the code from above.) There was some debate over how the commands worked so my boss asked me to do some investigation. He suggested running the code above and monitoring it against Fiddler. When I did that I didn’t get the result I expected. I would have expected the call to msdn.microsoft.com to occur after line (Task getStringTask = client.GetStringAsync("http://msdn.microsoft.com");). However, it didn't occur
So, I continued to do research on the commands. Which prompted me to create another demo. I set up the new demo like the one above. However, this demo made a call to a database using an async method, then did some independent work (just like above) and then used the await command (just like above). I monitored this demo using SQL Profiler. This demo gave me the results I would have expected. It proved to me that the async command kicks of the database call before it ran the IndependentWork method.
Here is the code just in case I didn’t explain it clearly
public class HomeController : Controller
{
public Task<string> Index()
{
var returnValue = MacroService();
return returnValue;
}
public async Task<string> MacroService()
{
Task<string> getStringTask = MicroDataService("MicroDataServiceCall");
string string2 = IndependentWork("IndependentWorkCall");
string stringTask1 = await getStringTask;
return $"{stringTask1}, {string2}";
}
public async Task<string> MicroDataService(string parm)
{
string connectionString = ConfigurationManager.ConnectionStrings["xzz"].ConnectionString;
var conn = new SqlConnection(connectionString);
var command = new SqlCommand("sproc", conn);
command.CommandType = CommandType.StoredProcedure;
SqlParameter param = new SqlParameter("#ID", SqlDbType.VarChar);
param.Direction = ParameterDirection.Input;
param.Value = "00000";
command.Parameters.Add(param);
conn.Open();
await command.ExecuteNonQueryAsync();
conn.Close();
return parm;
}
public string IndependentWork(string parm)
{
return parm;
}
So, I want to understand why I don’t get the same result from the original demo since I know my boss is going to ask.
I am thinking it is because I am running everything on my development machine for the first demo. In the second demo, I am actually using a database server so the second demo kicks off the database processing on that machine.
Let me know. Thanks!!!
Sorry if my description / expectation is not clear. Let me try to be clearer.
My expected result for the first demo was that I would see a call in Fiddler to http://msdn.microsoft.com after Task getStringTask = client.GetStringAsync("http://msdn.microsoft.com"); executed. However, I did not see a call in Fiddler until after line string urlContents = await getStringTask; executed.
Therefore, the first demo did not give me the result I expected.
However, the second demo gave me the results I expected. Which was that I saw a call to the database in SQL Profiler after line: Task getStringTask = MicroDataService("MicroDataServiceCall"); executed.
So, I am trying to understand why I got different behavior. I would have thought that demo1 would have kicked off the async method. I would have seen an entry in Fiddler. Processing would continue in the IndependentWork method. Then continue after the result were returned from the async call. This is what is happening in the second demo.

My guess is that the behavior you observe is because the debugger stops execution of the application before the request is made.
When you hit a breakpoint in a debugger, the whole application stops, it doesn't stop just the current thread. And if the async method you called did not make the request before it returned the Task, then you will not see it in Fiddler.
This probably also explains why you don't observe this with other methods: they do manage to make the request before returning.

Why can I not await the return value from my async method?

Situation
I have a raspberry pi set up as a server taking json via HTTP as input.
The "API" allows to set leds connected to the pi. That all works, I can send requests from the browser and everything is great.
It takes a while for the response to arrive. That's why I want to communicate asynchrounously.
I found this on msdn that explains how it's done.
// Three things to note in the signature:
// - The method has an async modifier.
// - The return type is Task or Task<T>. (See "Return Types" section.)
// Here, it is Task<int> because the return statement returns an integer.
// - The method name ends in "Async."
async Task<int> AccessTheWebAsync()
{
// You need to add a reference to System.Net.Http to declare client.
HttpClient client = new HttpClient();
// GetStringAsync returns a Task<string>. That means that when you await the
// task you'll get a string (urlContents).
Task<string> getStringTask = client.GetStringAsync("http://msdn.microsoft.com");
// You can do work here that doesn't rely on the string from GetStringAsync.
DoIndependentWork();
// The await operator suspends AccessTheWebAsync.
// - AccessTheWebAsync can't continue until getStringTask is complete.
// - Meanwhile, control returns to the caller of AccessTheWebAsync.
// - Control resumes here when getStringTask is complete.
// - The await operator then retrieves the string result from getStringTask.
string urlContents = await getStringTask;
// The return statement specifies an integer result.
// Any methods that are awaiting AccessTheWebAsync retrieve the length value.
return urlContents.Length;
}
For the top level overview, here's how my Main method looks like (it doesn't compile):
var pi1 = new RaspberryPi(#"http://192.168.0.100:8080"); // specify IP
var led = new Led(255, 100, 180); // r, g, b values wrapped in an Led object
Led result = await pi1.setLedAsync(2, led); // FAIL // what should be an async POST, awaiting the response
I hope that makes sense.
The Led class is just a data object holding 3 bytes for the 3 channels and some conversion methods to and from json.
The setLedAsync method:
async public Task<Led> setLedAsync(uint n, Led led)
{
var client = new HttpClient();
client.BaseAddress = _uri;
var content = new StringContent(led.ToJson(), Encoding.UTF8, "application/json");
Task<HttpResponseMessage> response = client.PutAsync("/led/" + n, content);
HttpResponseMessage responseMessage = await response;
string json = await responseMessage.Content.ReadAsStringAsync();
return new Led(json);
}
Error
This line is where I get an error for using await:
Led result = await pi1.setLedAsync(2, led);
await can only be used in an async method.
Questions
Why do I get this error? The last comment line in the example code
// Any methods that are awaiting AccessTheWebAsync retrieve the length value.
makes me think that this is how it should be done. As I understand it, the await basically unwrapps the Task<T> into a T.
If I do not use await, I get a type missmatch, because the method returns Task<Led>, not Led.
What's confusing for me is to understand the difference between the example and my situation. I have to use await twice in my async method:
HttpResponseMessage responseMessage = await response;
string json = await responseMessage.Content.ReadAsStringAsync();
The thing is that I have to deal with this HttpResponseMessage as a middleman. I suspect that I'm prematurely giving up the asynchronousity with this second await somehow (if that makes any sense) I think this is the origin of the problem, but I'm not sure how to solve it.
Edit
I wrapped the function call in an asyn method, which allows to compile the code.
But it's not asynchronous. I added a delay on the server side to test this.
class Program
{
static void Main(string[] args)
{
var prog = new Program();
Console.WriteLine("before init");
prog.init();
Console.WriteLine("after init"); // not happening before the response arrives
Console.Read();
}
private async void init()
{
var pi1 = new RaspberryPi(#"http://192.168.0.100:8080"); // specify IP
var led = new Led(255, 0, 0); // r, g, b values wrapped in an Led object
Console.WriteLine("before await");
Led result = await pi1.setLedAsync(2, led); // what should be an async POST, awaiting the response
Console.WriteLine("after await");
}
}
None of the "after" messages are written to the console before the response from the request arrives.

You get the error because an asynchronous wait -- an await -- implies that the method doing the awaiting is itself asynchronous. I think you do not understand what await means. Await does not mean synchronously block until the result is available -- that is the opposite of what it means. Await means return immediately so my caller can do important work without waiting for this result; schedule the remainder of this method at some time in the future when the result is available. Await is an asynchronous wait. When you await a letter to arrive in the mail you do not sit by the door doing nothing until it arrives; you do other work asynchronously and then resume the task of reading your mail at some time after the letter arrives.
You say that the method -- the method doing the awaiting, not the method returning the task being awaited -- is Main. If this is a console app then you cannot make Main asynchronous because when Main returns the program ends. Again, internalize this: an await is just a return from the perspective of the current method. The current method will be called again later in the future and will pick up where it left off, but when Main returns there is no future. So you cannot make Main async.
You note that async turns a task of T into a T, which is correct, but it does so asynchronously. So your principal choices here are here are:
turn the task of T into a T synchronously from Main
write some sort of app other than a Console app; say, a winforms or WPF app, which does not terminate until you tell it to