We Keep Coding

C# launching task with non-async function inside

Basic overview: program should launch task to parse some array of data and occasionally enqueue tasks to process it one at a time. Test rig have a button an two labels to display debug info. TaskQueue is a class for SemaphoreSlim from this thread
Dispatcher dispath = Application.Current.Dispatcher;
async void Test_Click(s, e)
{
TaskQueue queue = new TaskQueue();
// Blocks thread if SimulateParse does not have await inside
await SimulateParse(queue);
//await Task.Run(() => SimulateParse(queue));
lblStatus2.Content = string.Format("Awaiting queue"));
await queue.WaitAsync(); //this is just SemaphoreSlim.WaitAsync()
lblStatus.Content = string.Format("Ready"));
lblStatus2.Content = string.Format("Ready"));
MessageBox.Show("Ok");
}
async Task SimulateParse(TaskQueue queue)
{
Random rnd = new Random();
int counter = 0; // representing some piece of data
for(int i = 0; i < 500; i++)
{
dispatch.Invoke(() => lblStatus2.Content = string.Format("Check {0}", ++counter));
Thread.Sleep(25); //no await variant
//await Task.Delay(25);
// if some condition matched - queue work
if (rnd.Next(1, 11) < 2)
{
// Blocks thread even though Enqueue() has await inside
queue.Enqueue(SimulateWork, counter);
//Task.Run(() => queue.Enqueue(SimulateWork, counter));
}
}
}
async Task SimulateWork(object par)
{
dispatch.Invoke(() => lblStatus.Content = string.Format("Working with {0}", par));
Thread.Sleep(400); //no await variant
//await Task.Delay(400);
}
It seems, that it works only if launched task have await inside itself, i.e. if you trying to launch task without await inside it, it will block current thread.
This rig will work as intended, if commented lines are used, but it looks like excessive amount of calls, also, real versions of SimulateParse and SimulateWork does not need to await anything. Main question is - what is the optimal way to launch task with non-async function inside of it? Do i just need to encase them in a Task.Run() like in commented rows?

TaskQueue is used here to run task one by one
It will run them one at a time, yes. SemaphoreSlim does have an implicit queue, but it's not strictly a FIFO-queue. Most synchronization primitives have a mostly-but-not-quite-FIFO implementation, which is Close Enough. This is because they are synchronization primitives, and not queues.
If you want an actual queue (i.e., with guaranteed FIFO order), then you should use a queue, such as TPL Dataflow or System.Threading.Channels.
if you trying to launch task without await inside it, it will block current thread.
All async methods begin executing on the current thread, as described on my blog. async does not mean "run on a different thread". If you want to run a method on a thread pool thread, then wrap that method call in Task.Run. That's a much cleaner solution than sprinkling Task.Delay throughout, and it's more efficient, too (no delays).

Is parallel asynchronous execution where a thread sleeps using multiple threads?

This is the code that I wrote to better understand asynchronous methods. I knew that an asynchronous method is not the same as multithreading, but it does not seem so in this particular scenario:
class Program
{
static void Main(string[] args)
{
Thread.CurrentThread.CurrentCulture = new System.Globalization.CultureInfo("en-US");
//the line above just makes sure that the console output uses . to represent doubles instead of ,
ExecuteAsync();
Console.ReadLine();
}
private static async Task ParallelAsyncMethod() //this is the method where async parallel execution is taking place
{
List<Task<string>> tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Run(() => DownloadWebsite()));
}
var strings = await Task.WhenAll(tasks);
foreach (var str in strings)
{
Console.WriteLine(str);
}
}
private static string DownloadWebsite() //Imitating a website download
{
Thread.Sleep(1500); //making the thread sleep for 1500 miliseconds before returning
return "Download finished";
}
private static async void ExecuteAsync()
{
var watch = Stopwatch.StartNew();
await ParallelAsyncMethod();
watch.Stop();
Console.WriteLine($"It took the machine {watch.ElapsedMilliseconds} milliseconds" +
$" or {Convert.ToDouble(watch.ElapsedMilliseconds) / 1000} seconds to complete this task");
Console.ReadLine();
}
}
//OUTPUT:
/*
Download finished
Download finished
Download finished
Download finished
Download finished
It took the machine 1537 milliseconds or 1.537 seconds to complete this task
*/
As you can see, the DownloadWebsite method waits for 1.5 seconds and then returns "a". The method called ParallelAsyncMethod adds five of these methods into the "tasks" list and then starts the parallel asynchronous execution. As you can see, I also tracked the amount of time that it takes for the ExecuteAsync method to be executed. The result is always somewhere around 1540 milliseconds. Here is my question: if the DownloadWebsite method required a thread to sleep 5 times for 1500 milliseconds, does it mean that the parallel execution of these methods required 5 different threads? If not, then how come it only took the program 1540 milliseconds to be executed and not ~7500 ms?

I knew that an asynchronous method is not the same as multi-threading
That is correct, an asynchronous method releases the current thread whilst I/O occurs, and schedules a continuation after it's completion.
Async and threads are completely unrelated concepts.
but it does not seem so in this particular scenario
That is because you explicitly run DownloadWebsite on the ThreadPool using Task.Run, which imitates asynchronous code by returning a Task after instructing the provided delegate to run.
Because you are not waiting for each Task to complete before starting the next, multiple threads can be used simultaneously.
Currently each thread is being blocked, as you have used Thread.Sleep in the implementation of DownloadWebsite, meaning you are actually running 5 synchronous methods on the ThreadPool.
In production code your DownloadWebsite method should be written asynchronously, maybe using HttpClient.GetAsync:
private static async Task<string> DownloadWebsiteAsync()
{
//...
await httpClinet.GetAsync(//...
//...
}
In that case, GetAsync returns a Task, and releases the current thread whilst waiting for the HTTP response.
You can still run multiple async methods concurrently, but as the thread is released each time, this may well use less than 5 separate threads and may even use a single thread.
Ensure that you dont use Task.Run with an asynchronous method; this simply adds unnecessary overhead:
var tasks = new List<Task<string>>();
for (int i = 0; i < 5; i++)
{
tasks.Add(DownloadWebsiteAsync()); // No need for Task.Run
}
var strings = await Task.WhenAll(tasks);
As an aside, if you want to imitate an async operation, use Task.Delay instead of Thread.Sleep as the former is non-blocking:
private static async Task<string> DownloadWebsite() //Imitating a website download
{
await Task.Delay(1500); // Release the thread for ~1500ms before continuing
return "Download finished";
}

async await code executing not as expected

I created a simple example to understand async/await in C#.
class Program
{
static void Main(string[] args)
{
var t = BarAsync();
Console.WriteLine("Main");
}
private static async Task BarAsync()
{
Console.WriteLine("This happens before await");
int i = await QuxAsync();
Console.WriteLine("This happens after await. This result of await is " + i);
}
private static Task<int> QuxAsync()
{
int c = 0;
for (int i = 0; i < int.MaxValue; i++)
{
c++;
}
Console.WriteLine("in the middle processing...");
return Task.FromResult(c);
}
}
So the program prints This happens before await first then count the value from a return method. Afterwards it prints the result.
It looks good. My question is that since await doesn't block the thread that evaluates the async method. My understanding is if the async takes a long time it will return to its calling method.
For my example, because QuxAsync() takes a long time, the code
Console.WriteLine("Main");
is not blocked and will be evaluated very soon.
I think the print order should be
This happens before await
Main
in the middle processing...
This happens after await. This result of await is 2147483647
However it is not, why?

I'll second (third?) others' recommendations that you continue reading and learning about async. I'm partial to my own async intro, but these days there are a number of good ones out there.
My question is that since await doesn't block the thread that evaluates the async method. My understanding is if the async takes a long time it will return to its calling method.
This is the part that is wrong. Asynchrony has absolutely nothing to do with how long something takes.
There's two pieces of knowledge missing, both covered in my async intro.
First: await works by first checking its argument. If it is already completed, then await continues executing - synchronously.
Second: every method is called synchronously. Including async methods. The only time asynchrony happens is when an async method has an await whose argument is not already completed; in that case, that async method returns an incomplete task.
Putting both of these together should explain why your code actually runs synchronously.

Does async/await inside a loop create a bottleneck?

Lets say i have the following code for example:
private async Task ManageClients()
{
for (int i =0; i < listClients.Count; i++)
{
if (list[i] == 0)
await DoSomethingWithClientAsync();
else
await DoOtherThingAsync();
}
DoOtherWork();
}
My questions are:
1. Will the for() continue and process other clients on the list?, or it
will await untill it finishes one of the tasks.
2. Is even a good practice to use async/await inside a loop?
3. Can it be done in a better way?
I know it was a really simple example, but I'm trying to imagine what would happen if that code was a server with thousands of clients.

In your code example, the code will "block" when the loop reaches await, meaning the other clients will not be processed until the first one is complete. That is because, while the code uses asynchronous calls, it was written using a synchronous logic mindset.
An asynchronous approach should look more like this:
private async Task ManageClients()
{
var tasks = listClients.Select( client => DoSomethingWithClient() );
await Task.WhenAll(tasks);
DoOtherWork();
}
Notice there is only one await, which simultaneously awaits all of the clients, and allows them to complete in any order. This avoids the situation where the loop is blocked waiting for the first client.

If a thread that is executing an async function is calling another async function, this other function is executed as if it was not async until it sees a call to a third async function. This third async function is executed also as if it was not async.
This goes on, until the thread sees an await.
Instead of really doing nothing, the thread goes up the call stack, to see if the caller was not awaiting for the result of the called function. If not, the thread continues the statements in the caller function until it sees an await. The thread goes up the call stack again to see if it can continue there.
This can be seen in the following code:
var taskDoSomething = DoSomethingAsync(...);
// because we are not awaiting, the following is done as soon as DoSomethingAsync has to await:
DoSomethingElse();
// from here we need the result from DoSomethingAsync. await for it:
var someResult = await taskDoSomething;
You can even call several sub-procedures without awaiting:
var taskDoSomething = DoSomethingAsync(...);
var taskDoSomethingElse = DoSomethingElseAsync(...);
// we are here both tasks are awaiting
DoSomethingElse();
Once you need the results of the tasks, if depends what you want to do with them. Can you continue processing if one task is completed but the other is not?
var someResult = await taskDoSomething;
ProcessResult(someResult);
var someOtherResult = await taskDoSomethingelse;
ProcessBothResults(someResult, someOtherResult);
If you need the result of all tasks before you can continue, use Task.WhenAll:
Task[] allTasks = new Task[] {taskDoSomething, taskDoSomethingElse);
await Task.WhenAll(allTasks);
var someResult = taskDoSomething.Result;
var someOtherResult = taskDoSomethingElse.Result;
ProcessBothResults(someResult, someOtherResult);
Back to your question
If you have a sequence of items where you need to start awaitable tasks, it depends on whether the tasks need the result of other tasks or not. In other words can task[2] start if task[1] has not been completed yet? Do Task[1] and Task[2] interfere with each other if they run both at the same time?
If they are independent, then start all Tasks without awaiting. Then use Task.WhenAll to wait until all are finished. The Task scheduler will take care that not to many tasks will be started at the same time. Be aware though, that starting several tasks could lead to deadlocks. Check carefully if you need critical sections
var clientTasks = new List<Task>();
foreach(var client in clients)
{
if (list[i] == 0)
clientTasks.Add(DoSomethingWithClientAsync());
else
clientTasks.Add(DoOtherThingAsync());
}
// if here: all tasks started. If desired you can do other things:
AndNowForSomethingCompletelyDifferent();
// later we need the other tasks to be finished:
var taskWaitAll = Task.WhenAll(clientTasks);
// did you notice we still did not await yet, we are still in business:
MontyPython();
// okay, done with frolicking, we need the results:
await taskWaitAll;
DoOtherWork();
This was the scenario where all Tasks where independent: no task needed the other to be completed before it could start. However if you need Task[2] to be completed before you can start Task[3] you should await:
foreach(var client in clients)
{
if (list[i] == 0)
await DoSomethingWithClientAsync());
else
await DoOtherThingAsync();
}

Inside a loop,does each async call get chained to the returned task using task's continuewith?

The best practice is to collect all the async calls in a collection inside the loop and do Task.WhenAll(). Yet, want to understand what happens when an await is encountered inside the loop, what would the returned Task contain? what about further async calls? Will it create new tasks and add them to the already returned Task sequentially?
As per the code below
private void CallLoopAsync()
{
var loopReturnedTask = LoopAsync();
}
private async Task LoopAsync()
{
int count = 0;
while(count < 5)
{
await SomeNetworkCallAsync();
count++;
}
}
The steps I assumed are
LoopAsync gets called
count is set to zero, code enters while loop, condition is checked
SomeNetworkCallAsync is called,and the returned task is awaited
New task/awaitable is created
New task is returned to CallLoopAsync()
Now, provided there is enough time for the process to live, How / In what way, will the next code lines like count++ and further SomeNetworkCallAsync be executed?
Update - Based on Jon Hanna and Stephen Cleary:
So there is one Task and the implementation of that Task will involve
5 calls to NetworkCallAsync, but the use of a state-machine means
those tasks need not be explicitly chained for this to work. This, for
example, allows it to decide whether to break the looping or not based
on the result of a task, and so on.
Though they are not chained, each call will wait for the previous call to complete as we have used await (in state m/c, awaiter.GetResult();). It behaves as if five consecutive calls have been made and they are executed one after the another (only after the previous call gets completed). If this is true, we have to be bit more careful in how we are composing the async calls.For ex:
Instead of writing
private async Task SomeWorkAsync()
{
await SomeIndependentNetworkCall();// 2 sec to complete
var result1 = await GetDataFromNetworkCallAsync(); // 2 sec to complete
await PostDataToNetworkAsync(result1); // 2 sec to complete
}
It should be written
private Task[] RefactoredSomeWorkAsync()
{
var task1 = SomeIndependentNetworkCall();// 2 sec to complete
var task2 = GetDataFromNetworkCallAsync()
.ContinueWith(result1 => PostDataToNetworkAsync(result1)).Unwrap();// 4 sec to complete
return new[] { task1, task2 };
}
So that we can say RefactoredSomeWorkAsync is faster by 2 seconds, because of the possibility of parallelism
private async Task CallRefactoredSomeWorkAsync()
{
await Task.WhenAll(RefactoredSomeWorkAsync());//Faster, 4 sec
await SomeWorkAsync(); // Slower, 6 sec
}
Is this correct? - Yes. Along with "async all the way", "Accumulate tasks all the way" is good practice. Similar discussion is here

When count is zero, new task will be created because of await and be returned
No. It will not. It will simply call the async method consequently, without storing or returning the result. The value in loopReturnedTask will store the Task of LoopAsync, not related to SomeNetworkCallAsync.
await SomeNetworkCallAsync(); // call, wait and forget the result
You may want to read the MSDN article on async\await.

To produce code similar to what async and await do, if those keywords didn't exist, would require code a bit like:
private struct LoopAsyncStateMachine : IAsyncStateMachine
{
public int _state;
public AsyncTaskMethodBuilder _builder;
public TestAsync _this;
public int _count;
private TaskAwaiter _awaiter;
void IAsyncStateMachine.MoveNext()
{
try
{
if (_state != 0)
{
_count = 0;
goto afterSetup;
}
TaskAwaiter awaiter = _awaiter;
_awaiter = default(TaskAwaiter);
_state = -1;
loopBack:
awaiter.GetResult();
awaiter = default(TaskAwaiter);
_count++;
afterSetup:
if (_count < 5)
{
awaiter = _this.SomeNetworkCallAsync().GetAwaiter();
if (!awaiter.IsCompleted)
{
_state = 0;
_awaiter = awaiter;
_builder.AwaitUnsafeOnCompleted<TaskAwaiter, TestAsync.LoopAsyncStateMachine>(ref awaiter, ref this);
return;
}
goto loopBack;
}
_state = -2;
_builder.SetResult();
}
catch (Exception exception)
{
_state = -2;
_builder.SetException(exception);
return;
}
}
[DebuggerHidden]
void IAsyncStateMachine.SetStateMachine(IAsyncStateMachine param0)
{
_builder.SetStateMachine(param0);
}
}
public Task LoopAsync()
{
LoopAsyncStateMachine stateMachine = new LoopAsyncStateMachine();
stateMachine._this = this;
AsyncTaskMethodBuilder builder = AsyncTaskMethodBuilder.Create();
stateMachine._builder = builder;
stateMachine._state = -1;
builder.Start(ref stateMachine);
return builder.Task;
}
(The above is based on what happens when you use async and await except that the result of that uses names that cannot be valid C# class or field names, along with some extra attributes. If its MoveNext() reminds you of an IEnumerator that's not entirely irrelevant, the mechanism by which await and async produce an IAsyncStateMachine to implement a Task is similar in many ways to how yield produces an IEnumerator<T>).
The result is a single Task which comes from AsyncTaskMethodBuilder and makes use of LoopAsyncStateMachine (which is close to the hidden struct that the async produces). Its MoveNext() method is first called upon the task being started. It will then use an awaiter on SomeNetworkCallAsync. If it is already completed it moves on to the next stage (increment count and so on), otherwise it stores the awaiter in a field. On subsequent uses it will be called because the SomeNetworkCallAsync() task has returned, and it will get the result (which is void in this case, but could be a value if values were returned). It then attempts further loops and again returns when it is waiting on a task that is not yet completed.
When it finally reaches a count of 5 it calls SetResult() on the builder, which sets the result of the Task that LoopAsync had returned.
So there is one Task and the implementation of that Task will involve 5 calls to NetworkCallAsync, but the use of a state-machine means those tasks need not be explicitly chained for this to work. This, for example, allows it to decide whether to break the looping or not based on the result of a task, and so on.

When an async method first yields at an await, it returns a Task (or Task<T>). This is not the task being observed by the await; it is a completely different task created by the async method. The async state machine controls the lifetime of that Task.
One way to think of it is to consider the returned Task as representing the method itself. The returned Task will only complete when the method completes. If the method returns a value, then that value is set as the result of the task. If the method throws an exception, then that exception is captured by the state machine and placed on that task.
So, there's no need for attaching continuations to the returned task. The returned task will not complete until the method is done.
How / In what way, will the next code lines like count++ and further SomeNetworkCallAsync be executed?
I do explain this in my async intro post. In summary, when a method awaits, it captures a "current context" (SynchronizationContext.Current unless it is null, in which case it uses TaskScheduler.Current). When the await completes, it resumes executing its async method within that context.
That's what technically happens; but in the vast majority of cases, this simply means:
If an async method starts on a UI thread, then it will resume on that same UI thread.
If an async method starts within an ASP.NET request context, then it will resume with that same request context (not necessarily on the same thread, though).
Otherwise, the async method resumes on a thread pool thread.