We Keep Coding

What exactly in an awaitable is being awaited? How do I make a Task properly?

I've searched a lot of information surrounding this topic and I understand the general premises of:
Await is a handing off of control from the callee backer to the caller
Most Modern I/O doesn't use real threading in underlying architecture
Most async methods do not explicitly spin up their own threads (i.e. Web Requests)
The last bullet in particular is what I want to discuss. To future-proof this let's use an example as a medium for explanation. Let's assume this is the code block:
public async Task<int> LongOperationWithAnInt32ResultAsync(string input)
{
/// Section A
_ = int.TryParse(input, out var parsedInt)
parsedInt = SyncOperationWithAnInt32Result(parsedInt);
/// Section B
await MyCustomTaskThatIWantAwaited();
/// Section C
return parsedInt;
}
private Task MyCustomTaskThatIWantAwaited()
{
/// Section D
AnotherSyncOperationWithVoidResult();
/// Section E
return Task.CompletedTask;
}
The method LongOperationWithAnInt32ResultAsync(string) will perform synchronously even though this is not the intended effect.
This is because when the caller enters the callee at Section B, the
code from Section D and Section E are executed immediately and are not
awaited. This behavior is changed if, Section D is removed and,
Section E was "return Task.Run(() =>
AnotherSyncOperationWithVoidResult())" instead. In this new Section E,
the awaitable being tracked becomes the thread from Task.Run (wrapped
with the returned Task).
If you replace Section B with "await Task.Delay(10000);" or "await FunctionalWebRequestAsync();" it works as intended. However, to my knowledge, neither of these internally generate a thread to be followed - so what exactly is being awaited?
I've accepted the main answer because it really helped me understand my misconception on Task functionality, but please also refer to my answer as well. It may be what you're looking for.

so what exactly is being awaited?
Nothing is being awaited. Await means asynchronous wait. For a wait to be asynchronous, the awaitable (the Task) should not be completed at the await point. In your case the awaitable is already completed (the IsCompleted property of the TaskAwaiter returns true), so the async state machine grabs immediately its result and proceeds with the next line as usual. There is no reason to pack the current state of the machine, invoke the OnCompleted method of the awaiter, and hand back an incomplete Task to the caller.
If you want to offload specific parts of an asynchronous method to the ThreadPool, the recommended way is to wrap these parts in Task.Run. Example:
public async Task<int> LongOperationWithAnInt32ResultAsync(string input)
{
/// Section A
_ = int.TryParse(input, out var parsedInt)
parsedInt = await Task.Run(() => SyncOperationWithAnInt32Result(parsedInt));
/// Section B
await Task.Run(async () => await MyCustomTaskThatIWantAwaited());
/// Section C
return parsedInt;
}
If you like the idea of controlling imperatively the thread where the code is running, there is a SwitchTo extension method available in the Microsoft.VisualStudio.Threading package. Usage example:
await TaskScheduler.Default.SwitchTo(); // Switch to the ThreadPool
The opinion of the experts is to avoid this approach, and stick with the Task.Run.

Huge thanks to #TheodorZoulias for his explanation and answer as it was critical in me reaching this point.
My misconception was simple, Task (or Task{T}) cannot be used as a delegate. Task is a class through and through, meaning that if you define this:
public Task DoSomeReallyLongWork()
{
SyncTaskThatIsReallyLong();
AnotherSyncTaskThatIsReallyLong();
/* perform as much work as needed here */
return Task.CompletedTask;
}
This will run synchronously and ONLY synchronously. The thing actually being awaited is the Task.CompletedTask object that you returned, nothing else. And since the Task is already completed, the internal awaiter is also marked as completed.
This means that though the intention may have been to wrap multiple methods within a Task and then execute/await it, what's actually happening is that the methods are executing synchronously like any other call and then a completed Task is being returned.
If you want multiple methods to be awaited, this is done by making a new Task Object. Using our previous example:
public async Task DoSomeReallyLongWorkAsync()
{
/// Task.Run does not necessarily run on a separate thread
/// this is up to the scheduler (usually the .NET scheduler)
await Task.Run(LongSyncTasksWrapped);
}
public void LongSyncTasksWrapped()
{
SyncTaskThatIsReallyLong();
AnotherSyncTaskThatIsReallyLong();
/* perform as much work as needed here */
return;
}
There may be instances where you want cold Tasks (task that haven't been started yet) and then run them when needed. Using the previous example, this would be done by:
public async Task DoSomeReallyLongWorkAsync()
{
var coldTask = new Task(LongSyncTasksWrapped);
/// Must call .Start() whenever you want the Task to
/// actually start. Await will not start the Task, its
/// just an asynchronous form of .Wait()
coldTask.Start();
/// coldTask was considered "hot" from .Start()
/// await is waiting a hot task.
await coldTask;
}
public void LongSyncTasksWrapped()
{
SyncTaskThatIsReallyLong();
AnotherSyncTaskThatIsReallyLong();
/* perform as much work as needed here */
return;
}
This answers the question of what's being awaited, its the Task's awaiter that is internally generated by the class.

How does Task.CompletedTask work with thread pool?

I'm new to C# asynchronous programming, just a question on the relationship between task and thread pool.
My understanding is:
When we create a Task, this Task is queued in the thread pool and the thread pool will schedule a worker thread to run this Task
And I saw the code below:
public Task InputOutputC() {
return Task.CompletedTask;
}
I don't quite get it, it seems that return a Task has already completed, which means a worker thread has already run this Task, but the meaning of Task is to let a worker thread in the thread pool to run it, if it has already finished, what's the point to return it to thread pool again and get executed again?

the meaning of Task is to let a worker thread in the thread pool to run it,
Running code on the thread pool is one way in which Tasks manifest themselves.
Other ways to create Tasks are to write async methods and to use Task.CompletedTask1 or Task.FromResult<TResult>2.
Just because Task.Run causes code to run on the thread pool does not mean that these other uses of Task must also necessarily involve the thread pool.
For Task.CompletedTask, especially, this is "I've already done the work required, but I want to present it to other code as a Task. No additional code runs anywhere.
We can see in the reference source that this property just returns the task:
/// <summary>A task that's already been completed successfully.</summary>
private static Task s_completedTask;
/// <summary>Gets a task that's already been completed successfully.</summary>
/// <remarks>May not always return the same instance.</remarks>
public static Task CompletedTask
{
get
{
var completedTask = s_completedTask;
if (completedTask == null)
s_completedTask = completedTask = new Task(false, (TaskCreationOptions)InternalTaskOptions.DoNotDispose, default(CancellationToken)); // benign initialization ----
return completedTask;
}
}
1As shown in the reference source later though, we often aren't even creating a new Task here, just reusing an existing one. But the team have obviously decided to forgo thread-safe initialisation of the property in favour of documenting that they won't guarantee to always return the same Task.
2These latter two are quite similar, in that they represent "I've already done the work required, now for some reason I need to pass some other code a Task"3.
3Often, and I'm guessing as is the case here, when you're implementing an interface or overriding a base class method that is Task returning but your code is fast and synchronous so you have no need to be async.

C# await tasks + infinite loop still freezing the UI

I am trying to get the proper 'structure' for monitoring the state of a game from external source(s) using (Tasks) async/await in order to run the tasks in an infinite loop, however the current way its written seems to just freeze up my UI.
What I have so far:
(in the "state machine" class)
// Start monitoring the game state for changes
public void Start()
{
tokenSource = new CancellationTokenSource();
CancellationToken token = tokenSource.Token;
IsRunning = true;
task = Task.Factory.StartNew(async () =>
{
while (true)
{
await Task.Run(()=>CheckForStateChange());
await Task.Delay(1000); // Pause 1 second before checking state again
}
}, token, TaskCreationOptions.LongRunning, TaskScheduler.FromCurrentSynchronizationContext());
}
Without the above "Task.Delay" line the UI completely freezes up. With the "Task.Delay" line it doesn't freeze up, but if I try to drag the window it skips back to where I began dragging it.
My assumption with the current code is that the 'await Task.Run()' executes and upon completion the 'await Task.Delay()' executes and then on completion returns to the beginning of the while(true) infinite loop. (ie. not running in parallel).
The CheckForStateChange() signature is as follows:
private void CheckForStateChange()
{
// ... A bunch of code to determine and update the current state value of the object
}
Nothing special there, simple non-async method. I have read through lots of examples / questions here on StackOverflow and I used to have CheckForStateChange as returning a Task (with awaitable actions inside the method) and many other iterations of code (with the same results).
Finally I call the Start() method from the main win32 form (button) as follows:
private void btnStartSW_Click(object sender, EventArgs e)
{
// Start the subscription of the event handler
if(!state.IsRunning)
{
state.StateChange += new SummonersWar.StateChangeHandler(OnGameStateChange);
state.Start();
}
}
I think the above code is the simplest form I have written the code structure in so far, but apparently its still not written 'properly'. Any help would be appreciated.
UPDATE:
The publisher side (state machine class):
// ------ Publisher of the event ---
public delegate void StateChangeHandler(string stateText);
public event StateChangeHandler StateChange;
protected void OnStateChange() // TODO pass text?
{
if (StateChange != null)
StateChange(StateText());
}
Where the StateText() method is just a temporary way of retrieving a 'text' representation of the current state (and is really a placeholder at this point until I organize it into a tidier struct)
IsRunning is purely a public bool.
And the handler in the UI thread:
private void OnGameStateChange(string stateText)
{
// Game State Changed (update the status bar)
labelGameState.Text = "State: " + stateText;
}

Why the UI freezes
In terms of the main question: you're already calling your CheckForStateChange via Task.Run, so there is no way that your CheckForStateChange will freeze the UI unless it includes calls which are marshalled back to the UI thread (i.e. Control.Invoke or SynchronizationContext.Post/Send used explicitly, or implicitly via a Task started on the UI TaskScheduler).
The best place to start looking is your StateChange handlers (i.e. StateChangeHandler). Also have a look at where the StateChange event is raised. You'll find thread marshalling code at one of these sites.
Other issues
You're passing the TaskScheduler pointing to the UI SynchronizationContext to the outer task. You're also passing in TaskCreationOptions.LongRunning. In simple terms you're telling the task factory to "start a task on a dedicated thread, and on the current thread". These two are mutually exclusive requirements and you can pretty safely drop them both.
If, as a result of the above, your outer task happens to execute on the UI thread, it won't really trip you up as the inner call is wrapped in Task.Run, but this probably isn't the behaviour you expect.
You are storing the result of Task.Factory.StartNew inside a task field or property. Note, however, that your Task.Factory.StartNew call returns a Task<Task>, so the saved Task instance will transition to completed state almost immediately unless you call Unwrap on it and get to the inner task. To avoid this entire mess, just use Task.Run to create the outer task (as it has Unwrap semantics built in). If you do that, you can ditch the inner Task.Run completely, like so:
public bool IsRunning
{
get
{
return task.Status == TaskStatus.Running;
}
}
public void Start()
{
tokenSource = new CancellationTokenSource();
CancellationToken token = tokenSource.Token;
task = Task.Run(async () =>
{
while (true)
{
CheckForStateChange(token);
token.ThrowIfCancellationRequested();
await Task.Delay(1000); // Pause 1 second before checking state again
}
}, token);
// Uncomment this and step through `CheckForStateChange`.
// When the execution hangs, you'll know what's causing the
// postbacks to the UI thread and *may* be able to take it out.
// task.Wait();
}
Since you have a CancellationToken you need to be passing it to CheckForStateChange, and checking it periodically - otherwise it only gets checked once, when the Task is started, and then never again.
Note that I have also provided a different IsRunning implementation. Volatile state is hard to get right. If the framework is giving it to you for free, you should use it.
Final word
Overall this entire solution feels like a bit of a crutch for something that should be done more reactively - but I can think of scenarios where this sort of design is valid. I'm just not convinced that yours is really one of them.
EDIT: how to find what's blocking the UI
I'll get downvoted to oblivion for this, but here goes:
The sure way to find what's causing postbacks to the UI thread is to deadlock with it. There's plenty of threads here on SO telling you how to avoid that, but in your case - we'll cause it on purpose and you'll know exactly what calls you need to avoid when you're polling for changes - although whether or not it will be possible to avoid these calls, remains to be seen.
I've put a task.Wait instruction at the end of my code snippet. Provided that you call Start on the UI thread, that should cause a deadlock with something inside your CheckForStateChange, and you will know what it is that you need to work around.

How can I have two separate task schedulers?

I am writing a game, and using OpenGL I require that some work be offloaded to the rendering thread where an OpenGL context is active, but everything else is handled by the normal thread pool.
Is there a way I can force a Task to be executed in a special thread-pool, and any new tasks created from an async also be dispatched to that thread pool?
I want a few specialized threads for rendering, and I would like to be able to use async and await for example for creating and filling a vertex buffer.
If I just use a custom task scheduler and a new Factory(new MyScheduler()) it seems that any subsequent Task objects will be dispatched to the thread pool anyway where Task.Factory.Scheduler suddenly is null.
The following code should show what I want to be able to do:
public async Task Initialize()
{
// The two following tasks should run on the rendering thread pool
// They cannot run synchronously because that will cause them to fail.
this.VertexBuffer = await CreateVertexBuffer();
this.IndexBuffer = await CreateIndexBuffer();
// This should be dispatched, or run synchrounousyly, on the normal thread pool
Vertex[] vertices = CreateVertices();
// Issue task for filling vertex buffer on rendering thread pool
var fillVertexBufferTask = FillVertexBufffer(vertices, this.VertexBuffer);
// This should be dispatched, or run synchrounousyly, on the normal thread pool
short[] indices = CreateIndices();
// Wait for tasks on the rendering thread pool to complete.
await FillIndexBuffer(indices, this.IndexBuffer);
await fillVertexBufferTask; // Wait for the rendering task to complete.
}
Is there any way to achieve this, or is it outside the scope of async/await?

This is possible and basically the same thing what Microsoft did for the Windows Forms and WPF Synchronization Context.
First Part - You are in the OpenGL thread, and want to put some work into the thread pool, and after this work is done you want back into the OpenGL thread.
I think the best way for you to go about this is to implement your own SynchronizationContext. This thing basically controls how the TaskScheduler works and how it schedules the task. The default implementation simply sends the tasks to the thread pool. What you need to do is to send the task to a dedicated thread (that holds the OpenGL context) and execute them one by one there.
The key of the implementation is to overwrite the Post and the Send methods. Both methods are expected to execute the callback, where Send has to wait for the call to finish and Post does not. The example implementation using the thread pool is that Sendsimply directly calls the callback and Post delegates the callback to the thread pool.
For the execution queue for your OpenGL thread I am think a Thread that queries a BlockingCollection should do nicely. Just send the callbacks to this queue. You may also need some callback in case your post method is called from the wrong thread and you need to wait for the task to finish.
But all in all this way should work. async/await ensures that the SynchronizationContext is restored after a async call that is executed in the thread pool for example. So you should be able to return to the OpenGL thread after you did put some work off into another thread.
Second Part - You are in another thread and want to send some work into the OpenGL thread and await the completion of that work.
This is possible too. My idea in this case is that you don't use Tasks but other awaitable objects. In general every object can be awaitable. It just has to implement a public method getAwaiter() that returns a object implementing the INotifyCompletion interface. What await does is that it puts the remaining method into a new Action and sends this action to the OnCompleted method of that interface. The awaiter is expected to call the scheduled actions once the operation it is awaiting is done. Also this awaiter has to ensure that the SynchronizationContext is captured and the continuations are executed on the captured SynchronizationContext. That sounds complicated, but once you get the hang of it, it goes fairly easy. What helped me a lot is the reference source of the YieldAwaiter (this is basically what happens if you use await Task.Yield()). This is not what you need, but I think it is a place to start.
The method that returns the awaiter has to take care of sending the actual work to the thread that has to execute it (you maybe already have the execution queue from the first part) and the awaiter has to trigger once that work is done.
Conclusion
Make no mistake. That is a lot of work. But if you do all that you will have less problem down the line because you can seamless use the async/await pattern as if you would be working inside windows forms or WPF and that is a hue plus.

First, realize that await introduces the special behavior after the method is called; that is to say, this code:
this.VertexBuffer = await CreateVertexBuffer();
is pretty much the same as this code:
var createVertexBufferTask = CreateVertexBuffer();
this.VertexBuffer = await createVertexBufferTask;
So, you'll have to explicitly schedule code to execute a method within a different context.
You mention using a MyScheduler but I don't see your code using it. Something like this should work:
this.factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskContinuationOptions.None, new MyScheduler());
public async Task Initialize()
{
// Since you mention OpenGL, I'm assuming this method is called on the UI thread.
// Run these methods on the rendering thread pool.
this.VertexBuffer = await this.factory.StartNew(() => CreateVertexBuffer()).Unwrap();
this.IndexBuffer = await this.factory.StartNew(() => CreateIndexBuffer()).Unwrap();
// Run these methods on the normal thread pool.
Vertex[] vertices = await Task.Run(() => CreateVertices());
var fillVertexBufferTask = Task.Run(() => FillVertexBufffer(vertices, this.VertexBuffer));
short[] indices = await Task.Run(() => CreateIndices());
await Task.Run(() => FillIndexBuffer(indices, this.IndexBuffer));
// Wait for the rendering task to complete.
await fillVertexBufferTask;
}
I would look into combining those multiple Task.Run calls, or (if Initialize is called on a normal thread pool thread) removing them completely.

Aysnc/Await Vs. Task/Continuation UI Control Access

All, I have a situation where I have been asked to multi-thread a large 'Cost-Crunching' algorithm. I am relatively experienced with Tasks and would be confident in adopting a pattern like
CancellationTokenSource cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();
Task<bool> asyncTask = null;
asyncTask = Task.Factory.StartNew<bool>(() =>
SomeMethodAsync(uiScheduler, token, _dynamic), token);
asyncTask.ContinueWith(task =>
{
// For call back, exception handling etc.
}, uiScheduler);
and then for any operation where I need to provide and UI operation, I would use
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
Where this might be wrapped up in a method.
Now, I realise that I can make all this much less complicated, and leverage the async/await keywords of .NET 4.5. However, I have some questions: if I have a long running method that I launch using
// Start processing asynchroniously.
IProgress<CostEngine.ProgressInfo> progressIndicator =
new Progress<CostEngine.ProgressInfo>();
cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
CostEngine.ScriptProcessor script = new CostEngine.ScriptProcessor(this);
await script.ProcessScriptAsync(doc, progressIndicator, token);
where CostEngine.ProgressInfo is some basic class used to return progress information and the method ProcessScriptAsync is defined as
public async Task ProcessScriptAsync(SSGForm doc, IProgress<ProgressInfo> progressInfo,
CancellationToken token, bool bShowCompleted = true)
{
...
if (!await Task<bool>.Run(() => TheLongRunningProcess(doc)))
return
...
}
I have two questions:
To get ProcessScriptAsync to return control to the UI almost immediately I await on a new Task<bool> delegate (this seemingly avoids an endless chain of async/awaits). Is this the right way to call ProcessScriptAsync? ['Lazy Initialisation', by wrapping in an outer method?]
To access the UI from within TheLongRunningProcess, do I merely pass in the UI TaskScheduler uiScheduler; i.e. TheLongRunningProcess(doc, uiScheduler), then use:
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
as before?
Sorry about the length and thanks for your time.

It depends. You've shown a lot of code, and yet omitted the one bit that you're actually asking a question about. First, without knowing what the code is we can't know if it's actually going to take a while or not. Next, if you await on a task that's already completed it will realize this, and not schedule a continuation but instead continue on (this is an optimization since scheduling tasks is time consuming). If the task you await isn't completed then the continuation will still be executed in the calling SynchronizationContext, which will again keep the UI thread busy. You can use ConfigureAwait(false) to ensure that the continuation runs in the thread pool though. This should handle both issues. Note that by doing this you can no longer access the UI controls in the ... sections of ProcessScriptAsync (without doing anything special). Also note that since ProcessScriptAsync is now executing in a thread pool thread, you don't need to use Task.Run to move the method call to a background thread.
That's one option, yes. Although, if you're updating the UI based on progress, that's what IProgress is for. I see you're using it already, so that is the preferable model for doing this. If this is updating a separate type of progress than the existing IProgress you are passing (i.e. the status text, rather than the percent complete as an int) then you can pass a second.

I think trying to switch back and forth between a background thread (for CPU intensive operations or IO operations with no async support) and the UI thread (to manipulate UI controls) is often a sign of bad design. Your calculations and your UI code should be separate.
If you're doing this just to notify the UI of some sort of progress, then use IProgress<T>. Any marshaling between threads then becomes the responsibility of the implementation of that interface and you can use Progress<T>, which does it correctly using the SynchronizationContext.
If you can't avoid mixing background thread code and UI thread code and your UI work isn't progress reporting (so IProgress<T> won't fit), I would probably enclose each bit of background thread code into its own await Task.Run(), and leave the UI code top level.
Your solution of using a single Task.Run() to run the background thread code and then switch to the UI thread using StartNew() with uiScheduler will work too. In that case, some helper methods might be useful, especially if you wanted to use await in the UI code too. (Otherwise, you would have to remember to double await the result of StartNew())
Yet another option would be create a SwitchTo(TaskScheduler) method, which would return a custom awaiter that continues on the given scheduler. Such method was in some of the async CTPs, but it was removed because it was deemed too dangerous when it comes to handling exceptions.