We Keep Coding

Using Task.Yield to overcome ThreadPool starvation while implementing producer/consumer pattern

Answering the question: Task.Yield - real usages?
I proposed to use Task.Yield allowing a pool thread to be reused by other tasks. In such pattern:
CancellationTokenSource cts;
void Start()
{
cts = new CancellationTokenSource();
// run async operation
var task = Task.Run(() => SomeWork(cts.Token), cts.Token);
// wait for completion
// after the completion handle the result/ cancellation/ errors
}
async Task<int> SomeWork(CancellationToken cancellationToken)
{
int result = 0;
bool loopAgain = true;
while (loopAgain)
{
// do something ... means a substantial work or a micro batch here - not processing a single byte
loopAgain = /* check for loop end && */ cancellationToken.IsCancellationRequested;
if (loopAgain) {
// reschedule the task to the threadpool and free this thread for other waiting tasks
await Task.Yield();
}
}
cancellationToken.ThrowIfCancellationRequested();
return result;
}
void Cancel()
{
// request cancelation
cts.Cancel();
}
But one user wrote
I don't think using Task.Yield to overcome ThreadPool starvation while
implementing producer/consumer pattern is a good idea. I suggest you
ask a separate question if you want to go into details as to why.
Anybody knows, why is not a good idea?

There are some good points left in the comments to your question. Being the user you quoted, I'd just like to sum it up: use the right tool for the job.
Using ThreadPool doesn't feel like the right tool for executing multiple continuous CPU-bound tasks, even if you try to organize some cooperative execution by turning them into state machines which yield CPU time to each other with await Task.Yield(). Thread switching is rather expensive; by doing await Task.Yield() on a tight loop you add a significant overhead. Besides, you should never take over the whole ThreadPool, as the .NET framework (and the underlying OS process) may need it for other things. On a related note, TPL even has the TaskCreationOptions.LongRunning option that requests to not run the task on a ThreadPool thread (rather, it creates a normal thread with new Thread() behind the scene).
That said, using a custom TaskScheduler with limited parallelism on some dedicated, out-of-pool threads with thread affinity for individual long-running tasks might be a different thing. At least, await continuations would be posted on the same thread, which should help reducing the switching overhead. This reminds me of a different problem I was trying to solve a while ago with ThreadAffinityTaskScheduler.
Still, depending on a particular scenario, it's usually better to use an existing well-established and tested tool. To name a few: Parallel Class, TPL Dataflow, System.Threading.Channels, Reactive Extensions.
There is also a whole range of existing industrial-strength solutions to deal with Publish-Subscribe pattern (RabbitMQ, PubNub, Redis, Azure Service Bus, Firebase Cloud Messaging (FCM), Amazon Simple Queue Service (SQS) etc).

After a bit of debating on the issue with other users - who are worried about the context switching and its influence on the performance.
I see what they are worried about.
But I meant: do something ... inside the loop to be a substantial task - usually in the form of a message handler which reads a message from the queue and processes it. The message handlers are usually user defined and the message bus executes them using some sort of dispatcher. The user can implement a handler which executes synchronously (nobody knows what the user will do), and without Task.Yield that will block the thread to process those synchronous tasks in a loop.
Not to be empty worded i added tests to github: https://github.com/BBGONE/TestThreadAffinity
They compare the ThreadAffinityTaskScheduler, .NET ThreadScheduler with BlockingCollection and .NET ThreadScheduler with Threading.Channels.
The tests show that for Ultra Short jobs the performance degradation is
around 15%. To use the Task.Yield without the performance degradation (even small) - it is not to use extremely short tasks and if the task is too short then combine shorter tasks into a bigger batch.
[The price of context switch] = [context switch duration] / ([job duration]+[context switch duration]).
In that case the influence of the switching the tasks is negligible on the performance. But it adds a better task cooperation and responsiveness of the system.
For long running tasks it is better to use a custom Scheduler which executes tasks on its own dedicated thread pool - (like the WorkStealingTaskScheduler).
For the mixed jobs - which can contain different parts - short running CPU bound, asynchronous and long running code parts. It is better to split the task into subtasks.
private async Task HandleLongRunMessage(TestMessage message, CancellationToken token = default(CancellationToken))
{
// SHORT SYNCHRONOUS TASK - execute as is on the default thread (from thread pool)
CPU_TASK(message, 50);
// IO BOUND ASYNCH TASK - used as is
await Task.Delay(50);
// BUT WRAP the LONG SYNCHRONOUS TASK inside the Task
// which is scheduled on the custom thread pool
// (to save threadpool threads)
await Task.Factory.StartNew(() => {
CPU_TASK(message, 100000);
}, token, TaskCreationOptions.DenyChildAttach, _workStealingTaskScheduler);
}

How can I have two separate task schedulers?

I am writing a game, and using OpenGL I require that some work be offloaded to the rendering thread where an OpenGL context is active, but everything else is handled by the normal thread pool.
Is there a way I can force a Task to be executed in a special thread-pool, and any new tasks created from an async also be dispatched to that thread pool?
I want a few specialized threads for rendering, and I would like to be able to use async and await for example for creating and filling a vertex buffer.
If I just use a custom task scheduler and a new Factory(new MyScheduler()) it seems that any subsequent Task objects will be dispatched to the thread pool anyway where Task.Factory.Scheduler suddenly is null.
The following code should show what I want to be able to do:
public async Task Initialize()
{
// The two following tasks should run on the rendering thread pool
// They cannot run synchronously because that will cause them to fail.
this.VertexBuffer = await CreateVertexBuffer();
this.IndexBuffer = await CreateIndexBuffer();
// This should be dispatched, or run synchrounousyly, on the normal thread pool
Vertex[] vertices = CreateVertices();
// Issue task for filling vertex buffer on rendering thread pool
var fillVertexBufferTask = FillVertexBufffer(vertices, this.VertexBuffer);
// This should be dispatched, or run synchrounousyly, on the normal thread pool
short[] indices = CreateIndices();
// Wait for tasks on the rendering thread pool to complete.
await FillIndexBuffer(indices, this.IndexBuffer);
await fillVertexBufferTask; // Wait for the rendering task to complete.
}
Is there any way to achieve this, or is it outside the scope of async/await?

This is possible and basically the same thing what Microsoft did for the Windows Forms and WPF Synchronization Context.
First Part - You are in the OpenGL thread, and want to put some work into the thread pool, and after this work is done you want back into the OpenGL thread.
I think the best way for you to go about this is to implement your own SynchronizationContext. This thing basically controls how the TaskScheduler works and how it schedules the task. The default implementation simply sends the tasks to the thread pool. What you need to do is to send the task to a dedicated thread (that holds the OpenGL context) and execute them one by one there.
The key of the implementation is to overwrite the Post and the Send methods. Both methods are expected to execute the callback, where Send has to wait for the call to finish and Post does not. The example implementation using the thread pool is that Sendsimply directly calls the callback and Post delegates the callback to the thread pool.
For the execution queue for your OpenGL thread I am think a Thread that queries a BlockingCollection should do nicely. Just send the callbacks to this queue. You may also need some callback in case your post method is called from the wrong thread and you need to wait for the task to finish.
But all in all this way should work. async/await ensures that the SynchronizationContext is restored after a async call that is executed in the thread pool for example. So you should be able to return to the OpenGL thread after you did put some work off into another thread.
Second Part - You are in another thread and want to send some work into the OpenGL thread and await the completion of that work.
This is possible too. My idea in this case is that you don't use Tasks but other awaitable objects. In general every object can be awaitable. It just has to implement a public method getAwaiter() that returns a object implementing the INotifyCompletion interface. What await does is that it puts the remaining method into a new Action and sends this action to the OnCompleted method of that interface. The awaiter is expected to call the scheduled actions once the operation it is awaiting is done. Also this awaiter has to ensure that the SynchronizationContext is captured and the continuations are executed on the captured SynchronizationContext. That sounds complicated, but once you get the hang of it, it goes fairly easy. What helped me a lot is the reference source of the YieldAwaiter (this is basically what happens if you use await Task.Yield()). This is not what you need, but I think it is a place to start.
The method that returns the awaiter has to take care of sending the actual work to the thread that has to execute it (you maybe already have the execution queue from the first part) and the awaiter has to trigger once that work is done.
Conclusion
Make no mistake. That is a lot of work. But if you do all that you will have less problem down the line because you can seamless use the async/await pattern as if you would be working inside windows forms or WPF and that is a hue plus.

First, realize that await introduces the special behavior after the method is called; that is to say, this code:
this.VertexBuffer = await CreateVertexBuffer();
is pretty much the same as this code:
var createVertexBufferTask = CreateVertexBuffer();
this.VertexBuffer = await createVertexBufferTask;
So, you'll have to explicitly schedule code to execute a method within a different context.
You mention using a MyScheduler but I don't see your code using it. Something like this should work:
this.factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskContinuationOptions.None, new MyScheduler());
public async Task Initialize()
{
// Since you mention OpenGL, I'm assuming this method is called on the UI thread.
// Run these methods on the rendering thread pool.
this.VertexBuffer = await this.factory.StartNew(() => CreateVertexBuffer()).Unwrap();
this.IndexBuffer = await this.factory.StartNew(() => CreateIndexBuffer()).Unwrap();
// Run these methods on the normal thread pool.
Vertex[] vertices = await Task.Run(() => CreateVertices());
var fillVertexBufferTask = Task.Run(() => FillVertexBufffer(vertices, this.VertexBuffer));
short[] indices = await Task.Run(() => CreateIndices());
await Task.Run(() => FillIndexBuffer(indices, this.IndexBuffer));
// Wait for the rendering task to complete.
await fillVertexBufferTask;
}
I would look into combining those multiple Task.Run calls, or (if Initialize is called on a normal thread pool thread) removing them completely.

How does running several tasks asynchronously on UI thread using async/await work?

I've read (and used) async/await quite a lot for some time now but I still have one question I can't get an answer to. Say I have this code.
private async void workAsyncBtn_Click(object sender, EventArgs e)
{
var myTask = _asyncAwaitExcamples.DoHeavyWorkAsync(5);
await myTask;
statusTextBox.Text += "\r\n DoHeavyWorkAsync message";
}
It's called from the UI thread and returned to the UI Thread. Therefor I am able to do UI-specific things in this method and after the await myTask. If I had used .ConfigureAwait(false) I would get a thread exception when doing statusTextBox.Text += "\r\n DoHeavyWorkAsync message"; since I would have telled myTask it's ok to take any available thread from the thread pool.
My question. As I understand it I never leave the UI thread in this case, still it's run asynchronously, the UI is still responsive and I can start several Tasks at the same time and therefor speed up my application. How can this work if we only use one thread?
Thanks!
EDIT for Sievajet
private async void workAsyncBtn_Click(object sender, EventArgs e)
{
await DoAsync();
}
private async Task DoAsync()
{
await Task.Delay(200);
statusTextBox.Text += "Call to form";
await Task.Delay(200);
}

As I understand it I never leave the UI thread in this case, still
it's run asynchronously, the UI is still responsive and I can start
several Tasks at the same time and therefor speed up my application.
How can this work if we only use one thread?
First, i'd recommend reading Stephan Clearys blog post - There is no thread.
In order to understand how its possible to run multiple units of work altogether, we need to grasp one important fact: async IO bound operations have (almost) nothing to do with threads.
How is that possible? well, if we drill deep down all the way to the operating system, we'll see that the calls to the device drivers - those which are in charge of doing operations such as network calls and writing to disk, were all implemented as naturally asynchronous, they don't occupy a thread while doing their work. That way, while the device driver is doing its thing, there need not be a thread. only once the device driver completes its execution, it will signal the operating system that it's done via an IOCP (I/O completion port), which will then execute the rest of the method call (this is done in .NET via the threadpool, which has dedicated IOCP threads).
Stephans blog post demonstrates this nicely:
Once the OS executes the DPC (Deferred Procedure Call) and queue the IRP (I/O Request Packet), it's work is essentially done until the device driver signals it back with the I'm done messages, which causes a whole chain of operations (described in the blog post) to execute, which eventually will end up with invoking your code.
Another thing to note is that .NET does some "magic" for us behind the scenes when using async-await pattern. There is a thing called "Synchronization Context" (you can find a rather lengthy explanation here). This sync context is whats in-charge of invoking the continuation (code after the first await) on the UI thread back again (in places where such context exists).
Edit:
It should be noted that the magic with the synchronization context happens for CPU bound operations as well (and actually for any awaitable object), so when you use a threadpool thread via Task.Run or Task.Factory.StartNew, this will work as well.

The TaskParallelLibrary (TPL) uses a TaskScheduler which can be configured with TaskScheduler.FromCurrentSynchronizationContext to return to the SynchronizationContext like this :
textBox1.Text = "Start";
// The SynchronizationContext is captured here
Factory.StartNew( () => DoSomeAsyncWork() )
.ContinueWith(
() =>
{
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
},TaskScheduler.FromCurrentSynchronizationContext());
When an async method suspends at an await, by default it will capture the current SynchronizationContext and marshall the code after the await back on the SynchronizationContext it came from.
textBox1.Text = "Start";
// The SynchronizationContext is captured here
/* The implementation of DoSomeAsyncWork depends how it runs, this could run on the threadpool pool
or it could be an 'I/O operation' or an 'Network operation'
which doesnt use the threadpool */
await DoSomeAsyncWork();
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
async and await example:
async Task MyMethodAsync()
{
textBox1.Text = "Start";
// The SynchronizationContext is captured here
await Task.Run(() => { DoSomeAsyncWork(); }); // run on the threadPool
// Back on the SynchronizationContext it came from
textBox1.Text = "End";
}

When UI thread calls await it starts the async operation and returns immediately. When the async operation completes, it notifies a thread from the thread pool but the internal implementation of async await dispatches the execution to the UI thread which will continue the execution of the code after the await.
The Dispatch is implemented by means of SynchronizationContext which in turn calls System.Windows.Forms.Control.BeginInvoke.
CLR via C# (4th Edition) (Developer Reference) 4th Edition by Jeffrey Richter page 749
Actually, Jeffrey worked with MS to implement the async/await inspired by his AsyncEnumerator

Force a Task to continue on the current thread?

I'm making a port of the AKKA framework for .NET (don't take this too serious now, it is a weekend hack of the Actor part of it right now)
I'm having some problems with the "Future" support in it.
In Java/Scala Akka, Futures are to be awaited synchronously with an Await call.
Much like the .NET Task.Wait()
My goal is to support true async await for this.
It works right now, but the continuation is executed on the wrong thread in my current solution.
This is the result when passing a message to one of my actors that contain an await block for a future.
As you can see, the actor always executes on the same thread, while the await block executes on a random threadpool thread.
actor thread: 6
await thread 10
actor thread: 6
await thread 12
actor thread: 6
actor thread: 6
await thread 13
...
The actor gets a message using a DataFlow BufferBlock<Message>
Or rather, I use RX over the bufferblock to subscribe to messages.
It is configured like this:
var messages = new BufferBlock<Message>()
{
BoundedCapacity = 100,
TaskScheduler = TaskScheduler.Default,
};
messages.AsObservable().Subscribe(this);
So far so good.
However, when I await on a future result.
like so:
protected override void OnReceive(IMessage message)
{
....
var result = await Ask(logger, m);
// This is not executed on the same thread as the above code
result.Match()
.With<SomeMessage>(t => {
Console.WriteLine("await thread {0}",
System.Threading.Thread.CurrentThread.GetHashCode());
})
.Default(_ => Console.WriteLine("Unknown message"));
...
I know this is normal behavior of async await, but I really must ensure that only one thread has access to my actor.
I don't want the future to run synchronously, I want to to run async just like normal, but I want the continuation to run on the same thread as the message processor/actor does.
My code for the future support looks like this:
public Task<IMessage> Ask(ActorRef actor, IMessage message)
{
TaskCompletionSource<IMessage> result =
new TaskCompletionSource<IMessage>();
var future = Context.ActorOf<FutureActor>(name : Guid.NewGuid().ToString());
// once this object gets a response,
// we set the result for the task completion source
var futureActorRef = new FutureActorRef(result);
future.Tell(new SetRespondTo(), futureActorRef);
actor.Tell(message, future);
return result.Task;
}
Any ideas what I can do to force the continuation to run on the same thread that started the above code?

I'm making a port of the AKKA framework for .NET
Sweet. I went to an Akka talk at CodeMash '13 despite having never touched Java/Scala/Akka. I saw a lot of potential there for a .NET library/framework. Microsoft is working on something similar, which I hope will eventually be made generally available (it's currently in a limited preview).
I suspect that staying in the Dataflow/Rx world as much as possible is the easier approach; async is best when you have asynchronous operations (with a single start and single result for each operation), while Dataflow and Rx work better with streams and subscriptions (with a single start and multiple results). So my first gut reaction is to either link the buffer block to an ActionBlock with a specific scheduler, or use ObserveOn to move the Rx notifications to a specific scheduler, instead of trying to do it on the async side. Of course I'm not really familiar with the Akka API design, so take that with a grain of salt.
Anyway, my async intro describes the only two reliable options for scheduling await continuations: SynchronizationContext.Current and TaskScheduler.Current. If your Akka port is more of a framework (where your code does the hosting, and end-user code is always executed by your code), then a SynchronizationContext may make sense. If your port is more of a library (where end-user code does the hosting and calls your code as necessary), then a TaskScheduler would make more sense.
There aren't many examples of a custom SynchronizationContext, because that's pretty rare. I do have an AsyncContextThread type in my AsyncEx library which defines both a SynchronizationContext and a TaskScheduler for that thread. There are several examples of custom TaskSchedulers, such as the Parallel Extensions Extras which has an STA scheduler and a "current thread" scheduler.

Task scheduler decides whether to run a task on a new thread or on the current thread.
There is an option to force running it on a new thread, but none forcing it to run on the current thread.
But there is a method Task.RunSynchronously() which Runs the Task synchronously on the current TaskScheduler.
Also if you are using async/await there is already a similar question on that.

Aysnc/Await Vs. Task/Continuation UI Control Access

All, I have a situation where I have been asked to multi-thread a large 'Cost-Crunching' algorithm. I am relatively experienced with Tasks and would be confident in adopting a pattern like
CancellationTokenSource cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();
Task<bool> asyncTask = null;
asyncTask = Task.Factory.StartNew<bool>(() =>
SomeMethodAsync(uiScheduler, token, _dynamic), token);
asyncTask.ContinueWith(task =>
{
// For call back, exception handling etc.
}, uiScheduler);
and then for any operation where I need to provide and UI operation, I would use
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
Where this might be wrapped up in a method.
Now, I realise that I can make all this much less complicated, and leverage the async/await keywords of .NET 4.5. However, I have some questions: if I have a long running method that I launch using
// Start processing asynchroniously.
IProgress<CostEngine.ProgressInfo> progressIndicator =
new Progress<CostEngine.ProgressInfo>();
cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
CostEngine.ScriptProcessor script = new CostEngine.ScriptProcessor(this);
await script.ProcessScriptAsync(doc, progressIndicator, token);
where CostEngine.ProgressInfo is some basic class used to return progress information and the method ProcessScriptAsync is defined as
public async Task ProcessScriptAsync(SSGForm doc, IProgress<ProgressInfo> progressInfo,
CancellationToken token, bool bShowCompleted = true)
{
...
if (!await Task<bool>.Run(() => TheLongRunningProcess(doc)))
return
...
}
I have two questions:
To get ProcessScriptAsync to return control to the UI almost immediately I await on a new Task<bool> delegate (this seemingly avoids an endless chain of async/awaits). Is this the right way to call ProcessScriptAsync? ['Lazy Initialisation', by wrapping in an outer method?]
To access the UI from within TheLongRunningProcess, do I merely pass in the UI TaskScheduler uiScheduler; i.e. TheLongRunningProcess(doc, uiScheduler), then use:
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
as before?
Sorry about the length and thanks for your time.

It depends. You've shown a lot of code, and yet omitted the one bit that you're actually asking a question about. First, without knowing what the code is we can't know if it's actually going to take a while or not. Next, if you await on a task that's already completed it will realize this, and not schedule a continuation but instead continue on (this is an optimization since scheduling tasks is time consuming). If the task you await isn't completed then the continuation will still be executed in the calling SynchronizationContext, which will again keep the UI thread busy. You can use ConfigureAwait(false) to ensure that the continuation runs in the thread pool though. This should handle both issues. Note that by doing this you can no longer access the UI controls in the ... sections of ProcessScriptAsync (without doing anything special). Also note that since ProcessScriptAsync is now executing in a thread pool thread, you don't need to use Task.Run to move the method call to a background thread.
That's one option, yes. Although, if you're updating the UI based on progress, that's what IProgress is for. I see you're using it already, so that is the preferable model for doing this. If this is updating a separate type of progress than the existing IProgress you are passing (i.e. the status text, rather than the percent complete as an int) then you can pass a second.

I think trying to switch back and forth between a background thread (for CPU intensive operations or IO operations with no async support) and the UI thread (to manipulate UI controls) is often a sign of bad design. Your calculations and your UI code should be separate.
If you're doing this just to notify the UI of some sort of progress, then use IProgress<T>. Any marshaling between threads then becomes the responsibility of the implementation of that interface and you can use Progress<T>, which does it correctly using the SynchronizationContext.
If you can't avoid mixing background thread code and UI thread code and your UI work isn't progress reporting (so IProgress<T> won't fit), I would probably enclose each bit of background thread code into its own await Task.Run(), and leave the UI code top level.
Your solution of using a single Task.Run() to run the background thread code and then switch to the UI thread using StartNew() with uiScheduler will work too. In that case, some helper methods might be useful, especially if you wanted to use await in the UI code too. (Otherwise, you would have to remember to double await the result of StartNew())
Yet another option would be create a SwitchTo(TaskScheduler) method, which would return a custom awaiter that continues on the given scheduler. Such method was in some of the async CTPs, but it was removed because it was deemed too dangerous when it comes to handling exceptions.