We Keep Coding

Await a thread wont work when dispatch inside of the thread

I'm running a Task in a new thread and want to wait for it to finish.
var a = "logic before the task starts";
await Task.Factory.StartNew(() => MyHugeFunction(_token), _token);
var b = "logic after the task is finished";
It worked perfectly until I started to dispatch inside of the thread with:
await Application.Current.Dispatcher.BeginInvoke(new Action(async () =>
{
SomeLogic();
}));
The task itself works, but the await for my running task isn't working anymore. As soon as I am dispatching in the thread, the var b will be assigned in the main thread.
I already have some workarounds but I just wondered if I'm doing something stupid or this is caused by other circumstances
I'm working with C# 8.0 and .Net Framework 4.7.2.

I'm running a Task in a new thread and want to wait for it to finish.
You want to use Task.Run instead of StartNew for that. StartNew is a dangerous, low-level API that should almost never be used, and in the few cases where you should use it, you should always pass a TaskScheduler.
await Application.Current.Dispatcher.BeginInvoke(new Action(async () =>
Imma stop you right there. First, you're explicitly creating an Action delegate with async, which results in an async void method, which should be avoided. Next, using Dispatcher to do any kind of Invoke or BeginInvoke really shouldn't be done at all.
Instead, use the Progress<T> type. This keeps your logic in your background thread, which can pass objects as updates to the UI thread, and the UI thread decides how to display those progress updates in the UI. Note the nice separation of concerns there, which tends to go out the window whenever people start using Dispatcher.
Both StartNew and Dispatcher are commonly seen in SO answers and blogs, but they're suboptimal solutions regardless.

How can I have two separate task schedulers?

I am writing a game, and using OpenGL I require that some work be offloaded to the rendering thread where an OpenGL context is active, but everything else is handled by the normal thread pool.
Is there a way I can force a Task to be executed in a special thread-pool, and any new tasks created from an async also be dispatched to that thread pool?
I want a few specialized threads for rendering, and I would like to be able to use async and await for example for creating and filling a vertex buffer.
If I just use a custom task scheduler and a new Factory(new MyScheduler()) it seems that any subsequent Task objects will be dispatched to the thread pool anyway where Task.Factory.Scheduler suddenly is null.
The following code should show what I want to be able to do:
public async Task Initialize()
{
// The two following tasks should run on the rendering thread pool
// They cannot run synchronously because that will cause them to fail.
this.VertexBuffer = await CreateVertexBuffer();
this.IndexBuffer = await CreateIndexBuffer();
// This should be dispatched, or run synchrounousyly, on the normal thread pool
Vertex[] vertices = CreateVertices();
// Issue task for filling vertex buffer on rendering thread pool
var fillVertexBufferTask = FillVertexBufffer(vertices, this.VertexBuffer);
// This should be dispatched, or run synchrounousyly, on the normal thread pool
short[] indices = CreateIndices();
// Wait for tasks on the rendering thread pool to complete.
await FillIndexBuffer(indices, this.IndexBuffer);
await fillVertexBufferTask; // Wait for the rendering task to complete.
}
Is there any way to achieve this, or is it outside the scope of async/await?

This is possible and basically the same thing what Microsoft did for the Windows Forms and WPF Synchronization Context.
First Part - You are in the OpenGL thread, and want to put some work into the thread pool, and after this work is done you want back into the OpenGL thread.
I think the best way for you to go about this is to implement your own SynchronizationContext. This thing basically controls how the TaskScheduler works and how it schedules the task. The default implementation simply sends the tasks to the thread pool. What you need to do is to send the task to a dedicated thread (that holds the OpenGL context) and execute them one by one there.
The key of the implementation is to overwrite the Post and the Send methods. Both methods are expected to execute the callback, where Send has to wait for the call to finish and Post does not. The example implementation using the thread pool is that Sendsimply directly calls the callback and Post delegates the callback to the thread pool.
For the execution queue for your OpenGL thread I am think a Thread that queries a BlockingCollection should do nicely. Just send the callbacks to this queue. You may also need some callback in case your post method is called from the wrong thread and you need to wait for the task to finish.
But all in all this way should work. async/await ensures that the SynchronizationContext is restored after a async call that is executed in the thread pool for example. So you should be able to return to the OpenGL thread after you did put some work off into another thread.
Second Part - You are in another thread and want to send some work into the OpenGL thread and await the completion of that work.
This is possible too. My idea in this case is that you don't use Tasks but other awaitable objects. In general every object can be awaitable. It just has to implement a public method getAwaiter() that returns a object implementing the INotifyCompletion interface. What await does is that it puts the remaining method into a new Action and sends this action to the OnCompleted method of that interface. The awaiter is expected to call the scheduled actions once the operation it is awaiting is done. Also this awaiter has to ensure that the SynchronizationContext is captured and the continuations are executed on the captured SynchronizationContext. That sounds complicated, but once you get the hang of it, it goes fairly easy. What helped me a lot is the reference source of the YieldAwaiter (this is basically what happens if you use await Task.Yield()). This is not what you need, but I think it is a place to start.
The method that returns the awaiter has to take care of sending the actual work to the thread that has to execute it (you maybe already have the execution queue from the first part) and the awaiter has to trigger once that work is done.
Conclusion
Make no mistake. That is a lot of work. But if you do all that you will have less problem down the line because you can seamless use the async/await pattern as if you would be working inside windows forms or WPF and that is a hue plus.

First, realize that await introduces the special behavior after the method is called; that is to say, this code:
this.VertexBuffer = await CreateVertexBuffer();
is pretty much the same as this code:
var createVertexBufferTask = CreateVertexBuffer();
this.VertexBuffer = await createVertexBufferTask;
So, you'll have to explicitly schedule code to execute a method within a different context.
You mention using a MyScheduler but I don't see your code using it. Something like this should work:
this.factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskContinuationOptions.None, new MyScheduler());
public async Task Initialize()
{
// Since you mention OpenGL, I'm assuming this method is called on the UI thread.
// Run these methods on the rendering thread pool.
this.VertexBuffer = await this.factory.StartNew(() => CreateVertexBuffer()).Unwrap();
this.IndexBuffer = await this.factory.StartNew(() => CreateIndexBuffer()).Unwrap();
// Run these methods on the normal thread pool.
Vertex[] vertices = await Task.Run(() => CreateVertices());
var fillVertexBufferTask = Task.Run(() => FillVertexBufffer(vertices, this.VertexBuffer));
short[] indices = await Task.Run(() => CreateIndices());
await Task.Run(() => FillIndexBuffer(indices, this.IndexBuffer));
// Wait for the rendering task to complete.
await fillVertexBufferTask;
}
I would look into combining those multiple Task.Run calls, or (if Initialize is called on a normal thread pool thread) removing them completely.

starts new thread inside Task.ContinueWith

i am building a WPF app where it needs job1 runs in the background; when job1 ends, it switches back to the UI thread using Task.ContinueWith and makes changes to the UI according to job1's result (in my case, starts another popup) and starts job2 in the background.
The problem is that, while job1 runs in a separate thread as it should, job2 runs in the UI thread (it blocks popup2). Maybe because job2 is started inside the ContinueWith block of task1? But why? It worked the first time, why not second time? How to fix this problem?
Thanks in advance!
enterPop(PopupTypes.popup1);
var task1 = new Task(() => job1()); //task runs in another thread
task1.ContinueWith(previousTask =>
{
//**back in UI thread**
//check result of job 1, start another popup in UI
exitPopup();
enterPop(PopupTypes.popup2);
//**SUPPOSEDLY** starts another background thread for task 2
var task2 = new Task(()=> job2());
task2.ContinueWith(previousTask =>{
//do task in UI thread...
},TaskScheduler.FromCurrentSynchronizationContext());
task2.Start();
}, TaskScheduler.FromCurrentSynchronizationContext());
task.Start();

It's not clear to me what the enterPop() and exitPopup() methods are supposed to do, as the code example is missing that detail. Maybe they are displaying some kind of window in the GUI? It's also not clear what code exists in the "check result of job 1" part, "do task in UI thread", etc.
Without these details, it's impossible to know what your code is actually doing.
In any case, Hans's advice is correct. It's not really clear from the incomplete code example why you get the behavior you're seeing (for that matter, it's not really entirely clear to me what behavior you're seeing). But it's likely that if you'd written the code in the more usual async/await pattern, it would work fine:
enterPop(PopupTypes.popup1);
await Task.Run(() => job1()); //task runs in another thread
//**back in UI thread**
//check result of job 1, start another popup in UI
exitPopup();
enterPop(PopupTypes.popup2);
// starts another background thread for task 2
await Task.Run(()=> job2());
exitPopup();
//do task in UI thread...
For bonus points, have the job1() and job2() methods return the result that you check (per the commend in your code), and use something like var result = await Task.Run(() => job1()); to retrieve the value (instead of whatever mechanism you're using now).

In some cases TaskScheduler may decide to execute Task in the current thread, e.g. read the following example from Jeffrey Richter's book CLR via c#:
When a thread calls the Wait method, the system checks if
the Task that the thread is waiting for has started executing. If it
has, then the thread calling Wait will block until the Task has
completed running. But if the Task has not started executing yet, then
the system may (depending on the TaskScheduler) execute the Task using
the thread that called Wait. If this happens, then the thread calling
Wait does not block; it executes the Task and returns immediately
I believe something similar happens in your logic. TaskScheduler just follows your task queue and decides to execute it syncronously. The easiest way out is to start task2 in default scheduler
task2.Start(TaskScheduler.Default);

TPL Dataflow Blocks Running On UI Thread

I am building a dataflow pipeline to do various processing (mostly I/O, but some CPU processing) that is in a naturally occurring flow. The flow is currently in this basic pattern:
Load Data from File
Parse Record using Transform Block
Serialize & Upload object to server via REST
This processing pipeline can be started automatically, or via a GUI. For when it's started from the GUI I would like to provide progress messages to the end-user. If I add a BufferBlock between step 1 & 2 and an ActionBlock after step 3 and set the options for them to run on the same thread as the UI, will the other blocks still run off the UI using their own threadpool?
I was looking at this MSDN article: http://msdn.microsoft.com/en-us/library/hh228605(v=vs.110).aspx but it wasn't very clear regarding this behavior. Can I fire an event from the pipeline that can run on the UI thread as well to accomplish this?
EDIT: The pipeline would be started from a BackgroundWorker object on the UI, and not the UI thread directly.

Thanks to Noseratio's suggestions I actually redesigned how this done and was able to have it work without an issue. I eliminated the BackgroundWorker object since it wasn't really necessary. Instead I wrapped the entire dataflow in an asynchronous method, which takes various Progress<T> parameters as callbacks for progress updates. There were no extra TPL Blocks used in order to post progress since the Progress<T>'s Report() method is called within preexisting blocks. Additionally, since this is an async function the Task that represents the dataflow is not running on the UI thread, but instead on a threadpool thread. The main thing to take from this is that the Progress<T> objects's callbacks are run on the thread they are created on since during construction they capture the current synchronization context. Here is an example of what resolved my issue:
public static async Task ProcessData(IProgress<int> ReadProg, IProgress<int> UploadProg)
{
var loadBuffer = new BufferBlock<string>();
var parseBlock = new TransformBlock<string, MyObject>(async s =>
{
if(await DoSomething(s))
ReadProg.Report(1);
else
ReadProg.Report(-1);
});
...
//setup of other blocks
//link blocks
//feed data into pipeline by adding data into the head block: loadBuffer
//await ALL continuation tasks of the blocks
}
Then within the UI I created the Progress<int> objects and passed them into the async ProcessData method. Whenever the Process<T>.Report() method were called in the async processing method the UI updated without issue.

Aysnc/Await Vs. Task/Continuation UI Control Access

All, I have a situation where I have been asked to multi-thread a large 'Cost-Crunching' algorithm. I am relatively experienced with Tasks and would be confident in adopting a pattern like
CancellationTokenSource cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();
Task<bool> asyncTask = null;
asyncTask = Task.Factory.StartNew<bool>(() =>
SomeMethodAsync(uiScheduler, token, _dynamic), token);
asyncTask.ContinueWith(task =>
{
// For call back, exception handling etc.
}, uiScheduler);
and then for any operation where I need to provide and UI operation, I would use
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
Where this might be wrapped up in a method.
Now, I realise that I can make all this much less complicated, and leverage the async/await keywords of .NET 4.5. However, I have some questions: if I have a long running method that I launch using
// Start processing asynchroniously.
IProgress<CostEngine.ProgressInfo> progressIndicator =
new Progress<CostEngine.ProgressInfo>();
cancelSource = new CancellationTokenSource();
CancellationToken token = cancelSource.Token;
CostEngine.ScriptProcessor script = new CostEngine.ScriptProcessor(this);
await script.ProcessScriptAsync(doc, progressIndicator, token);
where CostEngine.ProgressInfo is some basic class used to return progress information and the method ProcessScriptAsync is defined as
public async Task ProcessScriptAsync(SSGForm doc, IProgress<ProgressInfo> progressInfo,
CancellationToken token, bool bShowCompleted = true)
{
...
if (!await Task<bool>.Run(() => TheLongRunningProcess(doc)))
return
...
}
I have two questions:
To get ProcessScriptAsync to return control to the UI almost immediately I await on a new Task<bool> delegate (this seemingly avoids an endless chain of async/awaits). Is this the right way to call ProcessScriptAsync? ['Lazy Initialisation', by wrapping in an outer method?]
To access the UI from within TheLongRunningProcess, do I merely pass in the UI TaskScheduler uiScheduler; i.e. TheLongRunningProcess(doc, uiScheduler), then use:
Task task = Task.Factory.StartNew(() =>
{
mainForm.progressLeftLabelText = _strProgressLabel;
}, CancellationToken.None,
TaskCreationOptions.None,
uiScheduler);
as before?
Sorry about the length and thanks for your time.

It depends. You've shown a lot of code, and yet omitted the one bit that you're actually asking a question about. First, without knowing what the code is we can't know if it's actually going to take a while or not. Next, if you await on a task that's already completed it will realize this, and not schedule a continuation but instead continue on (this is an optimization since scheduling tasks is time consuming). If the task you await isn't completed then the continuation will still be executed in the calling SynchronizationContext, which will again keep the UI thread busy. You can use ConfigureAwait(false) to ensure that the continuation runs in the thread pool though. This should handle both issues. Note that by doing this you can no longer access the UI controls in the ... sections of ProcessScriptAsync (without doing anything special). Also note that since ProcessScriptAsync is now executing in a thread pool thread, you don't need to use Task.Run to move the method call to a background thread.
That's one option, yes. Although, if you're updating the UI based on progress, that's what IProgress is for. I see you're using it already, so that is the preferable model for doing this. If this is updating a separate type of progress than the existing IProgress you are passing (i.e. the status text, rather than the percent complete as an int) then you can pass a second.

I think trying to switch back and forth between a background thread (for CPU intensive operations or IO operations with no async support) and the UI thread (to manipulate UI controls) is often a sign of bad design. Your calculations and your UI code should be separate.
If you're doing this just to notify the UI of some sort of progress, then use IProgress<T>. Any marshaling between threads then becomes the responsibility of the implementation of that interface and you can use Progress<T>, which does it correctly using the SynchronizationContext.
If you can't avoid mixing background thread code and UI thread code and your UI work isn't progress reporting (so IProgress<T> won't fit), I would probably enclose each bit of background thread code into its own await Task.Run(), and leave the UI code top level.
Your solution of using a single Task.Run() to run the background thread code and then switch to the UI thread using StartNew() with uiScheduler will work too. In that case, some helper methods might be useful, especially if you wanted to use await in the UI code too. (Otherwise, you would have to remember to double await the result of StartNew())
Yet another option would be create a SwitchTo(TaskScheduler) method, which would return a custom awaiter that continues on the given scheduler. Such method was in some of the async CTPs, but it was removed because it was deemed too dangerous when it comes to handling exceptions.