We Keep Coding

async/await create new thread, output window show that

I read many articles said that async/await doesn't create additional threads.
But the message from Output and Thread windows in debug mode of Visual Studio said the contrary.
I created a very simple example windows form with some code;
private void button2_Click(object sender, EventArgs e)
{
Task t = methodAsync();
//t.Wait();
}
async Task methodAsync()
{
Console.WriteLine($"==before DownloadStringTaskAsync");
using (var wc = new System.Net.WebClient())
{
string content = await wc.DownloadStringTaskAsync("https://stackoverflow.com");
}
Console.WriteLine($"==after DownloadStringTaskAsync");
}
I start app in debuging mode, I pause it by clicking pause button on Debug toolbar. Threads windows show there is only one Main thread, that's normal so far.
Then I click on button to execute methodAsync. When it complete DownloadString, I pause app again, and then I see serveral additional thread in Thread windows.
After about 10 seconds the Output windows shows message "The thread xxx has exited with code 0 (0x0)".
The same result when I replace WebClient.DownloadStringTaskAsync with await Task.Delay(xxx)
I wonder if async/await does really create new thread or not.
Any explaination?

async and await are just keywords that make a method awaitable, and then allow you to asynchronously wait for it and resume execution. Tasks are the underlying framework elements that represent the asynchronous result of the execution of the method, and the TaskScheduler is responsible for coordinating the execution of Tasks, which may involve using the Thread Pool, creating new threads, etc. The default Task Scheduler on Windows generally uses the Thread Pool to execute tasks.

The WebClient.DownloadStringTaskAsync method uses Task-based Asynchronous Pattern and uses resource thread resources that are automatically allocated from the thread pool.
When you implement a TAP method, you can determine where asynchronous
execution occurs. You may choose to execute the workload on the thread
pool, implement it by using asynchronous I/O (without being bound to a
thread for the majority of the operation’s execution), run it on a
specific thread (such as the UI thread), or use any number of
potential contexts.
As you can see in the method definition, it has an attribute ExternalThreading = true signalizing that it might allocate resources on external threads.

How can I have two separate task schedulers?

I am writing a game, and using OpenGL I require that some work be offloaded to the rendering thread where an OpenGL context is active, but everything else is handled by the normal thread pool.
Is there a way I can force a Task to be executed in a special thread-pool, and any new tasks created from an async also be dispatched to that thread pool?
I want a few specialized threads for rendering, and I would like to be able to use async and await for example for creating and filling a vertex buffer.
If I just use a custom task scheduler and a new Factory(new MyScheduler()) it seems that any subsequent Task objects will be dispatched to the thread pool anyway where Task.Factory.Scheduler suddenly is null.
The following code should show what I want to be able to do:
public async Task Initialize()
{
// The two following tasks should run on the rendering thread pool
// They cannot run synchronously because that will cause them to fail.
this.VertexBuffer = await CreateVertexBuffer();
this.IndexBuffer = await CreateIndexBuffer();
// This should be dispatched, or run synchrounousyly, on the normal thread pool
Vertex[] vertices = CreateVertices();
// Issue task for filling vertex buffer on rendering thread pool
var fillVertexBufferTask = FillVertexBufffer(vertices, this.VertexBuffer);
// This should be dispatched, or run synchrounousyly, on the normal thread pool
short[] indices = CreateIndices();
// Wait for tasks on the rendering thread pool to complete.
await FillIndexBuffer(indices, this.IndexBuffer);
await fillVertexBufferTask; // Wait for the rendering task to complete.
}
Is there any way to achieve this, or is it outside the scope of async/await?

This is possible and basically the same thing what Microsoft did for the Windows Forms and WPF Synchronization Context.
First Part - You are in the OpenGL thread, and want to put some work into the thread pool, and after this work is done you want back into the OpenGL thread.
I think the best way for you to go about this is to implement your own SynchronizationContext. This thing basically controls how the TaskScheduler works and how it schedules the task. The default implementation simply sends the tasks to the thread pool. What you need to do is to send the task to a dedicated thread (that holds the OpenGL context) and execute them one by one there.
The key of the implementation is to overwrite the Post and the Send methods. Both methods are expected to execute the callback, where Send has to wait for the call to finish and Post does not. The example implementation using the thread pool is that Sendsimply directly calls the callback and Post delegates the callback to the thread pool.
For the execution queue for your OpenGL thread I am think a Thread that queries a BlockingCollection should do nicely. Just send the callbacks to this queue. You may also need some callback in case your post method is called from the wrong thread and you need to wait for the task to finish.
But all in all this way should work. async/await ensures that the SynchronizationContext is restored after a async call that is executed in the thread pool for example. So you should be able to return to the OpenGL thread after you did put some work off into another thread.
Second Part - You are in another thread and want to send some work into the OpenGL thread and await the completion of that work.
This is possible too. My idea in this case is that you don't use Tasks but other awaitable objects. In general every object can be awaitable. It just has to implement a public method getAwaiter() that returns a object implementing the INotifyCompletion interface. What await does is that it puts the remaining method into a new Action and sends this action to the OnCompleted method of that interface. The awaiter is expected to call the scheduled actions once the operation it is awaiting is done. Also this awaiter has to ensure that the SynchronizationContext is captured and the continuations are executed on the captured SynchronizationContext. That sounds complicated, but once you get the hang of it, it goes fairly easy. What helped me a lot is the reference source of the YieldAwaiter (this is basically what happens if you use await Task.Yield()). This is not what you need, but I think it is a place to start.
The method that returns the awaiter has to take care of sending the actual work to the thread that has to execute it (you maybe already have the execution queue from the first part) and the awaiter has to trigger once that work is done.
Conclusion
Make no mistake. That is a lot of work. But if you do all that you will have less problem down the line because you can seamless use the async/await pattern as if you would be working inside windows forms or WPF and that is a hue plus.

First, realize that await introduces the special behavior after the method is called; that is to say, this code:
this.VertexBuffer = await CreateVertexBuffer();
is pretty much the same as this code:
var createVertexBufferTask = CreateVertexBuffer();
this.VertexBuffer = await createVertexBufferTask;
So, you'll have to explicitly schedule code to execute a method within a different context.
You mention using a MyScheduler but I don't see your code using it. Something like this should work:
this.factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskContinuationOptions.None, new MyScheduler());
public async Task Initialize()
{
// Since you mention OpenGL, I'm assuming this method is called on the UI thread.
// Run these methods on the rendering thread pool.
this.VertexBuffer = await this.factory.StartNew(() => CreateVertexBuffer()).Unwrap();
this.IndexBuffer = await this.factory.StartNew(() => CreateIndexBuffer()).Unwrap();
// Run these methods on the normal thread pool.
Vertex[] vertices = await Task.Run(() => CreateVertices());
var fillVertexBufferTask = Task.Run(() => FillVertexBufffer(vertices, this.VertexBuffer));
short[] indices = await Task.Run(() => CreateIndices());
await Task.Run(() => FillIndexBuffer(indices, this.IndexBuffer));
// Wait for the rendering task to complete.
await fillVertexBufferTask;
}
I would look into combining those multiple Task.Run calls, or (if Initialize is called on a normal thread pool thread) removing them completely.

How would I use Background Worker to get my GUI to respond?

I made a short program which has just a button. When the button is pressed, functionA is executed, which also uses functionB and functionC. Inside functionA is a loop which executes functionB and functionC X amount of times. At the end of each loop, the progressbar gets incremented by 1. At the beginning of functionA, before the loop, there's a webservice which pulls data from a website, and passes that onto B and C for processing (data file manipulation and saving to disk).
My problem is that everything works fine, but while functionA is still running, the GUI is stuck, so I can't close/minimize/drag the window around, I have to wait until A is done. I researched and they say I should use BackgroundWorker, but as being a new programmer, I've no idea on how to use it. Can someone give me a simple way to use it?
The progressbar loads fine, but it's just that while the function is running, the whole window is frozen, and I want it so I can move the window around, etc while the program is running, instead of waiting until the function is complete.
Thank you!

Call your function asynchronously like the following and it will not freeze the UI.
private async void BeginProcessingAsync(Data d)
{
//Execute the long running task asynchronously
await Task.Run(() => functionA(d));
//Anything after the await line will be executed only after the task is finished.
anotherFunction(d); // if you have one..
}
To run your task, simply call BeginProcessingAsync(d);. Also, please note: If you're using newer versions of .NET, you might have to use await Task.Factory.StartNew(() => functionA(d)); instead of the above

Overall, you'll want to make sure your GUI doesn't get updated from another thread. Instead, the messages should go to a threadsafe location. For instance, you could have the thread building into something like a database and have the GUI using a timer to look for updated data flags.
There is a question with a lot more detail using delegates here.
Marc's answer was the simplest and best, in my opinion:
///...blah blah updating files
string newText = "abc"; // running on worker thread
this.Invoke((MethodInvoker)delegate {
someLabel.Text = newText; // runs on UI thread
});
///...blah blah more updating files
From Dotnet Perls:
A Background Worker makes threads easy to implement in Windows
Forms. Intensive tasks need to be done on another thread so the UI
does not freeze. It is necessary to post messages and update the user
interface when the task is done.
Also, from MSDN, look at Task-based Asynchronous Pattern (TAP) if you're using C# 5.
The Task-based Asynchronous Pattern (TAP) is based on the
System.Threading.Tasks.Task and System.Threading.Tasks.Task
types in the System.Threading.Tasks namespace, which are used to
represent arbitrary asynchronous operations. TAP is the recommended
asynchronous design pattern for new development.

Tasks - how to ensure the ContinueWith action is an STA thread?

I am trying to use tasks in a little .net 4.0 application (written using Visual Studio 2010 if that matters) that needs to work on Windows 2003 and use a WriteableBitmap with the palette parameter.
The code using said class must, therefore, be running as an STA thread to avoid it throwing an invalid cast exception (see here for why I need an STA thread if you are interested, but it is not the thrust of my question).
I, therefore, checked on Stack overflow and came across How to create a task (TPL) running a STA thread? and The current SynchronizationContext may not be used as a TaskScheduler - perfect, so now I know what to do, except...
Here's a little console application:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace TaskPlayingConsoleApplication
{
class Program
{
[STAThread]
static void Main()
{
Console.WriteLine("Before Anything: "
+ Thread.CurrentThread.GetApartmentState());
SynchronizationContext.SetSynchronizationContext(
new SynchronizationContext());
var cts = new CancellationTokenSource();
var scheduler = TaskScheduler.FromCurrentSynchronizationContext();
var task = Task.Factory.StartNew(
() => Console.WriteLine(
"In task: " + Thread.CurrentThread.GetApartmentState()),
cts.Token,
TaskCreationOptions.None,
scheduler);
task.ContinueWith(t =>
Console.WriteLine(
"In continue: " + Thread.CurrentThread.GetApartmentState()),
scheduler);
task.Wait();
}
}
}
And here is its output:
Before Anything: STA
In task: STA
In continue: MTA
What the!?! Yup, it is back to an MTA thread on the Action<Task> passed into the ContinueWith method.
I am passing the same scheduler into the task and the continue but somehow in the continue it seems to be being ignored.
I'm sure it is something stupid, so how would I make sure that my callback passed into the ContinueWith uses an STA thread?

EDIT: before you read any of the following, here's an excellent on-topic article: http://blogs.msdn.com/b/pfxteam/archive/2012/01/20/10259049.aspx ; You can skip my post and go directly there!
Most important part describing the root cause:
The default implementation of SynchronizationContext.Post just turns around and passes it off to the ThreadPool via QueueUserWorkItem. But (...) can derive their own context from SynchronizationContext and override the Post method to be more appropriate to the scheduler being represented.
In the case of Windows Forms, for example, the WindowsFormsSynchronizationContext implements Post to pass the delegate off to Control.BeginInvoke. For DispatcherSynchronizationContext in WPF, it calls to Dispatcher.BeginInvoke. And so on.
So, you need to use something other than the base SynchronizationContext class. Try using any of the other existing ones, or create your own. Example is included in the article.
And now, my original response:
After thinking a bit, I think the problem is that in your console application there is no thing like "message pump". The default SynchronizationContext is just a piece of lock. It prevents threads from intersecting on a resource, but it does not provide any queueing or thread selection. In general you are meant to subclass the SynchroContext to provide your own way of proper synchronization. Both WPF and WinForms provide their own subtypes.
When you Wait on your task, most probably the MainThread gets blocked and all other are run on some random threads from the default threadpool.
Please try writing Thread IDs to the console along with the STA/MTA flag.
You will probably see:
STA: 1111
STA: 1111
MTA: 1234
If you see this, then most probably your first task is run synchronously on the calling thread and gets instantly finished, then you try to "continue" it's just 'appended' to 'the queue', but it is not started immediatelly (guessing, I dont know why so; the old task is finished, so ContinueWith could also just run it synchronously). Then main thread gets locked on wait, and since there's no message pump - it cannot switch to another job and sleeps. Then threadpool waits and sweps the lingering continuation task. Just guessing though. You could try to check this by
prepare synccontext
write "starting task1"
start task1 ( -> write "task1")
write "continuing task2" <--- add this one
continue: task2 ( -> write "task2")
wait
and check the order of messages in the log. Is "continuing" before "hello" from task1 or not?
You may also try seeing what happens if you don't create the Task1 by StartNew, but rather create it as prepared/suspended, then Continue, then start, then wait. If I'm right about the synchronous run, then in such setup main and continuation task will either both be run on the calling '1111' STA thread, or both on threadpool's '2222' thread.
Again, if all of these is right, the providing some message pump and proper SyncContext type will probably solve your issue. As I said, both WPF and WinForms provide their own subtypes. Although I don't remember the names now, you can try using them. If I remember correctly, the WPF starts its dispatcher automatically and you don't need any extra setup. I don't remember how's with WinForms. But, with the WPF's auto-start, if your ConsoleApp is actually some kind of a unit-test that will run many separate cases, you will need to shutdown the WPF's dispatcher before the cases.. but that's far from the topic now.

Signaling the main thread asynchronously

I may be going about this all wrong but I'm stuck. I have a GUI application that spawns a separate thread that downloads a bunch of data from a server. When this download thread is finished I want it to send a signal to the main thread so that it knows it can now display the downloaded data.
I've tried calling Invoke (from my main form) to call a delegate to do the display work, but this blocks my downloader thread until its finished. I kind of want to just do a BeginInvoke without an EndInvoke but I know its not proper to do so.

There are a few options.
My personal favorite is to use the TPL. On your UI thread, you can make a TaskFactory, like so:
// Given:
// TaskFactory uiFactory;
uiFactory = new TaskFactory(TaskScheduler.FromCurrentSynchronizationContext());
Then, in your background task, you can just create a Task to update your UI:
var task = uiFactory.StartNew( () => UpdateUserInterface(data));
This will marshal to the UI thread correctly, similar to a BeginInvoke call. If you need to block, you can call task.Wait() (or task.Result if the Update method returns a value).

There are several options:
For WinForms use the Control.BeginInvoke method.
For WPF use the Dispatcher.BeginInvoke method.
"The TPL has other schedulers in addition to the default one and also allows you to create custom schedulers. One of the schedulers that TPL provides is based on the current synchronization context, and it can be used to ensure that my task executes on the UI thread." (Source article):
var ui = TaskScheduler.FromCurrentSynchronizationContext();
Task.Factory.ContinueWhenAll(tasks.ToArray(),
result =>
{
var time = watch.ElapsedMilliseconds;
label1.Content += time.ToString();
}, CancellationToken.None, TaskContinuationOptions.None, ui);
In the case with download scenario, .ContinueWith() continuation would be appropriate.