I have an Excel workbook object containing one sheet, that I want to copy its content into a List.
I have this method:
private Task GeneratePagesList()
{
_pages = new List<Model.Page>();
short idCount = 0;
var generatePagesListTask = new Task(() =>
{
_pages.Add(new Model.Page()
{
Url = new Uri(_worksheetRange.Cells[i, j].Value2.ToString(),
UriKind.RelativeOrAbsolute),
Id = idCount
});
});
return generatePagesListTask;
}
Now I want to consume this method and the Task that it returns as follows:
public async void ConvertExelDataAsync()
{
var generatePagesListTask = GeneratePagesList();
generatePagesListTask.Start();
await generatePagesListTask;
}
When I run, The operation takes too long, and it never quits the ConvertExelDataAsync method, after a little while (that apparently is 60 sec), I receive an Exception that says:
Managed Debugging Assistant 'ContextSwitchDeadlock' has detected a problem in 'C:\Users\Aymen\Documents\Visual Studio
2013\Projects\WebGraphMaker\WebGraphMaker\bin\Debug\WebGraphMaker.vshost.exe'.
Additional information: The CLR has been unable to transition from COM context 0xd33a5e78 to COM context 0xd33a5fa0 for 60 seconds. The
thread that owns the destination context/apartment is most likely
either doing a non pumping wait or processing a very long running
operation without pumping Windows messages. This situation generally
has a negative performance impact and may even lead to the application
becoming non responsive or memory usage accumulating continually over
time. To avoid this problem, all single threaded apartment (STA)
threads should use pumping wait primitives (such as
CoWaitForMultipleHandles) and routinely pump messages during long
running operations.
Notice: This is the first time I interact with Com objects.
Update 1 :
The Excel consumption works just fine when it's not in a task, once in a task, and the task starts, the problem occurs !
Update 2 : when debugging, once the debugger reaches the line
int rowCount = _worksheetRange.Rows.Count;
It exits and nothing happen, can't explain it.
Update 3 :
After opening the Debug>Windows>Threads, it shows this :
The Convert method calls all what's above, defined like this :
public static async void Convert()
{
var excelDataConverter = new ExcelDataConverter(ExcelDataReader.ReadData());
excelDataConverter.ConvertExelDataAsync();
}
To add to #StepehCleary's answer, the message itself is quite informative:
To avoid this problem, all single threaded apartment (STA) threads
should use pumping wait primitives (such as CoWaitForMultipleHandles)
and routinely pump messages during long running operations.
You have a COM proxy to an out-of-proc Excel COM object, the proxy was created on your main thread (probably an STA UI thread). Then you're accessing it on a worker pool thread (which is an MTA thread).
While the COM proxy object itself may be thread-safe for calls from a worker thread like this, under the hood it most likely trying to marshal the call back to the main thread where the proxy was originally created on. That's where the deadlock occurs.
To stay on the safe side, I suggest you create a dedicated STA thread which does pump messages, create all your COM objects on that threads and call out there.
I have two helper classes for this, ThreadAffinityTaskScheduler and ThreadWithAffinityContext, available here, they should work in any execution environment.
When you're working with async and await, there are some general best practices. One is returning "hot" (running) tasks, so do not use new Task or call Task.Start; use Task.Run instead. Another is to avoid async void; use async Task instead.
However, the core problem is as #HansPassant pointed out: you are creating an STA COM object (the Excel stuff) and then accessing it from a thread pool thread after blocking the STA thread. This is doomed to fail.
Instead, just remove all the async and Task code, and create your list on the STA thread.
Related
I read many articles said that async/await doesn't create additional threads.
But the message from Output and Thread windows in debug mode of Visual Studio said the contrary.
I created a very simple example windows form with some code;
private void button2_Click(object sender, EventArgs e)
{
Task t = methodAsync();
//t.Wait();
}
async Task methodAsync()
{
Console.WriteLine($"==before DownloadStringTaskAsync");
using (var wc = new System.Net.WebClient())
{
string content = await wc.DownloadStringTaskAsync("https://stackoverflow.com");
}
Console.WriteLine($"==after DownloadStringTaskAsync");
}
I start app in debuging mode, I pause it by clicking pause button on Debug toolbar. Threads windows show there is only one Main thread, that's normal so far.
Then I click on button to execute methodAsync. When it complete DownloadString, I pause app again, and then I see serveral additional thread in Thread windows.
After about 10 seconds the Output windows shows message "The thread xxx has exited with code 0 (0x0)".
The same result when I replace WebClient.DownloadStringTaskAsync with await Task.Delay(xxx)
I wonder if async/await does really create new thread or not.
Any explaination?
async and await are just keywords that make a method awaitable, and then allow you to asynchronously wait for it and resume execution. Tasks are the underlying framework elements that represent the asynchronous result of the execution of the method, and the TaskScheduler is responsible for coordinating the execution of Tasks, which may involve using the Thread Pool, creating new threads, etc. The default Task Scheduler on Windows generally uses the Thread Pool to execute tasks.
The WebClient.DownloadStringTaskAsync method uses Task-based Asynchronous Pattern and uses resource thread resources that are automatically allocated from the thread pool.
When you implement a TAP method, you can determine where asynchronous
execution occurs. You may choose to execute the workload on the thread
pool, implement it by using asynchronous I/O (without being bound to a
thread for the majority of the operation’s execution), run it on a
specific thread (such as the UI thread), or use any number of
potential contexts.
As you can see in the method definition, it has an attribute ExternalThreading = true signalizing that it might allocate resources on external threads.
I've been working with a third party SDK as a referenced dll from within C# using .NET 4.5.2 on Windows 10. The IDE creates an Interop around the dll and I can see the appropriate namespaces, interfaces, enum, etc. The SDK I am working with is for the Blackmagic ATEM Television Studio, but I don't think that is directly relevant.
The SDK provides an object I can use to locate/discover the attached hardware and returns an instance of an object, which appears to be a wrapper (RCW) around the underlying COM object.
The discovery code looks like this:
string address = "192.168.1.240";
_BMDSwitcherConnectToFailure failureReason = 0;
IBMDSwitcher switcher = null;
var discovery = new CBMDSwitcherDiscovery();
discovery.ConnectTo(address, out switcher, out failureReason);
If I perform this code on my WPF UI thread, which is an STA apartment, it will work without any problem. I am able to use the object without error. However, if I try to access the resulting switcher object from any new thread (STA or MTA) I will receive an error similar to the following:
Unable to cast COM object of type 'System.__ComObject' to interface
type 'BMDSwitcherAPI.IBMDSwitcherMixEffectBlock'. This operation
failed because the QueryInterface call on the COM component for the
interface with IID '{11974D55-45E0-49D8-AE06-EEF4D5F81DF6}' failed due
to the following error: No such interface supported (Exception from
HRESULT: 0x80004002 (E_NOINTERFACE)).
Quoting from MikeJ and his answer to a similar question here:
This nasty, nasty exception arises because of a concept known as COM
marshalling. The essence of the problem lies in the fact that in order
to consume COM objects from any thread, the thread must have access to
the type information that describes the COM object.
At this point I decided to adapt my strategy slightly and try calling the discovery logic from within an MTA thread. Like this:
private IBMDSwitcher _switcher = null;
public void Connect()
{
Task.Run(() =>
{
string address = "192.168.1.240";
IBMDSwitcherDiscovery discovery = new CBMDSwitcherDiscovery();
_BMDSwitcherConnectToFailure failureReason = 0;
discovery.ConnectTo(address, out _switcher, out failureReason);
});
}
This seems to work but at a cost.
The tradeoff is that I can now work with my switcher object from any background (MTA) thread without problem, but my entire application will simply crash after running for approximately 15 minutes. Before someone says it, I know this isn't thread-safe. I recognize the need for some synchronization logic before multiple MTA threads try to use my switcher object. But that's down the road, right now if I run the above code and simply let the application idle for 15 minutes, it will crash, every time.
So on one hand I have a stable app but I can only use the library from the UI thread and on the other hand I have an unstable app but I can use the library from any (MTA) background thread.
Due to the nature of my app, it would be preferable to be able to use the library from multiple background threads. For example, one thread might be generating and uploading graphics to the device, while another thread is managing switching video inputs, and a third thread is managing audio inputs, and the UI thread is handling ID-10T bonk dialogs... All of this could be done from the UI thread, but I'd really rather avoid that if possible.
And so: Is there some way I can discover and create my switcher object on the UI (STA) thread (and theoretically avoid blowing up in 15 minutes) but make the type information accessible to my background (MTA) threads so they can also use the switcher? Or alternately, is it safe to discover/create my switcher object on a background (MTA) thread but I need to do something to 'mark' it in some special way to avoid that 15 minute blow-up?
EDIT:
After reading the comment by Hans Passant and investigating the link he provided, I adapted the referenced STAThread class for my own use in WPF. But I'm a bit fuzzy on SynchronizationContext's and Dispatcher's. Would my code listed below be safe to use as a wrapper 'around' my COM object without blocking the UI during any 'long running' operations? My use case would be to have a Task() running that would prep some data, then Invoke some code to interact with my COM object and when the operation is complete code flow resumes in the running Task to perform the next series of operations...
public class STAThread
{
private Thread thread;
private SynchronizationContext ctx;
private ManualResetEvent mre;
public STAThread()
{
using (mre = new ManualResetEvent(false))
{
thread = new Thread(() =>
{
ctx = new SynchronizationContext();
mre.Set();
Dispatcher.Run();
});
thread.IsBackground = true;
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
mre.WaitOne();
}
}
public void BeginInvoke(Delegate dlg, params Object[] args)
{
if (ctx == null) throw new ObjectDisposedException("STAThread");
ctx.Post((_) => dlg.DynamicInvoke(args), null);
}
public object Invoke(Delegate dlg, params Object[] args)
{
if (ctx == null) throw new ObjectDisposedException("STAThread");
object result = null;
ctx.Send((_) => result = dlg.DynamicInvoke(args), null);
return result;
}
}
I am writing a game, and using OpenGL I require that some work be offloaded to the rendering thread where an OpenGL context is active, but everything else is handled by the normal thread pool.
Is there a way I can force a Task to be executed in a special thread-pool, and any new tasks created from an async also be dispatched to that thread pool?
I want a few specialized threads for rendering, and I would like to be able to use async and await for example for creating and filling a vertex buffer.
If I just use a custom task scheduler and a new Factory(new MyScheduler()) it seems that any subsequent Task objects will be dispatched to the thread pool anyway where Task.Factory.Scheduler suddenly is null.
The following code should show what I want to be able to do:
public async Task Initialize()
{
// The two following tasks should run on the rendering thread pool
// They cannot run synchronously because that will cause them to fail.
this.VertexBuffer = await CreateVertexBuffer();
this.IndexBuffer = await CreateIndexBuffer();
// This should be dispatched, or run synchrounousyly, on the normal thread pool
Vertex[] vertices = CreateVertices();
// Issue task for filling vertex buffer on rendering thread pool
var fillVertexBufferTask = FillVertexBufffer(vertices, this.VertexBuffer);
// This should be dispatched, or run synchrounousyly, on the normal thread pool
short[] indices = CreateIndices();
// Wait for tasks on the rendering thread pool to complete.
await FillIndexBuffer(indices, this.IndexBuffer);
await fillVertexBufferTask; // Wait for the rendering task to complete.
}
Is there any way to achieve this, or is it outside the scope of async/await?
This is possible and basically the same thing what Microsoft did for the Windows Forms and WPF Synchronization Context.
First Part - You are in the OpenGL thread, and want to put some work into the thread pool, and after this work is done you want back into the OpenGL thread.
I think the best way for you to go about this is to implement your own SynchronizationContext. This thing basically controls how the TaskScheduler works and how it schedules the task. The default implementation simply sends the tasks to the thread pool. What you need to do is to send the task to a dedicated thread (that holds the OpenGL context) and execute them one by one there.
The key of the implementation is to overwrite the Post and the Send methods. Both methods are expected to execute the callback, where Send has to wait for the call to finish and Post does not. The example implementation using the thread pool is that Sendsimply directly calls the callback and Post delegates the callback to the thread pool.
For the execution queue for your OpenGL thread I am think a Thread that queries a BlockingCollection should do nicely. Just send the callbacks to this queue. You may also need some callback in case your post method is called from the wrong thread and you need to wait for the task to finish.
But all in all this way should work. async/await ensures that the SynchronizationContext is restored after a async call that is executed in the thread pool for example. So you should be able to return to the OpenGL thread after you did put some work off into another thread.
Second Part - You are in another thread and want to send some work into the OpenGL thread and await the completion of that work.
This is possible too. My idea in this case is that you don't use Tasks but other awaitable objects. In general every object can be awaitable. It just has to implement a public method getAwaiter() that returns a object implementing the INotifyCompletion interface. What await does is that it puts the remaining method into a new Action and sends this action to the OnCompleted method of that interface. The awaiter is expected to call the scheduled actions once the operation it is awaiting is done. Also this awaiter has to ensure that the SynchronizationContext is captured and the continuations are executed on the captured SynchronizationContext. That sounds complicated, but once you get the hang of it, it goes fairly easy. What helped me a lot is the reference source of the YieldAwaiter (this is basically what happens if you use await Task.Yield()). This is not what you need, but I think it is a place to start.
The method that returns the awaiter has to take care of sending the actual work to the thread that has to execute it (you maybe already have the execution queue from the first part) and the awaiter has to trigger once that work is done.
Conclusion
Make no mistake. That is a lot of work. But if you do all that you will have less problem down the line because you can seamless use the async/await pattern as if you would be working inside windows forms or WPF and that is a hue plus.
First, realize that await introduces the special behavior after the method is called; that is to say, this code:
this.VertexBuffer = await CreateVertexBuffer();
is pretty much the same as this code:
var createVertexBufferTask = CreateVertexBuffer();
this.VertexBuffer = await createVertexBufferTask;
So, you'll have to explicitly schedule code to execute a method within a different context.
You mention using a MyScheduler but I don't see your code using it. Something like this should work:
this.factory = new TaskFactory(CancellationToken.None, TaskCreationOptions.DenyChildAttach, TaskContinuationOptions.None, new MyScheduler());
public async Task Initialize()
{
// Since you mention OpenGL, I'm assuming this method is called on the UI thread.
// Run these methods on the rendering thread pool.
this.VertexBuffer = await this.factory.StartNew(() => CreateVertexBuffer()).Unwrap();
this.IndexBuffer = await this.factory.StartNew(() => CreateIndexBuffer()).Unwrap();
// Run these methods on the normal thread pool.
Vertex[] vertices = await Task.Run(() => CreateVertices());
var fillVertexBufferTask = Task.Run(() => FillVertexBufffer(vertices, this.VertexBuffer));
short[] indices = await Task.Run(() => CreateIndices());
await Task.Run(() => FillIndexBuffer(indices, this.IndexBuffer));
// Wait for the rendering task to complete.
await fillVertexBufferTask;
}
I would look into combining those multiple Task.Run calls, or (if Initialize is called on a normal thread pool thread) removing them completely.
I am trying to use tasks in a little .net 4.0 application (written using Visual Studio 2010 if that matters) that needs to work on Windows 2003 and use a WriteableBitmap with the palette parameter.
The code using said class must, therefore, be running as an STA thread to avoid it throwing an invalid cast exception (see here for why I need an STA thread if you are interested, but it is not the thrust of my question).
I, therefore, checked on Stack overflow and came across How to create a task (TPL) running a STA thread? and The current SynchronizationContext may not be used as a TaskScheduler - perfect, so now I know what to do, except...
Here's a little console application:
using System;
using System.Threading;
using System.Threading.Tasks;
namespace TaskPlayingConsoleApplication
{
class Program
{
[STAThread]
static void Main()
{
Console.WriteLine("Before Anything: "
+ Thread.CurrentThread.GetApartmentState());
SynchronizationContext.SetSynchronizationContext(
new SynchronizationContext());
var cts = new CancellationTokenSource();
var scheduler = TaskScheduler.FromCurrentSynchronizationContext();
var task = Task.Factory.StartNew(
() => Console.WriteLine(
"In task: " + Thread.CurrentThread.GetApartmentState()),
cts.Token,
TaskCreationOptions.None,
scheduler);
task.ContinueWith(t =>
Console.WriteLine(
"In continue: " + Thread.CurrentThread.GetApartmentState()),
scheduler);
task.Wait();
}
}
}
And here is its output:
Before Anything: STA
In task: STA
In continue: MTA
What the!?! Yup, it is back to an MTA thread on the Action<Task> passed into the ContinueWith method.
I am passing the same scheduler into the task and the continue but somehow in the continue it seems to be being ignored.
I'm sure it is something stupid, so how would I make sure that my callback passed into the ContinueWith uses an STA thread?
EDIT: before you read any of the following, here's an excellent on-topic article: http://blogs.msdn.com/b/pfxteam/archive/2012/01/20/10259049.aspx ; You can skip my post and go directly there!
Most important part describing the root cause:
The default implementation of SynchronizationContext.Post just turns around and passes it off to the ThreadPool via QueueUserWorkItem. But (...) can derive their own context from SynchronizationContext and override the Post method to be more appropriate to the scheduler being represented.
In the case of Windows Forms, for example, the WindowsFormsSynchronizationContext implements Post to pass the delegate off to Control.BeginInvoke. For DispatcherSynchronizationContext in WPF, it calls to Dispatcher.BeginInvoke. And so on.
So, you need to use something other than the base SynchronizationContext class. Try using any of the other existing ones, or create your own. Example is included in the article.
And now, my original response:
After thinking a bit, I think the problem is that in your console application there is no thing like "message pump". The default SynchronizationContext is just a piece of lock. It prevents threads from intersecting on a resource, but it does not provide any queueing or thread selection. In general you are meant to subclass the SynchroContext to provide your own way of proper synchronization. Both WPF and WinForms provide their own subtypes.
When you Wait on your task, most probably the MainThread gets blocked and all other are run on some random threads from the default threadpool.
Please try writing Thread IDs to the console along with the STA/MTA flag.
You will probably see:
STA: 1111
STA: 1111
MTA: 1234
If you see this, then most probably your first task is run synchronously on the calling thread and gets instantly finished, then you try to "continue" it's just 'appended' to 'the queue', but it is not started immediatelly (guessing, I dont know why so; the old task is finished, so ContinueWith could also just run it synchronously). Then main thread gets locked on wait, and since there's no message pump - it cannot switch to another job and sleeps. Then threadpool waits and sweps the lingering continuation task. Just guessing though. You could try to check this by
prepare synccontext
write "starting task1"
start task1 ( -> write "task1")
write "continuing task2" <--- add this one
continue: task2 ( -> write "task2")
wait
and check the order of messages in the log. Is "continuing" before "hello" from task1 or not?
You may also try seeing what happens if you don't create the Task1 by StartNew, but rather create it as prepared/suspended, then Continue, then start, then wait. If I'm right about the synchronous run, then in such setup main and continuation task will either both be run on the calling '1111' STA thread, or both on threadpool's '2222' thread.
Again, if all of these is right, the providing some message pump and proper SyncContext type will probably solve your issue. As I said, both WPF and WinForms provide their own subtypes. Although I don't remember the names now, you can try using them. If I remember correctly, the WPF starts its dispatcher automatically and you don't need any extra setup. I don't remember how's with WinForms. But, with the WPF's auto-start, if your ConsoleApp is actually some kind of a unit-test that will run many separate cases, you will need to shutdown the WPF's dispatcher before the cases.. but that's far from the topic now.
I having problem managed thread parallel in console application.
I am running 10 threads parallel & all thread doing some specific task.
In case if any task is over/completed then doing stop/end thread and immediate I started new thread instance. I want 10 threads so anyone thread is going to stop/end then It generates new thread. but every time I want 10 threads in running mode in console application & It should be parallel work using C# console application.
How I can running 10 threads in C# console application?
At the end of each thread put a lock on some shared object (lock (obj) {}).
Then remove the current thread from a collection of threads you have.
If the collection.Count is less than 10 create a new one and put inside the collection.
Release the lock.
private List<Thread> threads = new List<Thread>();
private void ThreadFunction() {
// do something
// here before the lock
lock (threads) {
threads.Remove(Thread.CurrentThread);
if (thread.Count < 10) {
Thread t = new Thread(ThreadFunction);
threads.Add(t);
t.Start();
}
}
}
Be sure to catch all exception inside the thread or you code will fail when a thread exception happens. That is make sure that the lock part of the code is always called (except on a Thread abord exception but that will not matter).
But as stated I think you should use a ThreadPool for such a task...
The book on threads in .Net is: http://www.albahari.com/threading/
This alone will probably answer any questions you have.
Depending on what you are using these threads for (I am guessing that you may be talking about running transactions in the background) you may want to use BackgroundWorker.
http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx
BackgroundWorker lets you deal with Begin/End/Progress Events only, making debugging much less error prone.