Long story short: in a C# application that works with COM inproc-server (dll), I encounter "0x80010100: System call failed" exception, and in debug mode also ContextSwitchDeadlock exception.
Now more in details:
1) C# app initializes STA, creates a COM object (registered as "Apartment"); then in subscribes to its connection-point, and begins working with the object.
2) At some stage the COM object generates a lot of events, passing as an argument a very big collection of COM objects, which are created in the same apartment.
3) The event-handler on C# side processes the above collection, occasionally calling some methods of the objects. At some stage the latter calls begin to fail with the above exceptions.
On the COM side the apartment uses a hidden window whose winproc looks like this:
typedef std::function<void(void)> Functor;
LRESULT CALLBACK WndProc(HWND hwnd, UINT msg, WPARAM wParam, LPARAM lParam)
{
switch(msg)
{
case AM_FUNCTOR:
{
Functor *f = reinterpret_cast<Functor *>(lParam);
(*f)();
delete f;
}
break;
case WM_CLOSE:
DestroyWindow(hwnd);
break;
default:
return DefWindowProc(hwnd, msg, wParam, lParam);
}
return 0;
}
The events are posted to this window from other parts of the COM server:
void post(const Functor &func)
{
Functor *f = new Functor(func);
PostMessage(hWind_, AM_FUNCTOR, 0, reinterpret_cast<LPARAM>(f));
}
The events are standard ATL CP implementations bound with the actual params, and they boil down to something like this:
pConnection->Invoke(id, IID_NULL, LOCALE_USER_DEFAULT, DISPATCH_METHOD, ¶ms, &varResult, NULL, NULL);
In C# the handler looks like this:
private void onEvent(IMyCollection objs)
{
int len = objs.Count; // usually 10000 - 25000
foreach (IMyObj obj in objs)
{
// some of the following calls fail with 0x80010100
int id = obj.id;
string name = obj.name;
// etc...
}
}
==================
So, can the above problem happen just because the message-queue of the apartment is too loaded with the events it tries to deliver? Or the message loop should be totally blocked to cause such a behaviour?
Lets assume that the message-queue has 2 sequential events that evaluate to "onEvent" call. The first one enters C# managed code, which attempts to re-enter the unmanaged code, the same apartment. Usually, this is allowed, and we do this a lot. When, under what circumstances can it fail?
Thanks.
This ought to work even with multiple apartments provided that:
Only one of the threads responds to external events such as network traffic, timers, posted messages etc.
Other threads only service COM requests (even if they call back to the main thread during the processing).
AND
neither thread queue ever gets full, preventing COM from communicating with the thread.
Firstly:
It looks like some objects are not in the same apartment as other objects. Are you sure that all objects are being created in the STA?
What you are describing is a classic deadlock - two independent threads, each waiting on the other. That is what I would expect to occur with that design operating with the C# and COM sides on different threads.
You should be OK if all the objects are on the same thread, as well as the hidden window being on that thread, so I think you need to check that. (Obviously this includes any other objects which are created by the COM side and passed over to the C# side.)
You could try debugging this by pressing "pause" in the debugger and checking what code was in each thread (if you see RPCRT*.DLL this means you are looking at a proxy). Alternately you could DebugPrint the current thread ID from various critical points in both C# and COM sides and your WndProc - they should all be the same.
Secondly: it ought to work with multiple threads provided that only one of the threads generates work items, and the other does nothing but host COM objects which respond to calls (i.e. doesn't generate calls from timers, network traffic, posted messages etc), in this case it may be that the thread queue is full and COM cannot reply to a call.
Instead of using the thread queue, you should use a deque protected by a critical section.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms644944(v=vs.85).aspx
There is a limit of 10,000 posted messages per message queue. This limit should be sufficiently large. If your application exceeds the limit, it should be redesigned to avoid consuming so many system resources.
You might maintain a counter of items on/off the queue to see if this is the issue.
Related
I am creating a Windows service in C#. Its purpose is to consume info from a feed on the Internet. I get the data by using zeromq's pub/sub architecture (my service is a subscriber only). To debug the service I "host" it in a WPF control panel. This allows me to start, run, and stop the service without having to install it. The problem I am seeing is that when I call my stop method it appears as though the service continues to write to the database. I know this because I put a Debug.WriteLine() where the writing occurs.
More info on the service:
I am attempting to construct my service in a fashion that allows it to write to the database asynchronously. This is accomplished by using a combination of threads and the ThreadPool.
public void StartDataReceiver() // Entry point to service from WPF host
{
// setup zmq subscriber socket
receiverThread = new Tread(SpawnReceivers);
receiverThread.Start();
}
internal void SpawnReceivers()
{
while(!stopEvent.WaitOne(0))
{
ThreadPool.QueueUserWorkItem(new WaitCallback(ProcessReceivedData), subscriber.Recv()); // subscriber.Recv() blocks when there is no data to receive (according to the zmq docs) so this loop should remain under control, and threads only created in the pool when there is data to process.
}
}
internal void ProcessReceivedData(Object recvdData)
{
// cast recvdData from object -> byte[]
// convert byte[] -> JSON string
// deserialize JSON -> MyData
using (MyDataEntities context = new MyDataEntities())
{
// build up EF model object
Debug.WriteLine("Write obj to db...");
context.MyDatas.Add(myEFModel);
context.SaveChanges();
}
}
internal void QData(Object recvdData)
{
Debug.WriteLine("Queued obj in queue...");
q.Enqueue((byte[])recvdData);
}
public void StopDataReceiver()
{
stopEvent.Set();
receiverThread.Join();
subscriber.Dispose();
zmqContext.Dispose();
stopEvent.Reset();
}
The above code are the methods that I am concerned with. When I debug the WPF host, and the method ProcessReceivedData is set to be queued in the thread pool everything seems to work as expected, until I stop the service by calling StopDataReceiver. As far as I can tell the thread pool never queues any more threads (I checked this by placing a break point on that line), but I continue to see "Write obj to db..." in the output window and when I 'Break All' in the debugger a little green arrow appears on the context.SaveChanges(); line indicating that is where execution is currently halted. When I test some more, and have the thread pool queue up the method QData everything seems to work as expected. I see "Queued obj in queue..." messages in the output window until I stop the service. Once I do no more messages in the output window.
TL;DR:
I don't know how to determine if the Entity Framework is just slowing things way down and the messages I am seeing are just the thread pool clearing its backlog of work items, or if there is something larger at play. How do I go about solving something like this?
Would a better solution be to queue the incoming JSON strings as byte[] like I do in the QData method then have the thread pool queue up a different method to work on clearing the queue. I feel that that solution will only shift the problem around and not actually solve it.
Could another solution be to write a new service dedicated to clearing that queue? The problem I see with writing another service would be that I would probably have to use WCF (or possibly zmq) to communicate between the two services which would obviously add overhead and possibly become less performant.
I see the critical section in all of this being the part of getting the data off the wire fast enough because the publisher I am subscribed to is set to begin discarding messages if my subscriber can't keep up.
I have the following code that uses new .NET 4.5 multi-threading functionality.
Action2 is a call to a windows API library MLang through Interop.
BlockingCollection<int> _blockingCollection= new BlockingCollection<int>();
[Test]
public void Do2TasksWithThreading()
{
Stopwatch stopwatch = new Stopwatch();
stopwatch.Start();
var tasks = new List<Task>();
for (int i = 0 ; i < Environment.ProcessorCount; i++)
{
tasks.Add((Task.Factory.StartNew(() => DoAction2UsingBlockingCollection(i))));
}
for (int i = 1; i < 11; i++)
{
DoAction1(i);
_blockingCollection.Add(i);
}
_blockingCollection.CompleteAdding();
Task.WaitAll(tasks.ToArray());
stopwatch.Stop();
Console.WriteLine("Total time: " + stopwatch.ElapsedMilliseconds + "ms");
}
private void DoAction2UsingBlockingCollection(int taskIndex)
{
WriteToConsole("Started wait for Action2 Task: " + taskIndex);
int index;
while (_blockingCollection.Count > 0 || !_blockingCollection.IsAddingCompleted)
{
if (_blockingCollection.TryTake(out index, 10))
DoAction2(index);
}
WriteToConsole("Ended wait for Action2 Task: " + taskIndex);
}
private void DoAction2()
{
... Load File bytes
//Call to MLang through interop
Encoding[] detected = EncodingTool.DetectInputCodepages(bytes[], 1);
... Save results in concurrent dictionary
}
I did some testing with this code and increasing number of threads from 1 to 2 to 3, etc.. doesn't make process run any faster. It looks like the the threads are waiting for interop call to finish, which makes me think that it is using single thread for some reason.
Here is the definition of Interop method:
namespace MultiLanguage
{
using System;
using System.Runtime.CompilerServices;
using System.Runtime.InteropServices;
using System.Security;
[ComImport, InterfaceType((short) 1), Guid("DCCFC164-2B38-11D2-B7EC-00C04F8F5D9A")]
public interface IMultiLanguage2
[MethodImpl(MethodImplOptions.InternalCall, MethodCodeType=MethodCodeType.Runtime)]
void DetectInputCodepage([In] MLDETECTCP flags, [In] uint dwPrefWinCodePage,
[In] ref byte pSrcStr, [In, Out] ref int pcSrcSize,
[In, Out] ref DetectEncodingInfo lpEncoding,
[In, Out] ref int pnScores);
I there anything that can be done to make this use multiple threads? The only thing I noticed that would require single thread is MethodImplOptions.Synchronized, but that's not being used in this case.
The code for EncodingTools.cs was taken from here:
http://www.codeproject.com/Articles/17201/Detect-Encoding-for-In-and-Outgoing-Text
... Load File bytes
Threads can speed up your program when your machine has multiple processor cores, easy to get these days. Your program is however liable to spend a good bit of time on this invisible code, disk I/O is very slow compared to the raw processing speed of a modern processor. And you still have only a single disk, there is no concurrency at all. Threads will just wait their turn to read data from the disk.
[ComImport, InterfaceType((short) 1), Guid("DCCFC164-2B38-11D2-B7EC-00C04F8F5D9A")]
public interface IMultiLanguage2
This is a COM interface, implemented by the CMultiLanguage coclass. You can find it back in the registry with Regedit.exe, the HKEY_LOCAL_MACHINE\SOFTWARE\Classes\CLSID\{275C23E2-3747-11D0-9FEA-00AA003F8646} key contains the configuration for this coclass. Threading is not a detail left up to the client programmer in COM, a COM coclass declares what kind to threading it supports with the ThreadingModel key.
The value for CMultiLanguage is "Both". Which is good news, but it now greatly matters exactly how you created the object. If the object is created on an STA thread, the default for the main thread in a Winforms or WPF project, then COM ensures all the code stays thread-safe by marshaling interface method calls from your worker thread to the STA thread. That will cause loss of concurrency, the threads take their turn entering the single-threaded apartment.
You can only get concurrency when the object was created on an MTA thread. The kind you get from a threadpool thread or your own Thread without a call to its SetApartmentState() method. An obvious approach to ensure this is to create the CMultiLanguage object on the worker thread itself and avoid having these worker threads shared the same object.
Before you start fixing that, you first need to identify the bottleneck in the program. Focus on the file loading first and make sure you get a realistic measurement, avoid running your test program on the same set of files over and over again. That gives unrealistically good results since the file data will be read from the file system cache. Only the first test after a reboot or file system cache reset gives you a reliable measurement. The SysInternals' RamMap utility is very useful for this, use its Empty + Empty Standby List menu command before you start a test to be able to compare apples to apples.
If that shows that the file loading is the bottleneck then you are done, only improved hardware can solve that. If however you measure that IMultiLanguage2 calls then focus on the usage of the CMultiLanguage object. Without otherwise a guarantee that you can get ahead, a COM server typically provides thread-safety by taking care of the locking for you. Such hidden locking can ruin your odds for getting concurrency. The only way to get ahead then is to get the file reading in one thread to overlap with the parsing in another.
Try running nunit-console with parameter /apartment=MTA
Can anyone point me to a good working solution to the following problem?
The application I'm working on needs to communicate over TCP to software running on another system. Some of the requests I send to that system can take a long time to complete (up to 15sec).
In my application I have a number of threads, including the main UI thread, which can access the service which communicates with the remote system. There is only a single instance of the service which is accessed by all threads.
I need to only allow a single request to be processed at a time, i.e. it needs to be serialized, otherwise bad things happen with the TCP comms.
Attempted Solutions so far
Initially I tried using lock() with a static object to protect each 'command' method, as follows:
lock (_cmdLock)
{
SetPosition(position);
}
However I found that sometimes it wouldn't release the lock, even though there are timeouts on the remote system and on the TCP comms. Additionally, if two calls came in from the same thread (e.g. a user double clicked a button) then it would get past the lock - after reading up about locking again I know that the same thread won't wait for the lock.
I then tried to use AutoResetEvents to only allow a single call through at a time. But without the locking it wouldn't work with multiple threads. The following is the code I used to send a command (from the calling thread) and process a command request (running in the background on its own thread)
private static AutoResetEvent _cmdProcessorReadyEvent = new AutoResetEvent(false);
private static AutoResetEvent _resultAvailableEvent = new AutoResetEvent(false);
private static AutoResetEvent _sendCommandEvent = new AutoResetEvent(false);
// This method is called to send each command and can run on different threads
private bool SendCommand(Command cmd)
{
// Wait for processor thread to become ready for next cmd
if (_cmdProcessorReadyEvent.WaitOne(_timeoutSec + 500))
{
lock (_sendCmdLock)
{
_currentCommand = cmd;
}
// Tell the processor thread that there is a command present
_sendCommandEvent.Set();
// Wait for a result from the processor thread
if (!_resultAvailableEvent.WaitOne(_timeoutSec + 500))
_lastCommandResult.Timeout = true;
}
return _lastCommandResult.Success;
}
// This method runs in a background thread while the app is running
private void ProcessCommand()
{
try
{
do
{
// Indicate that we are ready to process another commnad
_cmdProcessorReadyEvent.Set();
_sendCommandEvent.WaitOne();
lock (_sendCmdLock)
{
_lastCommandResult = new BaseResponse(false, false, "No Command");
RunCOMCommand(_currentCommand);
}
_resultAvailableEvent.Set();
} while (_processCommands);
}
catch (Exception ex)
{
_lastCommandResult.Success = false;
_lastCommandResult.Timeout = false;
_lastCommandResult.LastError = ex.Message;
}
}
I haven't tried implementing a queue of command requests as the calling code expects everything to be synchronous - i.e. the previous command must have completed before I sent the next one.
Additional Background
The software running on the remote system is a 3rd party product and I don't have access to it, it is used to control a laser marking machine with an integrated XY table.
I'm actually using a legacy VB6 DLL to communicate with the laser as it has all the code for formatting commands and processing the responses. This VB6 DLL uses a WinSock control for the comms.
I'm not sure why a queueing solution wouldn't work.
Why not put each request, plus details for a callback with result, on a queue ? Your application would queue these requests, and the module interfacing to your 3rd party system can take each queue item in turn, process, and return the result.
I think it's a cleaner separation of concerns between modules rather than implementing locking around request dispatch etc. Your requestor is largely oblivious of the serialisation constraints, and the 3rd-party interfacing module can look after serialisation, managing timeouts and other errors etc.
Edit: In the Java world we have BlockingQueues which are synchronised for consumers/publishers and make this sort of thing quite easy. I'm not sure if you have the same in the C# world. A quick search suggests not, but there's source code floating around for this sort of thing (if anyone in the C# world can shed some light that would be appreciated)
I'm working on implementing directshow capability in a C# dll. I'm basing my work off of the C++ based "CameraCapture" example Microsoft provides with the Windows Mobile 6 sdk.
Things were going well (thanks to earlier help on this site), but I've run into a bit of a snag while trying to listen for directshow events in C# land:
I have a thread that loops to listen for dshow events. It waits based on a manual reset event that gets defined here (this is defined at the end of graph initialization: the graph is built, renderstream is called, and controlstream is already blocking dataflow):
DshowRequestMan = new ManualResetEvent(false);
MediaEvent = (IMediaEvent)FilterGraph;
chk(MediaEvent.GetEventHandle(out DshowEventHandle));
DshowRequestMan.Handle = DshowEventHandle; //note: no "SafeHandle" in cf
There are 2 related problems I'm experiencing with this:
When my dshow event handler loop pulls the event via IMediaEvent.GetEvent() using a timeout of 0, I get a "timeout exceeded" hresult (-2147467260) on the third iteration. That third event trigger (and subsequent error) don't occur in the C++ example.
If I ignore the timeout case mentioned above, it will constantly trigger with a 73 event. This kills the processor since the loop basically never pauses.
When the C++ example runs its graph in preview mode, it gets two IMediaEvents: first 13, then 73. After that, it stops triggering until actual capturing is started. My C# version pulls 13, then 73, then 73 again, but with the timeout error.
In short, it seems like that third triggering of the DshowRequestMan shouldn't be happening because there is no dshowevent to actually "get", hence, the timeout.
Not sure what I'm doing wrong- I'm calling "FreeEventParams()" with each iteration... I'm suspecting the ManualResetEvent object is being used incorrectly since I'm just assigning something to its handle property, but the same thing happens when I use a Pinvoked "WaitForSingleObject" to listen to the DshowEventHandle... I'm confused at this point.
Any ideas would be greatly appreciated. Thanks in advance!
Here is my code for filter graph event handling:
while (!_stopReceivingEvents) {
IntPtr eventHandle;
var hr = _mediaEvent.GetEventHandle(out eventHandle);
DsError.ThrowExceptionForHR(hr);
//NOTE: Do not close the event handle, because it is used internally by the filter graph
using (var safeWaitHandle = new SafeWaitHandle(eventHandle, false)) {
using (var waitHandle = new AutoResetEvent(false) {SafeWaitHandle = safeWaitHandle}) {
if (WaitHandle.WaitAll(new[] {waitHandle}, 500)) {
//receive all available events
do {
EventCode eventCode;
IntPtr param1;
IntPtr param2;
hr = _mediaEvent.GetEvent(out eventCode, out param1, out param2, 500);
_mediaEvent.FreeEventParams(eventCode, param1, param2);
if (hr == 0) {
switch (eventCode) {
//add handling code here
}
}
} while (hr == 0);
}
}
}
}
Are you making sure to Reset you event after you get and handle it?
Whatr does your actual wait code look like? The problem is likely that you're waiting for 0ms, which basically means "check the event and return immediately." If you want it to block on the wait (like a native call with WAIT_INFINITE) you need to pass in System.Threading.Timeout.Infinite (which is actually -1).
If you want the checking thread to periodically yield (which is way safer than an infinite wait and will actually allow your app to shut down), then you need to check the return from the wait, and if it's a timeout (0x80004004) then just go back and wait again:
while(!shutdown)
{
if(DshowRequestMan.WaitOne(1000))
{
// handle the event
}
}
I have a a C# (FFx 3.5) application that loads DLLs as plug-ins. These plug-ins are loaded in separate AppDomains (for lots of good reasons, and this architecture cannot change). This is all well and good.
I now have a requirement to show a Dialog from one of those plug-ins. Bear in mind that I cannot return the dialog Form to the main application and have it displayed there (the current infrastructure doesn't support it).
Failure 1
In my DLL I created a Form and called Show. The dialog outline showed up but did not paint and it doesn't respond to mouse events. I assumed that this is becasue the DLL is in a separate AppDomain and the message pump for the app is somehow unable to dispatch messages to the new Form.
Failure 2
In my DLL I created a Form and called ShowDialog, which by all rights should create an internal message pump for the dialog.. The dialog is displayed and responded to clicks (hooray), but it appears that the primary app no longer is processing or dispatching windows messages because it quits painting and no longer responds to mouse events. For some reason now it seems that the main app's message pump is not dispatching.
Failure 3
In my DLL I created a Form and called Application.Run. This will certainly create a complete second message pump. I get the same behavior as Failure 2 - the Dialog behaves, but the calling app does not.
Any thoughts on what exactly is going on here and how I might go about showing a dialog from the other AppDomain's DLL and have both the caller and the callee still respond and paint properly?
Try using appdomain1's main form's BeginInvoke with a delegate that displays the form from appdomain2. So in Pseudocode:
Appdomain1:
AppDomain2.DoSomething(myMainForm);
AppDomain2:
DoSomething(Form parent)
{
Form foolishForm = new Form();
parent.BeginInvoke(new Action( delegate { foolishForm.Show(); } ));
}
The code may not be perfect, but it demonstrates the concept.
By the way, if you are having problems passing forms around because of remoting you can:
public class Container<T> : MarshalByRefObject
{
private T _value;
public T Value { get { return _value; } set { _value = value; } }
public Container() { }
public Container(T value) { Value = value; }
public static implicit operator T(Container<T> container)
{
return container.Value;
}
}
That will contain object you throw at it.
We have a very similarly architected application that loads DLL files and plugins. Each DLL file is loaded in a separate application domain, which is created on a separate thread. We have a third-party control in a form that would not appear unless we call System.Windows.Forms.Application.DoEvents() regularly.
Pseudo code:
<In new thread>
<Application domain created. Start called inside new application domain.>
<Start loads new DLL file, calls init function in DLL file>
<Start loops, calling DoEvents until the DLL file exits>
<Application domain unloaded>
<Thread exits>
This solved all of our GUI issues.
One thing that I've used before is implementing a DomainManager. It's possible to customize the various application domain security/binding/context's to handle complex or chicken-egg type problems with respect to pumping your data where you want ;)
I've ususally done this from a native.exe, bootstrapping the CLR through the COM interfaces (psudo code but the order and method names are correct ;):
CorBindToRuntimeEx()
SetHostControl()
GetCLRControl()
SetAppDomainManagerType("yourdomainmanger","info")
// Domain manager set before starting runtime
Start()
HostControl -- GetDomainManagerForDefaultDomain()
DomainManager -- Run()
Your domain manager can be any CLR class library, so their's not that much more native C.
A side note, if you were in WPF; I really like using the "Microsoft.DwayneNeed.Controls" method. Where you may have disperate threads with their own Dispatcher pump in the same UI control (not needing to resort to entirely new Window()'s).
The unique thing about using this approach, is that even if the primary UI thread is blocked/busy (some heavy operation, scanning the filesystem, etc...), these other threads may paint/update their UIElement's without any hiccup.