I am using a third party dll that is exposed via a COM Interop wrapper. However, one of the COM calls often freezes (never returns at least). To try to at least make my code a little more robust, I wrapped the call asynchronously (_getDeviceInfoWaiter is a ManualResetEvent)
var backgroundWorker = new BackgroundWorker();
backgroundWorker.DoWork +=
(sender, eventArgs) =>
{
var deviceInfo = _myCom.get_DeviceInfo(0);
_serialNumber = deviceInfo.SerialNumber;
_getDeviceInfoWaiter.Set();
};
backgroundWorker.RunWorkerAsync();
var waitFifteenSecondsForGetInfo = new TimeSpan(0, 0, 0, 15);
_getDeviceInfoWaiter.WaitOne(waitFifteenSecondsForGetInfo, true);
if(String.IsNullOrEmpty(_serialNumber))
throw new ArgumentNullException("Null or empty serial number. " +
"This is most likely due to the get_DeviceInfo(0) COM call freezing.");
However, the very next call to any COM component will freeze the code. Is there something I am not thinking of, or is there some way to keep my main thread from dying?
UPDATE
Basically, this is a COM call that is called whenever a new device is plugged into the PC so that we can log the information appropriately. However, as I said, ANY COM component will freeze if this one is waiting (a custom COM of our own locks if the third party locked)
UPDATE 2
The above code DOES work, and delays the hanging of the UI thread until the next COM call. The reason for this attempted workaround was because var deviceInfo = _myCom.get_DeviceInfo(0); was already locking the UI thread. However, this info is not important and is only used for logging, so this approach is to allow for "give up and move on after 15 seconds" scenario
Another workaround here would be to find a way to cancel the COM call after x seconds?
UPDATE - after the second update from the OP
IF you have some problematic component you can always make your usage of it more robust by using the following approach:
Create a process (EXE) which wraps the usage of that component and exposes an API (for example via any IPC mechanism). You can then start that EXE as a separate process (from your main EXE) and use it... IF you need to kill that component after a certain time and/or when some condition is met you can always kill that "wrapper EXE" from your main EXE... depending on the specific component it might even be useful to implement some special "cleanup code" (possibly in a separate thread) within that "wrapper EXE" which gets executed when you need to kill that "wrapper EXE".
Since you are implementing this in .NET you can even have that "wrapper EXE" as "embedded resource" in your main executable and start it even from RAM without writing it to the filesystem...
The third party DLL has some kind of indefinite wait, loop or deadlock inside it. Attempts to work around it like this will not work. You may have farmed out the hanging call to a worker thread, but that thread doesn't go away; it keeps hanging in that call.
The next call to the COM component freezes most likely because the previous one froze. Maybe it's trying to acquire a lock that the previous one acquired before hanging. Or maybe it's hanging for exactly the same reason, rather than a dependent reason.
Better contact the developers/vendors of this third party thing and ask them if you're misusing it in some way. Is there some missing precondition. Some initialization that wasn't performed. Some necessary configuration, etc.
Related
Recently I worked with an external dll library where I have no influence on it.
Under some special circumstances, a method of this third party dll is blocking and never returning.
I tried to work around this issue by executing this method in a new AppDomain. After a custom timeout, I wanted to Unload the AppDomain and kill all this crap ;)
Unfortunately, it does not work - as someone would expect.
After some time it throws CannotUnloadAppDomainException since the blocking method does not allow aborting the thread gracefully.
I depend on using this library and it does not seem that there will be an update soon.
So can I work around this issue, even if it's not best practice?
Any bad hack appreciated :)
An AppDomain cannot typically solve that problem, it's only good to throw away the state of your program. The real issue is that your thread is stuck. In cases like these, calling Thread.Abort() is unlikely to work, it will just get stuck as well. A thread can only be aborted if it is a "alertable wait state", blocking on a CLR synchronization object. Or executing managed code. In a state that the CLR knows how to safely clean up. Most 3rd party code falls over like this when executing unmanaged code, no way to ever clean that up in a safe way. A decisive hint that this is the case is AppDomain.Unload failing to get the job done, it can only unload the AppDomain when it can abort the threads that are executing code in the domain.
The only good alternative is to run that code in a separate process. Which you can kill with Process.Kill(). Windows do the cleanup. You'd use a .NET interop mechanism to talk to that code. Like named pipes, sockets, remoting or WCF. Plus the considerable hassle of having to write the code that can detect the timeout, kills the process, starts it back up and recovers internal state since you now restart with an uninitialized instance of that 3rd party code.
Do not forget about the real fix. Create a small repro project that reproduces the problem. When it hangs, create a minidump of the process. Send both to the 3rd party support group.
after reading this (scroll down the end to Blocking Issues) I think your only solution is to run the method in a different process - this might involve quite a bit of refactoring and/or a 'host' project (eg Console application) that loads the method in question and makes it easy to call (eg reading args from command line) when launching the new process using the Process class
You can always use background worker, no need to create a new appdomain. This will ensure that you have complete control over the execution of the thread.
However, there is no way to ensure that you can gracefully abort the thread. As the dll is unmanaged, chances are there that it may cause memory leaks. However, spawning a new thread will ensure that your application does not crash when the Dll does not respond.
I have an application that I am writting that communicates with a third-party program through a COM connection. I was tasked to strengthen the resilency of our application, in order to do that I wait until the third-party application is ready. I do this by waiting until the return value on a method that returns true only when the third-party application has been fully loaded.
In an effort to avoid the condition where the application crashes during this procedure, and my application just sits there waiting throwing COM errors( caught of course ) I was wondering if the Exited event on the Process itself, is guaranteed to always happen?
I am more then willing to provide additional details when its possible. I am also for the time being limited to .NET Framework 2.0 with this current build.
It depends on the sort of crash - some crashes leave the "process" in memory (for example just stuck in some never-ending loop OR that COM object opened some modal error dialog and waits for user)... these cases are not covered by any method mentioned in your description...
In my experience to be really resilient you need a combination:
hook the Exited event (if you started the application via Process) OR find the process ID and monitor that
catch any exceptions from that COM object
implement a timeout + abort logic for calls to that COM object/application
"monitor" the COM object/app
I usually start a separate thread before calling such a COM object... that thread monitors the COM object/application for example if it open any window it is not supposed or memory consumption goes through the roof when it is not supposed to etc. - the things monitored and the reaction to each of them is rather specific to that COM object/application...
I am having problems closing an application that uses WaitForSingleObject() with an INFINITE timout.
The full picture is this. I am doing the following to allow my application to handle the device wakeup event:
Register the event with:
CeRunAppAtEvent("\\\\.\\Notifications\\NamedEvents\\WakeupEvent",
NOTIFICATION_EVENT_WAKEUP);
Start a new thread to wait on:
Thread waitForWakeThread = new Thread(new ThreadStart(WaitForWakeup));
waitForWakeThread.Start();
Then do the following in the target method:
private void WaitForWakeup()
{
IntPtr handle = CreateEvent(IntPtr.Zero, 0, 0, "WakeupEvent");
while (true)
{
WaitForSingleObject(handle, INFINITE);
MessageBox.Show("Wakey wakey");
}
}
This all works fine until I try to close the application when, predictably, WaitForSingleObject continues to wait and does not allow the app to close properly. We only allow one instance of our app to run at a time and we check for this on startup. It appears to continue running until the device is soft reset.
Is there a way to kill the handle that WaitForSingleObject is waiting for, to force it to return?
Many thanks.
Use WaitForMultipleObject instead, and pass 2 handles. The existing one, and one for an event called something like 'exit'. During app shutdown, SetEvent on the exit event, and the WaitForMultipleObject will return and you can get it to exit the thread gracefully.
You need to switch on the return value of WaitForMultipleObject to do the appropriate behaviour depending on which one of the handles was triggered.
Possibly, also, you can set the thread to be a background thread. This will prevent it from stopping your application from shutting down when the main thread terminates.
See:
http://msdn.microsoft.com/en-us/library/system.threading.thread.isbackground.aspx
This is what I would do...
Use the EventWaitHandle class instead of calling CreateEvent directly. There shouldn't be any need to use the Windows API other than CeRunAppAtEvent (and API calls make code ugly...). Get this working first.
Before creating the thread, create a ManualResetEvent variable that is not initially flagged. Call it "TerminateEvent".
Replace the WaitForSingleObject API call with WaitHandle.WaitAny(WaitHandle[]) and pass an array containing "TerminateEvent" and the EventWaitHandle class wrapping the CeRunAppAtEvent notification.
Your loop can use the return value of WaitAny to determine what to do. The return value is the array index of the wait handle that unblocked the thread, so you can determine whether to continue the loop or not.
To cleanly end the thread, you can call "Set" on your "TerminateEvent" and then "Join" the thread to wait for it to terminate.
'This all works fine until I try to close the application when, predictably, WaitForSingleObject continues to wait and does not allow the app to close properly.'
Any app can close, no matter what its threads are doing. If you call ExitProcess(0) from any thread in your app, the app will close, no matter if there are threads waiting INFINITE on some API/sychro, sleeping, running on another processor, whatever. The OS will change the state of all theads that are not running to 'never run again' and use its interprocessor driver to hard-interrupt any other processors that are actually running your thread code. Once all the threads are stopped, the OS frees handles, segments etc and your app no longer exists.
Problems arise when developers try to 'cleanly' shut down threads that are stuck - like yours, when the app is closing. So..
Do you have a TThread.WaitFor, or similar, in an OnClose/OnCloseQuery handler, FormDestroy or destructor? If you have, and have no vital reason to ensure that the thread is terminated, just comment it out!
This allows the main form to close and so your code will finally reach the ExitProcess() it has been trying to get at since you clicked on the red cross button
You could, of coure, just call ExitProcess() yourself, but this may leave you with resources leaked in other proceses - database connections, for example.
'216/217 errors on close if I don't stop the threads'. This often happens because developers have followed the er... 'unfortunate' Delphi thread examples and communicate with threads by directly exchanging data between secondary thread fields and main thread fields, (eg. TThread.synchronize). This just sucks and is hell-bent on causing problems, even in the app run, never mind at shutdown when a form has been destroyed and a thread is trying to write to it or a thread has been destroyed and a main-thread form is trying ot call methods on it. It is much safer to communicate asynchronously with threads by means of queueing/PostMessaging objects that outlive both of them, eg. objects created in the thread/form and freed in the form/thread, or by means of a (thread-safe), pool of objects created in an initialization section. Forms can then close/free safely while associated threads may continue to pointlessly fill up objects for handling until the main form closes, ExitProcess() is reached and the OS annihilates the threads.
'My Form handle is invalid because it has closed but my thread tries to post a message to it'. If the PostMessage excepts, exit your thread. A better way is similar to the approach above - only post messages to a window that outlives all forms. Create one in an initialization section with a trivial WndProc that only handles one const message number that all threads use for posting. You can use wParam to pass the TwinControl instance that the thread is trying to communicate with, (usually a form variable), while lParam passes the object being communicated. When it gets a message from a thread, WndProc calls 'Peform' on the TwinControl passed and the TwinControl will get the comms object in a message-handler. A simple global boolean, 'AppClosing', say, can stop the WndProc calling Peform() on TwinControls that are freeing themselves during shutdown. This approach also avoids problems arising when the OS recreates your form window with a different handle - the Delphi form handle is not used and Windows will not recreate/change the handle of the simple form created in initialization.
I have followed these approaches for decades and do not get any shutdown problems, even with apps with dozens of threads slinging objects around on queues.
Rgds,
Martin
Of course the preferable way to solve this is to use WaitForMultipleObjects, or any other suitable function that is able to wait for multiple criterias (such as WaitForMultipleObjects, MsgWaitForMultipleObjects, etc.).
However if you have no control over which function is used - there're some tricky methods to solve this.
You may hack the functions imported from system DLL, by altering in memory the import table of any module. Since WaitForMultipleObjects is exported from kernel32.dll - it's ok.
using this technics you may redirect the function caller into your hands, and there you will be able to use the WaitForMultipleObjects.
I have a c# .NET multi-threaded application that is freezing the interface. What is unusual about this is that the interface does not freeze unless I let the system sit idle long enough for the screen saver to start (which requires me to reenter my password to re-gain access to the system). When the interface becomes visible again (after I have successfully entered my password) the interface is locked up. As long as I don't let the screensaver start, then the interface does not lockup.
I should point out that I have two different executables that access the same dll and this problem is occurring no matter which application I use to access the DLL. This seems to imply that the problem is in the DLL as the two applications are completely different (C++/MFC) and (C#/.NET) apart from how they relate to the DLL.
Both exes perform similar steps in how they interact with the DLL. They make calls into the dll to setup the serial port communication, open a status window in the DLL, start a thread in the DLL to monitor the comm port, and then starts a thread in the main app that monitors a stack in the dll.
When data is obtained from the comm port by the thread in the DLL, it is parsed and its results are placed on the stack and then posted to the status window via a delegate. When the thread in the exe sees data in the stack, it outputs the data in the main window, also using a delegate.
I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal.
I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock.
This issue occurs both on my development machine and on a test machine.
I have tried completely removing the output of data to the status window inside the DLL, but that didn't make any difference.
I have been doing multi-threaded programming for years and never seen anything like this; so any advice would be greatly appreciated.
Thanks.
This is a problem that's commonly induced by the SystemEvents class when you have a non-standard way to initialize your user interface. Using threads, specifically. Start your program, Debug + Break All, Debug + Windows + Threads. If you see a thread named ".NET SystemEvents" then you're pretty much guaranteed to get this hang.
Some background: the SystemEvent class supports both console mode apps and GUI apps. For the latter, it should fire its event handlers on the UI thread. The very first time one of its events is subscribed, it creates a little invisible helper window to get the system notifications. It can do this two ways, either by creating the window on the calling thread or by starting up a helper thread. It makes the decision based on the value of Thread.GetApartmentState(). If it is STA then it can create the window on the calling thread and all event callbacks can be properly marshaled to that thread.
This goes wrong if the first window you create is not created on the UI thread. A splash screen for example. That window may contain controls that are interested in a system event like UserPreferenceChanged so they can properly repaint themselves. It now uses the helper thread and any event will be fired from that helper thread, not the UI thread. Poison to any window that runs on the UI thread. The session switch out of a locked workstation (including the screen saver) is for some mysterious reason very likely to cause deadlock. You may also see an occasional painting mishap, the less nasty result of using windows from the wrong thread.
Short from fixing the initialization order, a workaround is to put this in your Main() method, before any windows are created:
Microsoft.Win32.SystemEvents.UserPreferenceChanged += delegate { };
The problem does appear to be related to the ActiveX control is was probably using incorrectly in a form. I switched to using the serial port library in .NET and have not been able to reproduce my problem. Thanks to everyone, especially Hans for their assistance.
I am having the same issue as my PC just hangs up when the screen saver kicks off or I lock my PC and monitor goes to sleep.
I am 95% sure that there are deadlocks appearing in my multithreaded app. Look and identify whether there are any deadlocks in your code.
I have a c# .NET multi-threaded application that is freezing the interface. What is unusual about this is that the interface does not freeze unless I let the system sit idle long enough for the screen saver to start (which requires me to reenter my password to re-gain access to the system). When the interface becomes visible again (after I have successfully entered my password) all the windows are white. I can see the window titles, move the windows around, minimize them and such, but the screens are not repainting. When I break all and enter the debugger, the call stack has Application.Run(), external code, and then "in a sleep, wait, or join". I put break points in all four of the threads I open and they are still running, it is just the main app's UI thread that is blocked. When I look at my thread list, what was my main thread and my four worker threads now consists of my main thread and 11 worker threads. I didn't open this many threads so it must be the serialport class.
Now let me describe my program.
My main app allows users to collect and monitor data from serial ports. I have implemented this in the following way. When a connection is desired, a button is pressed on the main app which calls a function in a DLL which opens a status window and then launches a thread which monitors the serial port. When that function returns, the main app launches a thread to monitor a queue created in the DLL when it is initialized. When data is received from the serial port, the data is parsed and then the status window is updated (via a delegate) and the data is pushed onto the queue. When the main apps worker thread sees data in the queue it retrieves it and posts it in a list box on the main app, using a delegate. In all cases I use BeginInvoke to call these delegates.
My DLL contains two libraries for the two different types of equipment it can communicate with.
This problem occurs when I have a connection to two devices; hence the four worker threads two for each device. The DLL itself is setup as a comm object so I can access it easily from a C++/MFC app and a c# app, both of which utilize it.
I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal. I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock. This is not a solution, just something of interest.
I would appreciate any suggestions you might have. Thanks.
I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal. I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock. This is not a solution, just something of interest.
I would recommend running your program under the new Visual Studio 2010 Concurrency Profiler. This will show you, at runtime, which threads are blocked, and which objects they are waiting on. Thread contention is explicitly marked and highlighted for you.
You can use this to easily determine what code is causing the deadlock on your UI thread.
Try changing your Thread Start code to Thread.Start() instead of BeginInvoke(). BeginInvoke does not keep threads tryky seperate from your UI, as it and it may be interacting strangely with DoEvents. You can read up on BeginInvoke and how it works here: http://www.codeproject.com/KB/cs/begininvoke.aspx
Also, DoEvents is NEVER necessary in an application, and can cause a lot of unexpected behavior. Use threadding with UI calls wrapped in a Control.Invoke(...) statement. If you're using .NET 3.5+, you can make this easy with delegates that look like this: Invoke((Action)delegate() {*code goes here*});