Yet another C# Deadlock Debugging Question - c#

I have a multi-threaded application build in C# using VS2010 Professional. It's quite a large application and we've experienced the classing GUI cross-threading and deadlock issues before, but in the past month we've noticed the appears to lock up when left idle for around 20-30 minutes.
The application is irresponsive and although it will repaint itself when other windows are dragged in front of the application and over it, the GUI still appears to be locked... interstingly (unlike if the GUI thread is being used for a considerable amount of time) the Close, Maximise and minimise buttons are also irresponsive and when clicked the little (Not Responding...) text is not displayed in the title of the application i.e. Windows still seems to think it's running fine.
If I break/pause the application using the debugger, and view the threads that are running. There are 3 threads of our managed code that are running, and a few other worker threads whom the source code cannot be displayed for.
The 3 threads that run are:
The main/GUI thread
A thread that loops indefinitely
A thread that loops indefinitely
If I step into threads 2 and 3, they appear to be looping correctly. They do not share locks (even with the main GUI thread) and they are not using the GUI thread at all. When stepping into the main/GUI thread however, it's broken on Application.Run...
This problem screams deadlock to me, but what I don't understand is if it's deadlock, why can't I see the line of code the main/GUI thread is hanging on?
Any help will be greatly appreciated! Let me know if you need more information...
Cheers,
Roo
-----------------------------------------------------SOLUTION--------------------------------------------------
Okay, so the problem is now solved. Thanks to everyone for their suggestions! Much appreciated! I've marked the answer that solved my initial problem of determining where on the main/UI thread the application hangs (I handn't turned off the "Enable Just My Code" option).
The overall issue I was experiencing was indeed Deadlock, however. After obtaining the call-stack and popping the top half of it into Google I came across this which explains exactly what I was experiencing...
http://timl.net/
This references a lovely guide to debugging the issue...
http://www.aaronlerch.com/blog/2008/12/15/debugging-ui/
This identified a control I was constructing off the GUI thread. I did know this, however, and was marshalling calls correctly, but what I didn't realise was that behind the scenes this Control was subscribing to an event or set of events that are triggered when e.g. a Windows session is unlocked or the screensaver exits. These calls are always made on the main/UI thread and were blocking when it saw the call was made on the incorrect thread. Kim explains in more detail here...
http://krgreenlee.blogspot.com/2007/09/onuserpreferencechanged-hang.html
In the end I found an alternative solution which did not require this Control off the main/UI thread.
That appears to have solved the problem and the application no longer hangs. I hope this helps anyone who's confronted by a similar problem.
Thanks again to everyone on here who helped! (and indirectly, the delightful bloggers I've referenced above!)
Roo
-----------------------------------------------------SOLUTION II--------------------------------------------------
Aren't threading issues delightful...you think you've solved it, and a month down the line it pops back up again. I still believe the solution above resolved an issue that would cause simillar behaviour, but we encountered the problem again.
As we spent a while debugging this, I thought I'd update this question with our (hopefully) final solution:
The problem appears to have been a bug in the Infragistics components in the WinForms 2010.1 release (no hot fixes). We had been running from around the time the freeze issue appeared (but had also added a bunch of other stuff too). After upgrading to WinForms 2010.3, we've yet to reproduce the issue (deja vu). See my question here for a bit more information: '.NET 4.0 and the dreaded OnUserPreferenceChanged Hang'. Hans has given a nice summary of the general issue.
I hope this adds a little to the suggestions/information surrounding the nutorious OnUserPreferenceChanged Hang (or whatever you'd like to call it).
Cheers,
Roo

I would check to make sure there is no modal dialog that is being launched behind the main form. This can happen if the dialog is launched and some event causes the main form to steal back focus. You can avoid this by making sure the modal dialog has the main form as its Owner when you create it.

Check where it's hanging in Application.Run; disable Just My Code in Debugger Options, then look at the call stack.
I've fixed an application that deadlocked in Application.Run because it was trying to remove an event handler from an object that a different thread was locked on. (Until the C# 4.0 compiler, event handler accessors implicitly lock(this))
However, if you're using VS2010, that isn't the exact issue.

Related

Identifying UI hang issue

Is there any tool which gives you the view of current execution of you code? I need this because my WPF application's UI is hanging without any notification/exceptions. I am not able to debug the issue. System stays Idle and every time I have to kill the process explicitly. Any suggestions. FYI.. many third party tools are being used.
I am not sure what you have tested already and maybe I answer is invalid but:
Have you tried to use the pause function in Visual Studio. It pauses all threads of your application and you can see per Thread where it is paused. You can also step through each Thread to find possible issues.
Some issues I had in the past:
Too much actions where executed immediately from the UI Thread
Dispatch big chunk of work to background threads using the BackgroundWorker
Incorrect locking model so that the UI thread is not able to gather the information it wants
Make sure data in the ViewModel cannot be changed by background threads without locking properties.
Some interesting links to read about WPF and MultiThreading:
WPF Fundamentals - Threading Model
Debugging locks and deadlocks
Hopefully this helps you a bit
You can use windbg to fully investigate all aspects of a process. While there is a learning curve involved, in my opinion, becoming familiar with this tool is vital for a developer. To get started get a full memory dump of your running process via task manager & download & install debugging tools for windows. Once you've this done take a look at some tutorials & I'll try and help if you post a comment back
If you are doing Memory/CPU Consuming tasks using the main thread the UI will become Idle even without any exceptions.To solve this issue use a BackgroudWorker/a Separate thread to do the time consuming operations.So that UI will not freeze.

Interface freezes in multi-threaded c# application

I have a c# .NET multi-threaded application that is freezing the interface. What is unusual about this is that the interface does not freeze unless I let the system sit idle long enough for the screen saver to start (which requires me to reenter my password to re-gain access to the system). When the interface becomes visible again (after I have successfully entered my password) the interface is locked up. As long as I don't let the screensaver start, then the interface does not lockup.
I should point out that I have two different executables that access the same dll and this problem is occurring no matter which application I use to access the DLL. This seems to imply that the problem is in the DLL as the two applications are completely different (C++/MFC) and (C#/.NET) apart from how they relate to the DLL.
Both exes perform similar steps in how they interact with the DLL. They make calls into the dll to setup the serial port communication, open a status window in the DLL, start a thread in the DLL to monitor the comm port, and then starts a thread in the main app that monitors a stack in the dll.
When data is obtained from the comm port by the thread in the DLL, it is parsed and its results are placed on the stack and then posted to the status window via a delegate. When the thread in the exe sees data in the stack, it outputs the data in the main window, also using a delegate.
I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal.
I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock.
This issue occurs both on my development machine and on a test machine.
I have tried completely removing the output of data to the status window inside the DLL, but that didn't make any difference.
I have been doing multi-threaded programming for years and never seen anything like this; so any advice would be greatly appreciated.
Thanks.
This is a problem that's commonly induced by the SystemEvents class when you have a non-standard way to initialize your user interface. Using threads, specifically. Start your program, Debug + Break All, Debug + Windows + Threads. If you see a thread named ".NET SystemEvents" then you're pretty much guaranteed to get this hang.
Some background: the SystemEvent class supports both console mode apps and GUI apps. For the latter, it should fire its event handlers on the UI thread. The very first time one of its events is subscribed, it creates a little invisible helper window to get the system notifications. It can do this two ways, either by creating the window on the calling thread or by starting up a helper thread. It makes the decision based on the value of Thread.GetApartmentState(). If it is STA then it can create the window on the calling thread and all event callbacks can be properly marshaled to that thread.
This goes wrong if the first window you create is not created on the UI thread. A splash screen for example. That window may contain controls that are interested in a system event like UserPreferenceChanged so they can properly repaint themselves. It now uses the helper thread and any event will be fired from that helper thread, not the UI thread. Poison to any window that runs on the UI thread. The session switch out of a locked workstation (including the screen saver) is for some mysterious reason very likely to cause deadlock. You may also see an occasional painting mishap, the less nasty result of using windows from the wrong thread.
Short from fixing the initialization order, a workaround is to put this in your Main() method, before any windows are created:
Microsoft.Win32.SystemEvents.UserPreferenceChanged += delegate { };
The problem does appear to be related to the ActiveX control is was probably using incorrectly in a form. I switched to using the serial port library in .NET and have not been able to reproduce my problem. Thanks to everyone, especially Hans for their assistance.
I am having the same issue as my PC just hangs up when the screen saver kicks off or I lock my PC and monitor goes to sleep.
I am 95% sure that there are deadlocks appearing in my multithreaded app. Look and identify whether there are any deadlocks in your code.

c# isBackground Thread is NOT terminating properly

I have one background thread I use for processing separately from the GUI, and everything works fine between the threads. However when I close the application, the program "closes" but does not kill the process. The background thread is keeping the program alive it seems.
I have set "myThreadInstance.IsBackground = true;", and I thought this would require C# to kill it when it is killed.
I am testing this all in Visual Studio (2010, using .NET 4.0), and after the first build, the rest all fail because the exe is still in use so it cannot overwrite it. Looking in task manager, it is there. Killing Visual Studio releases the vbhost process which releases my exe. Killing my exe's process, makes vbhost respawn it in a second or two.
Actually, based on your description and other things you've tried (and their results), I believe the most likely cause is this:
You have another foreground thread in your application, other than the one you're examining.
Try using Application.Exit(0); in the form_closing/form_closed event.
Bug: I think this might be a bug. Look at the comments at the bottom of this MSDN page: http://msdn.microsoft.com/en-us/library/system.threading.thread.isbackground.aspx
Also, try using the BackgroundWorker. Here's a good description in the VisualStudio Magazine: http://visualstudiomagazine.com/articles/2007/09/01/simplify-background-threads.aspx
This type of problem usually requires code to figure out. Take your app and trim it down to the bare minimum necessary to show the issue.
However, most likely you are either not signaling the thread to terminate or the thread is such a long running beast that it doesn't ever see the signal.

Knowing At what point the application freezes

I am developing a WinForm multithreaded application with C#.
Sometimes it happens that my application hangs, or freezes or blocks.
When this happens and I am running in DEBUG mode, is there anyway to understand at what line of code my application is currently? Since it is freezed I expect to find a point where the application is locked or blocked or whatever.
Is it possible to do that someway?
When It is freezed I tried to open the CALL STACK window, but this doesn't display any info; is there some action that I might do? Some "pause and check" or whatever?
You may need to open the Threads window, and change the current thread. While debugging, choose Debug->Break All, and open the Threads window. If you go through each thread, by double clicking on the thread, you should be able to investigate the call stack for each thread.
That being said, if you can run your program in VS 2010 - this gets a lot easier. In VS 2010, you can use the new Concurrency Profiler, and run your code under the Concurrency Profiler with the option to Visualize the Behavior of a multithreaded application. Once your application locks up, kill it, and let the profiler churn -
Eventually, you'll get a diagram which shows each thread in your program, and when they're locked. The callstack for each blocked thread will be shown, as well as what lock is being held (with the line of source code). This makes it very easy to track down a dead lock.
When debugger is attached go to Debug menu and choose 'Break All'... Then you can examine call stacks for all thread.
You could press pause and see where it ends up (use the aforementioned call stack window). Chances are however that you'll end up in native code. You could try to step out a bit or just look at the stack for a managed function to debug.
Alternatively you could put a breakpoint after the application "freezes" and try to pinpoint a loop that doesn't end.
Both the above are assuming your application is busy (100% cpu usage). If your application is however stuck in a dead lock or just plain waiting for a mutex that won't tick, you'll have to manually re-read your code and try to figure it out on your own.

App doesn't exit when main window is closed

I keep having this problem, solving it, and then when I implement new code it comes back again. It's driving me crazy!
What I finally found was that if you instantiate a Window of any kind, even if you never call Show() or ShowDialog(), when you close your application, it will not terminate. So now I make sure to call Close() when appropriate, and the problem hasn't ever come back with all of the Windows that I've created.
I've implemented more new features that don't create windows (as far as I can tell!), yet now my app will not terminate again. Hitting pause in the VS IDE is useless, I think, because the threads don't have any context so I can't figure out what code caused the hanging.
Normally, I would expect that a thread executing in the background that hasn't exited (and wasn't set as a Background thread) would cause this behavior, but I am not creating any threads at this point.
Can anyone recommend a good tool (free or license required) that will help me quickly resolve these sorts of stupid problems? For now, I'm going to go back, comment out a ton of the new code, and then uncomment line by line until the problem reappears. Brute force is how I typically end up fixing these sorts of things, and would really appreciate a tool to make my life easier. :)
It sounds like you may be having other issues with background threads that the other answers are addressing but with regard to WPF Windows, have you tried changing the ShutdownMode of your App class? You can also try forcing the app to quit by calling Shutdown explicitly:
Application.Current.Shutdown();
You might get more information if you attached using both managed and unmanaged code. In Visual Studio 2008, you can change the mode in the Attach to Process form. Press the "Select..." button and specify debugging for both "Managed" and "Native".
(Before you do this, make sure your symbol path is setup. Go to Tools/Options, Debugging, Symbols. Enter http://msdl.microsoft.com/download/symbols in the list of symbol file locations. Cache the symbols locally in some directory.)
When you attach in both managed and unmanaged modes, you should get a larger call stack. I recommend right-clicking in the Call Stack debug window and choosing "Include Calls To/From Other Threads".
If your main thread shows System.Windows.Forms.Application.Run or ThreadContext.RunMessageLoop, then your UI thread's message pump is alive and still pumping messages. If, for some reason, it is transitioning to another thread, then it can't exit until it's done.
You can also see the full stack traces of the rest of the managed and unmanaged threads. You might want to look for a garbage collector thread and see what it's doing. Look for one that is has a stack with "GCHeap::FinalizerThreadStart" in it. That might be doing something.
There may also be a GID+ thread that's busy trying to do work.
I don't mean to over-simplify things, but have you tried setting the owner of the dialog window to the MainWindow? This will force the dialog window to close when the MainWindow is closed. In other words, it would look like this:
dialog.Owner = Window.GetWindow(this);
// Or...
dialog.Owner = Application.Current.MainWindow;
This may not be an option for you, but I just wanted to throw it out there since your post didn't mention you didn't want to set the window's owner.
While it may not help in your particular case, Process Explorer is an excellent tool for looking inside running processes and seeing how many threads etc. there are.

Categories