I have a windows service, that is occasionally crashing and stopping when an unhandled ThreadInteruptedException is thrown. The main function of this service is to run a sync every x minutes (user defined timespan). I only once use the Thread.Interrupt() method twice (because I know how messy it can be) -
A client can connect via a TCP connection, but if the data is being sent to the client too fast for it to deal with it, it can send a "pause" signal back to the service, and then will resume when all the data has been handled (these could be wireless devices and could have multiple clients running at different speeds, so I didn't want to leave any of this in the TCP buffers). Pause then puts the thread to sleep, and resume interrupts this and continues.
The other use is in a similar situation when the TCP connection is originally connecting.
However, I've let this service run and this exception is thrown even when I do not connect any clients! No TCP connections are being made, and therefore neither of these interrupt methods are reached! I have put try statements to catch this exception around every Thread.Sleep() in the service, but this exception is still being raised!
I'm sure I'm missing something really obvious, but it's just not jumping out at me and I've been looking for days!
Has anyone ever encountered this
before?
Does anyone have any cheap
alternatives to sleeping and
interrupting a thread (not
while(paused) continue;)
Does anyone know of an easy way of
debugging an unhandled exception
like this? I can't even see the
stack to find out where this is
coming from!
Can any other method throw a ThreadInterruptedException? Or can any other methods cause a sleeping thread to be interrupted?
Thanks in advance
For anyone reading this. My accepted answer would be Hans Passant's comment. This solution worked just as well and got rid of any ThreadInterruptedExceptions
A TIE doesn't fall from the sky. Improve your code by removing the Interrupt() calls and replacing them with Auto/ManualResetEvent.Set().
I can't see that exceptions are being raised from methods that aren't even called.
Also, the ThreadInteruptedException is not raised by the Sleep method, but by the Interupt method - so try / catch on Sleep seems irrelevant here.
From MSDN:
You can interrupt a waiting thread by
calling Thread.Interrupt on the
blocked thread to throw a
ThreadInterruptedException, which
breaks the thread out of the blocking
call. The thread should catch the
ThreadInterruptedException and do
whatever is appropriate to continue
working. If the thread ignores the
exception, the runtime catches the
exception and stops the thread.
Related
I'm working in a legacy project that has this exception handling code in many methods.
catch(ThreadAbortException e)
{
...
}
I don't see anywhere in the project Thread.Abort() or Thread.Interrupt() calls. Is it safe to delete all these ThreadAbortException handling or it is some other way that can be raised.
Well if answering specifically your question I would say that it would be better not to delete these exception handlers as it’s most likely that they were added by some developer trying to solve a problem. And I think there was a reason to add those handlers so if you just remove this code it can lead to appearing of some bugs again in the future.
Regarding the ThreadAbordException: I know for sure that it can be throwed not only with calling Thread.Abort() method when you are debugging (it might be a bug in VS, I’m not sure) and it forces your program to just crash silently. So depending on what’s inside of those handlers it could be possible that a developer was trying to solve such problem.
Also remember that you could be invoking methods of third-party libraries, web-services etc. in a separate thread, too. I’m not sure if they can throw such an exception but that’s a possible case to consider.
Official docs: "The exception that is thrown when a call is made to the Abort method." If you are completely sure there are no calls to Thread.Abort then you might as well erase those catch blocks.
EDIT: Be mindful that your code may be running in the context of an external app that may call Thread.Abort on your threads.
Not that it matters anyway as a ThreadAbortException can't really be handled as the CLR itself will rethrow it to actually kill the thread ASAP.
"Actually yes, a ThreadAbortException is special. Even if you handle it, it will be automatically re-thrown by the CLR at the end of the try/catch/finally. (As noted in the comments, it can be suppressed with ResetAbort but by that point the code smells like rotten fish.)"
- Read this question for more details: ThreadAbortException
Is the project running on a main thread and spinning up background worker threads? If the main thread exits while background threads are running, a ThreadAbortedException can occur on the background threads.
The catch statement could specifically handle this scenario, where no error actually occurred on the background thread, in a different manner than any other exception.
I have a situation on a production machine where a thread in my Windows Service appears simply to die, without throwing an exception. To date my logging hasn't been verbose enough to pinpoint the exact line of code where it is dying; I've deployed a new version with more verbose logging for this purpose. But until I get a smoking gun, my suspicion falls on the line of code where I create a new DB context.
The error is not predictable, except that it tends to happen during periods of high activity, and is often correlated with other threads throwing DB timeout exceptions (hence my suspicion above). Exceptions I can handle. Dead threads I can't.
Any ideas why a thread might die silently, or otherwise simply freeze? Or what to do about it?
EDIT: To be clear, the code is surrounded by a try-catch block, and the catch does some logging (using log4net). So does the "finally". I know it's working because other threads have left logs when they threw exceptions. All that I see in my log is that thread x hits a certain debug point, then is never seen or heard from again, and the work it was supposed to do remains not done.
No, not really (but kind of). Threads don't die, they finish. A thread can finish successfully or by failure--if a thread fails then its exception is stored until it is handled. Depending on how you instantiate the thread it's possible that you "fire and forget", meaning that if an exception occurs you have no code to handle and retrieve it. This is doubly bad, as it results in unreleased resources (they're waiting for you to retrieve and handle the exception).
However, you haven't provided any details of what you've diagnosed. What threading approach/framework are you using (there's quite a few)? Are you watching the processes threads and seeing it suddenly disappear? Are you taking heap snapshots, or attaching to the running process after it appears to have died?
If you have access to the system i would debug it with: Visual Studio 2013 Remote Debugger. This is often a good choice if you can't reproduce the error in your development environment.
Threads do not silently freeze or crash, check whether you have some empty catch blocks in your code or it runs into an infinite loop (or very long timeout at any point).
Maybe you have some code for us.
Situation
Well, as you can see below, I have a main App, creating a thread, which creates a bunch of background workers. When the RunWorkerCompleted is fired on some BG worker, I sometimes end up dropping an Unhandled Exception, (for a reason I have clearly pinpointed, but this is not the matter).
So the Logs clearly show that my UnhandledException handler is entered and reports the exception.
I know I am deliberately misusing the OnUnhandledException by stopping the App from exiting, using a Console.Read(). I did it on purpose because I want a human intervention/check before terminating the app.
public static void OnUnhandledException(object sender, UnhandledExceptionEventArgs e)
{
Logger.log("UNHANDLED EXCEPTION : " + e.ExceptionObject.ToString());
Mail.Sendmail("ADMIN ALERT : " + e.ExceptionObject.ToString());
Console.Read(); // Yes, this is an ungraceful trick, I confess.
}
However, what was supposed just to be a "pause" before manual exit turned out to keep the App alive, ie, the Thread is still generating workers, and running as if nothing happened.
Even Weirder : The UnhandledException is still being dropped from time to time when it happens again. And it is logged each time, behaving just as if it was a plain ol' try catch.
(This was the default behaviour before .Net 3.0)
Question
So, why is everything happening as if the UnhandledException was not thrown, and the thread and BG's keeping running as if nothing happened ? My guess is that as long as the OnUnhandledException handler is not done, and the App is still alive, all running threads that are still alive keep on living in a free world and doing their job.
This gives me a bad temptation, which is to keep this design, as a try/catch to unHandled Exception. I know this is not a tidy idea, as you never know if the exception is serious or something that could be skipped. However, I would really prefer my program to keep running all day and watch the log report / mail alerts and decide for myself if it needs to be restarted or not.
As this is a server application that needs to be running 24/7 I wanted to avoid ill-timed and repetitive interruptions due to minor still unhandled exceptions.
The biggest advantage to that being that the server program keeps running while we are tracking down unhandled exceptions one by one and deliver patches to handle them as they occur.
Thank you for your patience, and please feel free to give feedback.
PS : No, I did not smoke pot.
My guess is that as long as the OnUnhandledException handler is not
done, and the App is still alive, all running threads that are still
alive keep on living in a free world and doing their job.
Correct. But your other threads are now doing their job in a potentially unstable process. There are good reasons for terminating the App when an unhandled exception happens.
I would really prefer my program to keep running
Besides the (potential) stability problems, you would accumulate the halted state of whatever is involved in the unhandled handling. My guess is that you would at least leak a Thread each time it happens. And you can't spare too many of them.
So it would be a very bad design, and only delay the crashing.
First of all you need to subscribe the UnhadledThreadException handler. There you can not "Hijack" the exception, but you have access to it and also to the thread it belongs. What you can do now is putting the throwing Thread into suspetion. The Thread will be still alive!!
This however is a bad, dirty hack! Do not do this!
Take a look at this article for some details.
I recommend switching your threading model to TPL (design remains). There you have access to those ugly cross-thread excepions and can handle/supress em gracefully.
If you want to allow a BackgroundWorker thread to fail without killing the whole process, there's a cleaner way to do this.
First, intercept the RunWorkerCompleted event. This event is raised when the BackgroundWorker thread terminates, either normally or via an unhandled exception. Then in your event handler, check for the unhandled exception:
// Runs when the BackgroundWorker thread terminates.
private void ThreadFinished(object sender, RunWorkerCompletedEventArgs e)
{
// Process any unhandled exception
if (e.Error != null)
{
this.LogError(e.Error.Message, e.Error.StackTrace, blah, blah);
In this way, you can let the BackgroundWorker thread die and log any exception without terminating the whole process.
I'm looking for some guidance on best practices on handling exceptions in events. Currently, when exceptions are thrown in my application, the exception message is displayed in a pop up dialog and then the application restarts after hitting ok. The issue I'm seeing is that a lot of exceptions are occurring in the event handlers of some third party libraries and these exceptions are swallowed and never displayed since they are on a background thread. Here are a few solutions that various people have thought of and I would like to know of any of these are the best approach.
Forward the background thread to the UI thread in every event handler in the application.
Wrap the events in another class which has a try/catch around every method that invokes the event. The catch will then forward the exception to the UI thread if one occurs.
Get access to the third party libraries and put try/catch around where the events are being invoked, which could then be forwarded to the main application by a global event.
Lets discuss your options the way you listed them:
1) Forward the background thread to the UI thread in every event
handler in the application.
You should not and you can't! because:
You can't pretend if that event "which may comes from a third party" starts another background thread
or timer.
The whole point of having background threads is not
to freeze the UI, you will then struggle the user interface by a long execution codes.
2) Wrap the events in another class which has a try/catch around every method that invokes the event. The catch will then forward the exception to the UI thread if one occurs.
The same point as the previous. you can't pretend it that event starts another thread..
3) Get access to the third party libraries and put try/catch around where the events are being invoked, which could then be forwarded to the main application by a global event.
This is a bad choice, really, You should not change that code of third party libraries even if you can. because:
You are not the author for those libraries.
Whenever you want to update the library version or even change the libraries, you have to rewrite there code..
So what you have to do? The answer is some sort of what you are currently doing:
Whenever any exception thrown, log the exception restart your process, and at some point in your process send the logged bugs to your email and then start fixing them.
None of the above. Instead hook up the events on the Application and AppDomain for unhandled exceptions.
AppDomain.CurrentDomain.UnhandledException Event
Application.DispatcherUnhandledException Event
Further Information: The global exception handling event for WPF applications Application.DispatcherUnhandledException fires only for exceptions thrown on the main UI thread. However the AppDomain.CurrentDomain.UnhandledException should fire on any unhandled exception in any thread (unfortunately there is no preventing the application from shutting down after ward it reaches here).
Did some quick research on best practices and found that it is recommend that you manually handle exceptions on the background threads with try\catch block. Read the exception part on this page http://www.albahari.com/threading/ and also take a look at this StackOverflow question Catching unhandled exception on separate threads
If you're hooking the AppDomain.UnhandledException event, then the issue isn't the fact that they might be calling back on a background thread, but rather than the third-party library is explicitly swallowing exceptions thrown by the handlers. This is a badly behaving library, but, since you're writing the event handler code, you can catch the exceptions in the event handler and force the application to shut down.
Since the third-party library can't be stopped by throwing an exception, you can force the thread to terminate by calling:
Thread.CurrentThread.Abort();
Thread.Abort() is generally considered a bad practice, but it's slightly less dangerous here, as you are aborting yourself and therefore won't be injecting the exception into potentially nasty locations (you also know that the thread is not in an unmanaged context, so it will abort right away.) It's still a nasty hack, but the third-party application isn't giving you much choice. ThreadAbortException cannot be 'stopped', so the third-party code will terminate its thread as soon as it reaches the end of its exception handlers (even if they try to swallow it). If the third-party library is really nasty, it might invoke lots of code in its catch or finally blocks, even using dirty tricks to stay alive like launching other threads, but it would have to be pretty malicious to do this.
To get the friendly error message behavior, you can marshal the Exception to your UI thread using a SynchronizationContext or Dispatcher to call your exception-handler display (the easiest way may be to just re-throw the exception; I recommend embedding it as an InnerException so that you don't lose stack trace.)
If you really distrust the library (and don't want to give it the opportunity to stay alive even through its catch and finally blocks), you can explicitly marshal to the UI thread with a blocking invoke (to display your error message in a friendly way) and then call Environment.FailFast() to kill your process. This is a particularly harsh way to terminate (it will not call any cleanup code and literally tears down the process as quickly as possible), but it minimizes the possibility for any potentially damaging side effects if application state has become very corrupted. One possible thing to watch for here is that the UI thread could be blocked by some other UI invocation tied to the third-party code. If this happens, the application will deadlock. Again, this would be a pretty malicious thing for it to do, so you can probably ignore it.
There are already some pretty good threads on this topic on Stack Overflow, but there doesn't really seem to be a concise answer on any of them. My C# console application (running as a Windows service) launches a Java process and manages it (starts/stops/restarts it), but my issue is that I will remote into machines, and see it has started about 20 Java processes sometimes.
This is obviously an issue with my application crashing at some point, and not shutting down the Java process it started. I have hooked "UnhandledExceptionEventHandler" in the AppDomain.CurrentDomain, and I call TerminateProcess() from it (shuts down the active Java process) but this issue is still occuring on occassion.
My application has the Main thread, a TCP Server Thread (which accepts async connections), and a UDP Server Thread. Is there anything else I should be hooking into on top of UnhandledException?
EDIT
It also just occured to me that I have a few Try/Catch blocks in my code that simply write to console, which I never see. Should I just remove these so these are caught by the UnhandledException or add a logger there instead?
First of all you have to change the Console.WriteLine.. lines in you code to Debug.WriteLine.. if you don't want to log, so the output of it will only be on debug.
Second when any exception occurs if you don't know how to handle it or fix it then rethrow it catch { throw; } after logging. I personally do
try
{
...
}
catch (Exception exception)
{
Log(exceptiosn);//log it first or Debug.WriteLine...
#if DEBUG
System.Diagnostics.Debugger.Break();//break at the debugger here.
#endif//DEBUG
throw;
}
After you cleaning up you code, now you can whenever DomainUnhandledException thrown, you can restart your application. an example will be here, The idea is you start a new instance from your application and then terminate the first one. you also define a mutex so only one instance at time will be alive.
Something to consider is whether you want the .NET application to be responsible for the spawned processes. If possible, they might be made responsible for shutting down when no longer receiving input. If the spawned processes are running on other machines I would try to do that anyway because network problems might interfere with sending a shutdown message from the .NET application to the Java processes. Each its own responsibilities.
That said, getting exception handling in the .NET application fixed (especially when you are missing some exceptions) is also important. Make each thread responsible for its exceptions and make sure you log them for easy debugging.