There are already some pretty good threads on this topic on Stack Overflow, but there doesn't really seem to be a concise answer on any of them. My C# console application (running as a Windows service) launches a Java process and manages it (starts/stops/restarts it), but my issue is that I will remote into machines, and see it has started about 20 Java processes sometimes.
This is obviously an issue with my application crashing at some point, and not shutting down the Java process it started. I have hooked "UnhandledExceptionEventHandler" in the AppDomain.CurrentDomain, and I call TerminateProcess() from it (shuts down the active Java process) but this issue is still occuring on occassion.
My application has the Main thread, a TCP Server Thread (which accepts async connections), and a UDP Server Thread. Is there anything else I should be hooking into on top of UnhandledException?
EDIT
It also just occured to me that I have a few Try/Catch blocks in my code that simply write to console, which I never see. Should I just remove these so these are caught by the UnhandledException or add a logger there instead?
First of all you have to change the Console.WriteLine.. lines in you code to Debug.WriteLine.. if you don't want to log, so the output of it will only be on debug.
Second when any exception occurs if you don't know how to handle it or fix it then rethrow it catch { throw; } after logging. I personally do
try
{
...
}
catch (Exception exception)
{
Log(exceptiosn);//log it first or Debug.WriteLine...
#if DEBUG
System.Diagnostics.Debugger.Break();//break at the debugger here.
#endif//DEBUG
throw;
}
After you cleaning up you code, now you can whenever DomainUnhandledException thrown, you can restart your application. an example will be here, The idea is you start a new instance from your application and then terminate the first one. you also define a mutex so only one instance at time will be alive.
Something to consider is whether you want the .NET application to be responsible for the spawned processes. If possible, they might be made responsible for shutting down when no longer receiving input. If the spawned processes are running on other machines I would try to do that anyway because network problems might interfere with sending a shutdown message from the .NET application to the Java processes. Each its own responsibilities.
That said, getting exception handling in the .NET application fixed (especially when you are missing some exceptions) is also important. Make each thread responsible for its exceptions and make sure you log them for easy debugging.
Related
I have a situation on a production machine where a thread in my Windows Service appears simply to die, without throwing an exception. To date my logging hasn't been verbose enough to pinpoint the exact line of code where it is dying; I've deployed a new version with more verbose logging for this purpose. But until I get a smoking gun, my suspicion falls on the line of code where I create a new DB context.
The error is not predictable, except that it tends to happen during periods of high activity, and is often correlated with other threads throwing DB timeout exceptions (hence my suspicion above). Exceptions I can handle. Dead threads I can't.
Any ideas why a thread might die silently, or otherwise simply freeze? Or what to do about it?
EDIT: To be clear, the code is surrounded by a try-catch block, and the catch does some logging (using log4net). So does the "finally". I know it's working because other threads have left logs when they threw exceptions. All that I see in my log is that thread x hits a certain debug point, then is never seen or heard from again, and the work it was supposed to do remains not done.
No, not really (but kind of). Threads don't die, they finish. A thread can finish successfully or by failure--if a thread fails then its exception is stored until it is handled. Depending on how you instantiate the thread it's possible that you "fire and forget", meaning that if an exception occurs you have no code to handle and retrieve it. This is doubly bad, as it results in unreleased resources (they're waiting for you to retrieve and handle the exception).
However, you haven't provided any details of what you've diagnosed. What threading approach/framework are you using (there's quite a few)? Are you watching the processes threads and seeing it suddenly disappear? Are you taking heap snapshots, or attaching to the running process after it appears to have died?
If you have access to the system i would debug it with: Visual Studio 2013 Remote Debugger. This is often a good choice if you can't reproduce the error in your development environment.
Threads do not silently freeze or crash, check whether you have some empty catch blocks in your code or it runs into an infinite loop (or very long timeout at any point).
Maybe you have some code for us.
I am new to multithreaded application. I have few doubts before starting working on it.
Can anyone clear these doubts?
How to handle exceptions in multithreaded application?
If there is any exceptions in any 1 thread, will the whole application will come
down? or all other threads will continue?
How to handle exceptions in multithreaded application?
The best way is inside the thread, but sometimes eg. (thread pool) this is hard. To handle unhandled exceptions depends on the type of application:
WPF: Use Application.DispatcherUnhandledException
Console or Service: Use AppDomain.UnhandledException
ASP.NET, WinForms, ... have their own mechanisms.
But consider: especially while developing to allow the default handling and falling into the debugger (adding if (Debugger.IsAttached) { Debugger.Break(); } can be very helpful to define a permanant breakpoint
If there is any exceptions in any 1 thread, will the whole application will come down?
It depends. In ASP.NET: No (if debugging is enabled you'll see a Yellow Screen of Death, otherwise a 500 server error result); in other hosts: depends (but generally the answer is Yes the process will terminate).
Some of the details of this have changed between .NET versions – typically getting stricter, so you need to do your research. Finally: in .NET 4 certainly, possibly before as well, certain exceptions (eg. StackOverflowException) cannot be caught because there is no reliable way to have consistent process state if they are thrown.
If you are using TPL you can use Aggregate exception. This aggregate exception is a collection of exceptions you can loop through them to see which thread failed to execute more information can be found here. http://msdn.microsoft.com/en-us/library/dd997415.aspx
I wrote a C# NET application (Console app that is run as a Service) that manages a Java process (Minecraft Game Server), and our Web Panel software manages my application. When the Java process stops, my application stops itself, and when my application stops, it stops the Java process.
The issue I am running into is that I deployed the software to all of our machines after extensive bug testing, but there seems to be a bug we missed where it is NOT shutting down the Java process sometimes. This bug is horrible as the Web Software tries to start my application, my application tries to start the Java process, but fails due to it being unable to IP bind (since the old process stayed open) and we wind up with 15-30 bugged Java processes running.
I am using CurrentDomain_UnhandledException to catch my application's crashes and call TerminateProcess().
I am using CtrlTypes.CTRL_C_EVENT and CtrlTypes.CTRL_CLOSE_EVENT to detect my application being closed which also calls the TerminateProcess() function.
I have tried both of the following methods...
static void TerminateProcess()
{
log.LogMessage("Minecraft Process Shutdown.");
SendProcessCmd("stop");
}
and
static void TerminateProcess()
{
log.LogMessage("Minecraft Process Shutdown.");
minecraftProcess.Kill();
}
However, I seem to be missing another way that my C# application is being shut down, because both ways seem to leave a Java process running every once in a while that I can't reproduce locally.
Well, you did not state any question, I’m going to guess you wanted to ask for other ways a process can get shut down, so that you can intercept it and ensure the Java process termination. Simply said: That is impossible (in full generality).
Just look at your own code: You are doing exactly the same thing to Minecraft: Calling TerminateProcess causes the target process to terminate immediately, without any chance to clean up. Should anyone call TerminateProcess on you (e.g. a user killing the process from Task Manager), your process just terminates immediately.
Also, some fatal exceptions are uncatchable, e.g. when your process dies on a stack overflow, you are not told, just terminated.
I guess you’d need to create another process, watching over your process… (Or rethink the whole architecture, creating and killing processes, especially with TerminateProcess, seems a bit rough to me.)
I have a windows service, that is occasionally crashing and stopping when an unhandled ThreadInteruptedException is thrown. The main function of this service is to run a sync every x minutes (user defined timespan). I only once use the Thread.Interrupt() method twice (because I know how messy it can be) -
A client can connect via a TCP connection, but if the data is being sent to the client too fast for it to deal with it, it can send a "pause" signal back to the service, and then will resume when all the data has been handled (these could be wireless devices and could have multiple clients running at different speeds, so I didn't want to leave any of this in the TCP buffers). Pause then puts the thread to sleep, and resume interrupts this and continues.
The other use is in a similar situation when the TCP connection is originally connecting.
However, I've let this service run and this exception is thrown even when I do not connect any clients! No TCP connections are being made, and therefore neither of these interrupt methods are reached! I have put try statements to catch this exception around every Thread.Sleep() in the service, but this exception is still being raised!
I'm sure I'm missing something really obvious, but it's just not jumping out at me and I've been looking for days!
Has anyone ever encountered this
before?
Does anyone have any cheap
alternatives to sleeping and
interrupting a thread (not
while(paused) continue;)
Does anyone know of an easy way of
debugging an unhandled exception
like this? I can't even see the
stack to find out where this is
coming from!
Can any other method throw a ThreadInterruptedException? Or can any other methods cause a sleeping thread to be interrupted?
Thanks in advance
For anyone reading this. My accepted answer would be Hans Passant's comment. This solution worked just as well and got rid of any ThreadInterruptedExceptions
A TIE doesn't fall from the sky. Improve your code by removing the Interrupt() calls and replacing them with Auto/ManualResetEvent.Set().
I can't see that exceptions are being raised from methods that aren't even called.
Also, the ThreadInteruptedException is not raised by the Sleep method, but by the Interupt method - so try / catch on Sleep seems irrelevant here.
From MSDN:
You can interrupt a waiting thread by
calling Thread.Interrupt on the
blocked thread to throw a
ThreadInterruptedException, which
breaks the thread out of the blocking
call. The thread should catch the
ThreadInterruptedException and do
whatever is appropriate to continue
working. If the thread ignores the
exception, the runtime catches the
exception and stops the thread.
I have a console application written in C#/.NET that I want to run from a script (nant). If an exception occurs in the console application, I would like nant to continue, but in Windows Vista there is a popup that searches for solutions and asks for debug etc.
I would like to avoid the popup with "program stopped working" when an exception happens in the console application. How can I control this from C#/.NET?
(A similar question addresses the issue for the C language, but I would like a solution for C#/.NET.)
(To clarify: I would like the exception to be passed to nant, but without the popup.)
The JIT debugger popup occurs when there's an unhandled exception. That is, an exception tunnels all the way up the stack to the root of any thread in the runtime.
To avoid this, you can handle the AppDomain.CurrentDomain.UnhandledException event and just call Environment.Exit(1) to exit gracefully.
This will handle all exceptions on all threads within your AppDomain. Unless you're doing anything special, your app probably only has one AppDomain, so putting this in your public static void Main method should suffice:
AppDomain.CurrentDomain.UnhandledException += (sender, args) =>
{
Console.Error.WriteLine("Unhandled exception: " + args.ExceptionObject);
Environment.Exit(1);
};
You should probably use the NAnt logger to write out the error in this case too (can't recall the API for this offhand though.)
You can also disable JIT debugging on the machine. I would only recommend this in certain circumstances such as for a dedicated build server.
Under Windows Vista you can disable this dialog for your programms.
Disable the "Problem Reports and Solutions feature". You find it under Control Panel-->Problem Reports and Solutions-->Change Settings-->Advanced Settings-->Turn off for my programs, problem reporting
Just catch the exception and log/ignore it.
The popup appears due to an unhandled exception. To avoid that make sure your main method captures all exceptions and turn them into some other useful piece of info you can pick up. Just ignoring the exception is not recommended.
Btw remember that exceptions are per thread, so if your application spawns threads or uses thread pool threads, you need a handler for these too.
Usually this only happens when your app doesnt handle an exception. If you wrap your whole console app in a try/catch bblock, and just pass back a fail code, then you will avoid this.
Sometimes, a windows application will stop working if you are using a System.Timers.Timer.
To fix this, change System.Timers.Timer by System.Windows.Forms.Timer
Greetings