Detect unresponsive service process - c#

I have a service daemon (running in Windows Server) that runs 24/7 (or it should) that I didn't write or have access to the source code.
For some unknown reason the service sometimes stops responding (my guess is that it deadlocks). This is a mail service so I know it has stopped working when I don't receive any mails from it in 'n' time (n is never fixed).
Currently the only programmatical way I can check if it's non-responsive is trying to stop the service, and if that times-out, I know it's dead/zombie. I can then force-kill the service process and just run the service again.
I want to automate this process (i.e., code a "watchdog"), and would like to periodically check if the service has stopped responding without trying to stop it first (I don't want to stop the service if it's really working).
Is there any programmatical way to check if a process is not responding which is not trying to stop it?
I don't have any clue how the service works internally so I can't just send a signal to it and wait for response (since I don't know what signals it would respond to, other than attempting to cleanly stop it).
This has been on and off in my head for some time and I haven't been able to find an answer.
If there's nothing else, I'd just make it stop every night or so (that'd be acceptable even if it's working), but if there's any way, I'd prefer to have some daemon check it every 10 minutes or so (stopping and restarting the service every 10 minutes would NOT be acceptable if it's working).
I'd appreciate any hints on where to look, if there's any.
PS: the code for service stopping, process killing or service starting is not a problem and I'm not looking for that... I'd only like a hint or direction on how to check if it has became non-responsive without knowing how it is programmed.

Related

Get Notification when .EXE from task manager or in system Stopped using .NET technology

I have some windows application .exe which are run on my Domain server, i have a problem that if the .exe is stooped how can i get the notification that the .exe has stopped.
is there any solution thru i can manipulate my code with operating system and get notification thru mail or any other resource
We have critical windows services that must always be running at work and the way we handled this was to write a monitioring program that listens to communication from the critical services and sends out an email if any of them stop "calling in". Here are a few details (although a bit simplified):
1) We use MSMQ messages to have the computers running the services talk to the monitor. Each service writes to a specific queue and the monitor is set to read from those queues. Note that MSMQ has pros and cons--if you choose to use this method, be sure to read up on it a bit.
2) The critical programs write messages to the queue detailing what it is they are doing and if they have had any errors.
3) If it has been more than 20 seconds (adjust accordingly for your situation) and the services haven't had anything to do, they simply write a message to the queue saying that they haven't had anything to do for the last 20 seconds.
4) The monitor reads these messages, keeps track of how long it has been since it has heard from each queue and if any of them haven't sent at least an "I've been idle for 20 seconds" message within 30 seconds (note that this should be longer than the other time period), it sends an alert to our emails saying which service has been idle for too long. Similarly, if any service has had any critical errors, the monitor may report those right away, too.
ALTERNATELY
There are off-the-shelf programs you can buy to do some or all of this for you, but this solution has worked well for us. If you are interested in 3rd party tools, you may consider looking at Splunk, Big Brother, and Tripwire. I don't have much/any experience with these tools, so I'm not sure if they will do what you want or not.
You can make use of the OnStop() event in the ServiceBase as given here. Related discussion post is here

C# windows service does not always stop on shutdown

I created a windows service with c# 2010. The problem is that when computer shuts down the service does not have time to stop and onstop is not always executing. I say not always because sometime it manages to stop. I have tried to use windows pre-shutdown notification that was introduced at vista, however the results are better but not absolute.
Is there anyway to get windows wait for my service to stop?
Is there anyway to change the order windows stops the services?
According to http://msdn.microsoft.com/en-us/library/windows/desktop/ms685149(v=vs.85).aspx, a service normally has about 20 seconds to shut down before the system gives up and shuts down anyway.
You might be able to send STOP_PENDING messages back if your service needs more time to shut down, but even then you're limited to about 125 seconds before the system figures you're never going to shut down, and pulls the rug out from under you.
In C#, you would have your service call RequestAdditionalTime during shutdown. Again, there's no guarantee that you'll get that extra time, but you can ask for it. Remember, though, if you ask for more time you better be done before that time expires, or you can ask for more. But eventually the system will shut you down (again, probably in less than 2 minutes).
In general, I've found that it's best if you construct your service so that you can shut it down quickly. If you can't shut down in a few seconds, you probably need to change your design.
This can be answer for you (and for somebody, who search same answer):
Why doesn't the RequestAdditionalTime() method work on restart in Vista/7?:
... Suppose Windows allows 13 seconds for all services to shutdown. Some other service takes 12 seconds of cpu time (so other services can't execute during that time) and finally shuts down. That leaves 1 second for all the rest. Windows is going to kill them.
This is the reason, why it sometimes shutdown correctly a sometimes doesn't.
By the way, Vista has 20 sec. to kill services, W7 has 12 sec. and W8 has 5 sec.
In your service class derived from ServiceBase, in override of OnStop you can request for additional time:
base.RequestAdditionalTime(1000 * 60 * 2);

How can I make sure a C# console program ALWAYS exits?

I've written a small C# console app that is used by many users on a shared storage server. It's runtime should always be < 3 seconds or so, and is run automatically in the background to assist another GUI app the user is really trying to use. Because of this, I want to make sure the program ALWAYS exits completely, no matter if it throws an error or what not.
In the Application_Startup, I have the basic structure of:
try
{
// Calls real code here
}
catch
{
// Log any errors (and the logging itself has a try with empty catch around it
// so that there's no way it can causes problems)
}
finally
{
Application.Shutdown();
}
I figured that with this structure, it was impossible for my app to become a zombie process. However, when trying to push new versions of this app, I repeatedly find that I cannot delete and replace the executable because the "file is in use", meaning that it's hanging on someone's computer out there, even though it should only run for a few seconds and always shutdown.
So, how is it that my app is seemingly becoming a hanging process on peoples' computers with the code structure I have? What am I missing?
Edit: Added "Application." to resolve ShutDown() for clarity.
There are two options here:
Your console application doesn't really finish in 3 seconds, but rather takes a lot longer. You need to debug it and see what takes it that long.
Your console application takes 3 seconds to exit, but it is run every minute by the GUI, and you have more than 40 users, so the probability of finding the executable unused are slim.
If it's the first one, and you don't want to debug it, you can always start a second thread, wait for 3 seconds and then kill the entire process.
Maybe the code inside the try block is still executing for at least one of the clients and is not really limited to 3s or so. To prevent such case, you would need multithreaded application - one thread for processing and one in the background killing the working thread after a timeout. Prior to that you should ask yourself if such infrastructure is really needed.
Another thing that comes to mind would be that one of the users had the application running right at the moment, probability depends on the number of your users.
Maybe designing your support app as a always running multithreaded service would be a much better idea instead of instantiating one running application for each client request.

Prevent C# app from process kill

How can I protect my C# app from someone killing its process via taskman or programmatically?
Here is my scenario:
App A is an MFC app developed by another team. It has an unpublished text-based remote interface that is enabled via a backdoor.
I'm developing app B, a C# WinForms app which interacts with A. B enables A's backdoor when it needs remote access closes it when finished (or on failure).
I'm exploring ways users could abuse B in order to gain access to A's hidden functionality, such as killing B's process after it has enabled A's remote interface. I'd like have one last chance for B to close A's backdoor when that happens.
B uses localhost to interact with A, so I'm not worried about the power-down scenario.
I'm looking for a solution that doesn't involve changing A.
I'm not expecting to be able to stop Dark Tangent (though that would be a bonus), but right now a script kiddie could have his way with this design :)
These apps run on Windows XP, but will also soon support Vista & 7.
Thanks in advance,
Jim
I'm willing shut the app down when they try but need to do some things first.
Having necessary steps at program shutdown leads to fragile programs that break easily. Even if you could prevent someone from killing your program via the task manager, you cannot stop them from turning off the computer, or even pulling the cable out of the wall. Whatever task that was so vitally important to complete will be lost. And what if there is a power cut? Again your task won't complete and your vital clean up code will not be run.
Instead you should make your program robust to failures at any point. Use transactions, and always save state to files atomically - make sure that you always have at least one valid copy of your data. Don't overwrite important files in a way that they become temporarily invalid.
Finally, you can add a dialog box to your program that when they try to close it, warns them that the program needs to shut down properly. If you make your shutdown fast users won't want to kill it and will let it terminate properly. If your shutdown takes ages then people will try to kill it. If you are nice to your users, they will be nice to you too.
If shutting down fast means that the user will lose some unfinished work then warn them about this and give them the opportunity to wait for the task to finish, but if they really want to quit your program then let them quit.
You can't - as long as the user has the right to call TerminateProcess on your program, you can't prevent End Process from killing you immediately in task manager. Raymond Chen posted on this some time ago: The arms race between programs and users
You really, really, really don't want to do this. It makes users very angry!! However, if it is supposed to be a service, run it as a service account and don't give admin rights to users.
Short answer: you can't and you shouldn't.
Long answer: You can try to start a second 'helper' process, that checks every x seconds if your app is still running. If it isn't, it restarts it.
If you want a process to run for a long time just don't trust users to keep it running, consider windows services. They are designed for this.
I think everybody has missed the point. If I read it correctly (after your edit) you wish to know when you are being "killed" so you can shut down gracefully?
The point of "killing" is that you "can't" stop it. There are of course workarounds like using a second app to revive a killed app, but that has nothing to do with simply being able to shut down gracefully.
The best approach is to either run as a service (so you can't be killed, just asked to shut down), or to restructure the way your app works so that it doesn't need to "tidy up" before it quits. When an app is quit, most resources it holds are automatically cleaned up, so it's only really your own data that you have to close cleanly. Approaches you could try are:
Frequently commit your state to disk so that you don't lose much (or anything) if you are unexpectedly quit. (Remember to flush all I/O streams to be sure they are committed to disk)
Save information to disk that allows you to detect an unexpected shutdown the next time your program runs, so it is able to detect and rectify whatever problems might have been caused by being killed.
Tell your users not to be idiots, and quit your application nicely. Poke them in the eye if they ignore you. Usually after no more than two times they listen :-)
In order to prevent your application from being terminated, you run your application as another user (i.e. as a service, or as another user account), and limit users to be Standard User.
This way no malicious users can kill your process, since only administrators can kill it, and that is a privilege that you, apparently, don't trust anyone with.
It has the advantage of following the intended design of the operating system.
#Jim
If App A can receive modification requests
Preferably, I would an architecture where all App B's are registered upon opening the backdoor and are required to ping App A with the registration at an interval so that App A can close it's own backdoor upon App B not informing it that it still needs access. This is still not perfectly secure but App A should not be structured with such an interface without some sort of self regulation for "secure" means of communication.
Or, you could suggest App A be modified to check for valid processes and if none are found while it's backdoor is open then it gets closed (this is spoofable since it goes by processed name).
Otherwise, it sounds like App B should shut the backdoor as often as possible when it does not need immediate access.
Requiring an App B to provide security of access to App A is a poor model indeed.
As far as i know you can't, and even if you could you really shouldn't. imagine how annoying it would be if you couldn't force kill an application.
If its important that your application keep running you could always create a windows service that "pings" the application to ensure it is running (you could use named pipes, sockets, pid files... whatever). if the service detects that the process has died then it can just restart it. this is probably your best bet.
When the application initiates for the first time could you not execute a 3rd ap/process that is running in the background and attempts to callback to App B every so ofter, so when that App B is closed.. App C can see that and executes a procedure to close App A's backdoor.
So that when App B closes successfully via the intended Close button it will disable App C from checking App B is still working fine...
Im not really the best with C# at the moment but looking at your problem thats probably one of the ways i would try to do it..
Also if App B checks App C aswell then if App C has gone down App B will close the backdoor if it can.
As the others say this may not be a good idea tho.

Guaranteed way to force application running continuously (overriding taskkill, task manager etc.)

I have a C# security/monitoring application that I need to have running no matter what. However, I can not remove privileges or restrict access to parts of the OS (Windows).
I thought of having a protection service running which monitors continuously if an application is running, and starts it back up when the application is killed somehow, while the application monitors the protection service and starts the service if the service is killed. To my knowledge you can't simultaneously kill multiple processes at the same time.
Any better idea to guarantee that an application is always running?
The Windows team gets requests like this all the time:
"I want to make a process that can never be killed".
"Well I want to make a tool that can kill any process".
One of those two people is going to be disappointed.
"I want to make a window that is always on top of all other windows no matter what"
"I also want to make a window that is always on top of all other windows no matter what"
One of those two people is going to be disappointed.
"I want to make a process that does not show up in any list of processes no matter how cleverly the listing program is trying to find me"
"I want to make a program that lists all processes, no matter how cleverly the process is trying to hide".
One of those two people is going to be disappointed.
And so on.
I think you're going to be one of the disappointed people.
There is no guarantee - if the user has enough privilege it can terminate your program or any other monitoring code you have running that will restart your application.
Imagine if this were true - every piece of spyware on the planet would be using it.
I would write your program as a windows service - configure it to run automatically on startup, and to restart automatically if it is terminated unexpectedly. You cannot do any better than this since the user must be allowed to control the computer.

Categories