.NET windows service aborted during system shutdown - c#

My windows service is a data server with substantial cache. During service OnStop I save the cache so that no data is lost. Saving cache may take several minutes so to prevent windows service manager from timeout I use SetServiceStatus Win32 callback:
this.serviceStatus.currentState = (int)State.SERVICE_STOP_PENDING;
this.serviceStatus.checkPoint = 1;
this.serviceStatus.waitHint = 60000;
SetServiceStatus(Process.GetCurrentProcess().Handle, ref this.serviceStatus);
That works fine.
I have also set CanShutdown to true and added OnShutdown so that service would be system shutdown proof. Here I effectively do the same thing as in OnStop:
protected override void OnShutdown()
{
this.OnStop();
base.OnShutdown();
}
That does not work too good. When system shuts down, when cache is being saved I get "The device is not ready". This suggests that Windows aborts service before it is done stopping / shutting down. Preventing that with SetServiceStatus apparently does not work.
How do I get more time (delay reboot) to get saving done?
Any suggestions welcome.

The preferred way to handle this can be found in this article on the BCL Team's blog. The article includes a bit of background on how managed services interact with the SCM, which is important for understanding why the approach shown in the question is no longer recommended.
Simply put, ServiceBase now handles service state management for you and your code just needs to request additional time if required. Read the article for a few other good guidelines for writing SCM-friendly services in .NET.

Using ManualResetEvent might be the trick.

After some testing it looks that it was a problem with the system rather than with my service.

Another easy way, after hours of searching on how to include the setservicestatus method:
protected override void OnStop()
{
stopTimer();
m_run = false;
while (m_queue.Count > 0)
Thread.Sleep(500);
}

Related

How can I avoid duplicate background task processing in Service Fabric hosted services?

Sorry about the vague title, it's rather hard to explain. I have the following setup:
I'm running a .NET Core 2.2 Web API hosted in Service Fabric.
Part of this API's responsibilities is to monitor an external FTP storage for new incoming files.
Each file will trigger a Mediator Command to be invoked with processing logic.
I've implemented a hybrid solution based on https://learn.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/background-tasks-with-ihostedservice and https://blog.maartenballiauw.be/post/2017/08/01/building-a-scheduled-cache-updater-in-aspnet-core-2.html. In essence this is an IHostedService implementation that is registered in the Startup.cs of this API. Its basically a background service running in-process.
As for the problem. The solution above works fine on a 1-node cluster, but causes "duplicates" to be processed when running on a 5-node cluster. The problem lies in the fact that on a 5-node cluster, there are ofcourse 5 identical ScheduledTasks running and will all access the same file on the FTP at the same time.
I've realised this is caused somewhat by improper separation of concerns - aka the API shouldn't be responsible for this, rather a completely separate process should handle this.
This brings me to the different services supported on Service fabric (Stateful, Stateless, Actors and Hosted Guest Exe's). The Actor seems to be the only one that runs single-threaded, even on a 5-node cluster. Additionally, an Actor doesn't seem to be well suited for this kind of scenario, as it needs to be triggered. In my case, I basically need a daemon that runs all the time on a schedule. If I'm not mistaken, the other stateful/stateless services will run with 5 "clones" as well and just cause the same issue as I currently have.
I guess my question is: how can I do efficient background processing with Service Fabric and avoid these multi-threaded/duplicate issues? Thanks in advance for any input.
In service farbic you have 2 options with actors:
Reliable actor timers
Reliable actor reminders
You can use the state to determine if the actor has processed your ftp file.
Have a look at this blog post, to see how they used a reminder to run every 30 seconds.
It's important that the code in your actor allows reantrancy.
Basically because the actors are reliable, your code might get executed multiple times and be canceled in the middle of an execution.
Instead of doing this:
public void Method()
{
_ftpService.Process(file);
}
Consider doing this:
public void Method(int fileId)
{
if (_ftpService.IsNotProcessed(fileId))
{
_ftpService.Process(file);
_ftpService.SetProcessed(fileId);
}
}
If your actor has trouble disposing, you might want to check if you are handling cancelationtokens in your code. I never had this issue, but we are using autofac, with Autofac.ServiceFabric to register our actors with RegisterActor<T>() and we have cancelationtokens in most of our logic. Also the documentation of CancellationTokenSource can help you.
Example
public Ctor()
{
_cancelationTokenSource = new CancellationTokenSource();
_cancellationToken= _cancelationTokenSource.Token;
}
public async Task SomeMethod()
{
while(/*condition*/)
{
_cancellationToken.ThrowIfCancellationRequested();
/*Other code*/
}
}
protected override async Task OnDeactivateAsync()
{
_cancelationTokenSource.Cancel();
}

What's the recommended process for building Windows Services in .NET

I'm quite new to development of Windows Services (my background is in Web Development) and I'm a bit confused by the development process for Windows Services.
As I understand it, the service has to be installed every time you compile the code, and then run from the Services MMC snapin.
This seems like an awfully painful process to go through every time you make a change to your code.
Can someone shed some light on the 'recommended' approach for building services?
Thanks
To debug services, I generally use the following boilerplate for Main:
static void Main()
{
ServiceBase[] servicesToRun = new ServiceBase[] { new MyService(); };
#if !DEBUG
//run the service normally using ServiceBase.Run
ServiceBase.Run(servicesToRun);
#else
//debug the process as a non-service by invoking OnStart and then sleeping
foreach (ServiceBase s in servicesToRun)
{
var serviceType = s.GetType();
var onStartMethod = serviceType.GetMethod("OnStart", System.Reflection.BindingFlags.Instance |
System.Reflection.BindingFlags.NonPublic);
onStartMethod.Invoke(s, new object[] { new string[0] });
}
Debug.WriteLine("done starting services");
while (true)
Thread.Sleep(200);
#endif
}
What this does is use reflection to get the OnStart protected method for each service, invoke it, then sit in a sleep loop to keep the process running.
This example is assuming MyService is the service class that inherits from ServiceBase, which overrides OnStart to spawn off thread(s) and do its thing.
Here I'm using the DEBUG compile-time constant as the controlling factor of whether to start normally or debug as a non-service. If DEBUG is defined it will run in-process as a non-service. Otherwise it does its normal thing calling ServiceBase.Run. Feel free to use a command-line argument or whatever works best for you.
You may need some tweaks or re-factoring most of the logic to console app for easier testing, debugging, as what comes out of the box is really painful.
Here are some resources :
http://candordeveloper.com/2012/12/27/debugging-a-windows-service-application-without-install/
Running Windows Service Application without installing it
the easiest of them might be putting
#if DEBUG
//your logic
#endif
in your logic, putting a breakpoint and hitting F5, this is what i do most of the time.
Would be interesting to know if there is even better ways :)
Answers from Bravo 11 and Ic. give some good tips and tricks. I will add some:
Be aware that running as as service has also some implications regarding login / security context:
running under SYSTEM, you cannot access network file shares normally (there is a way adding the machine account to the ACL however)
running under a (domain) user account needs "logon as service" privileges, which is IMHO the most common cause of service starting problems in enterprise environments (the GPO settings may be adjusted). In addition, you do not have access to the desktop, so if a popup will be displayed waiting for the user, the service hangs.
in the development phase, you can swap DLLs or even the EXE if the service is not currently running without uninstall / install
use Windows event log extensively.
These differences between services and "normal application" are important enough for me to let this run directly on a server as often as possible.
In my services, I use:
protected override void OnStart(string[] args)
{
// this is not just a simple message, this has to be called very early before any worker thread
// to prevent a race condition in the .NET code of registering the event source
EventLog.WriteEntry("XXXXService is starting", EventLogEntryType.Information, 1000);
....
And do not forget that OnStart() and OnStop()should not contain long running code. You normally start a worker thread in OnStart() which runs until some notification (set in OnStop) was triggered.

C# Windows Service fails to start fast enough

I have a C# Windows Service. The OnStart() method contains 3 lines that all they do is start a thread. The class itself has no static objects that would cause a delay. It is as simple as a service can be. My code is signed by a self made key.
Edit: I just figured out that assembly isn't signed, the rest are. It uses a few assemblies which are signed but the service itself isn't.
This code is running inside a virtual machine, whenever the host is running slow and this VM is booting up, the service will fail to start with this error:
A timeout was reached (30000 milliseconds) while waiting for the ServiceName service to connect.
Setting the service to delay start fixes it but it's a long delay and it's very annoying. Has anyone else had this problem with .NET services (2.0)?
The OnStart method:
protected override void OnStart(string[] args)
{
Thread startThread = new Thread(new ThreadStart(StartThread));
startThread.IsBackground = true;
startThread.Start();
}
The StartThread method, in which all I do is call another class so that the OnStart won't have to wait for static variable initialization or for the constructor method to end.
private void StartThread()
{
Worker mainThread = new Worker(this);
mainThread.RunWorker();
}
Thank you.
I think this is most likely due to the strong name signing that you have on your executable. In general, it's not good practice to do that on executables (see here). I have also experienced the same problem in one of my projects, where an executable took an incredible amount of time to start, and turned out it was due to strong name signing the executable (.NET 2.0 as well).
Consider turning AutoLog = false. I've seen where this flag can cause the timeout you're describing.

System calls from .NET to mono [duplicate]

Here is the premise:
I have a desktop that I need to be able to start up and stop applications on, but cannot get remote access to. What I had in mind is setting up a service on the machine that will start/stop a list of applications as told. This windows service will periodically pole a web service for new commands and execute them accordingly.
These are my questions.
1) Is this the easiest solution? What else would you recommend?
2) How hard is it to run an exe from a windows service? How about stopping one?
This isn't for a project or anything, just something I am interested in implementing (mostly for fun). Any answers or even thoughts are appreciated. General discussion is also welcome (feel free to leave comments).
As for creating the Windows service itself in C#, see my post here.
The polling mechanism would work, but in general, I prefer event-driven processes instead of polling processes. You didn't mention what version of .NET you were using, but if it is .NET 3.0/3.5, I would suggest using WCF. When the command is posted to the web service, the web service could send the command to the Windows service to be executed. Pretty straightforward. Juval Lowy, the author of Programming WCF Services, offers a bunch of WCF examples/libraries that are free to use at his website.
So I guess PsExec is out of question?
Other than that, it's not hard to implement running of programs inside a Win service. Simply use the .NET Process class to do it, sample from my code:
ProcessStartInfo processStartInfo = new ProcessStartInfo (programExePath, commandLineArgs);
consoleLogger.WriteLine (log, Level.Debug, "Running program {0} ('{1}')", programExePath, commandLineArgs);
processStartInfo.CreateNoWindow = true;
processStartInfo.ErrorDialog = false;
processStartInfo.RedirectStandardError = true;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.UseShellExecute = false;
using (Process process = new Process ())
{
process.StartInfo = processStartInfo;
process.ErrorDataReceived += new DataReceivedEventHandler (process_ErrorDataReceived);
process.OutputDataReceived += new DataReceivedEventHandler (process_OutputDataReceived);
process.Start ();
process.BeginOutputReadLine ();
process.BeginErrorReadLine ();
if (false == process.WaitForExit ((int)TimeSpan.FromHours(1).TotalMilliseconds))
throw new ArgumentException("The program '{0}' did not finish in time, aborting.", programExePath);
if (process.ExitCode != 0)
throw new ArgumentException ("failed.");
}
I have done a service that starts other exe files. There are some things to consider: As a service you have no visual desktop -> no direct interaction is possible. Also, the program must be designed to run a long while.
Stopping an exe is to kill the process. Consider the usual side-effects.
It wouldn't be too difficult - you can just have it poll some webservice for a list of "banned" applications every so often, and then on another timer, have it check for a banned application that's running and kill it if it is. Here's some code to get a list of processes, and you can use the methods on the process class to launch or kill something:
http://www.vbdotnetheaven.com/UploadFile/prvn_131971/machineprocessvb11182005001454AM/machineprocessvb.aspx
The timers could run independently as well - it can check the webservice once/day for a new add/prohibited list, but then check the processes running on the computer every 2 minutes, for example.
I'll answer question 2 first:
To start or stop programs, you just need to use the System.Diagnostics.Process object, and the example code is right in the MSDN library.
For question 1:
You could have it poll a web service periodically if you want to do that.
As an alternate, you can set up the service for remoting and have it just listen on a tcp port for function calls, then write a Windows app that can use remoting to call the service. I've had to do this for a legitimate business app and it works very well.
However, I would be very irresponsible if I didn't include this warning:
Either way, setting up a computer so that it can execute arbitrary code is a bad idea and should be done very carefully. Just because it can be done doesn't mean it should be done. If you go the web service route, how are you going to ensure that someone doesn't tamper with the web service and get malicious code to execute? How do you know someone won't mess with your app.config and point the app to their own web service? In the remoting scenario, how to you ensure that some other .Net developer doesn't just create a proxy of their own?
In all honesty, I'm amazed that Microsoft even allows the System.Diagnostocs.Process to be used in a .Net application. You can literally do anything you want with it, including launching the cmd shell and executing system commands. Therefore, I'd urge you to seriously consider if this is truly necessary to do. You may be better served installing vnc or using remote desktop, or another app for remote access.
One other question you need to answer is "should the application be visible to the user on that remote machine?" If so, then you need to ensure that the service is set up to run under that user's context. otherwise, you could, for example, launch Word using the local system account, and it would never be visible to the person logged into that machine.

Windows Service Application Controller

Here is the premise:
I have a desktop that I need to be able to start up and stop applications on, but cannot get remote access to. What I had in mind is setting up a service on the machine that will start/stop a list of applications as told. This windows service will periodically pole a web service for new commands and execute them accordingly.
These are my questions.
1) Is this the easiest solution? What else would you recommend?
2) How hard is it to run an exe from a windows service? How about stopping one?
This isn't for a project or anything, just something I am interested in implementing (mostly for fun). Any answers or even thoughts are appreciated. General discussion is also welcome (feel free to leave comments).
As for creating the Windows service itself in C#, see my post here.
The polling mechanism would work, but in general, I prefer event-driven processes instead of polling processes. You didn't mention what version of .NET you were using, but if it is .NET 3.0/3.5, I would suggest using WCF. When the command is posted to the web service, the web service could send the command to the Windows service to be executed. Pretty straightforward. Juval Lowy, the author of Programming WCF Services, offers a bunch of WCF examples/libraries that are free to use at his website.
So I guess PsExec is out of question?
Other than that, it's not hard to implement running of programs inside a Win service. Simply use the .NET Process class to do it, sample from my code:
ProcessStartInfo processStartInfo = new ProcessStartInfo (programExePath, commandLineArgs);
consoleLogger.WriteLine (log, Level.Debug, "Running program {0} ('{1}')", programExePath, commandLineArgs);
processStartInfo.CreateNoWindow = true;
processStartInfo.ErrorDialog = false;
processStartInfo.RedirectStandardError = true;
processStartInfo.RedirectStandardOutput = true;
processStartInfo.UseShellExecute = false;
using (Process process = new Process ())
{
process.StartInfo = processStartInfo;
process.ErrorDataReceived += new DataReceivedEventHandler (process_ErrorDataReceived);
process.OutputDataReceived += new DataReceivedEventHandler (process_OutputDataReceived);
process.Start ();
process.BeginOutputReadLine ();
process.BeginErrorReadLine ();
if (false == process.WaitForExit ((int)TimeSpan.FromHours(1).TotalMilliseconds))
throw new ArgumentException("The program '{0}' did not finish in time, aborting.", programExePath);
if (process.ExitCode != 0)
throw new ArgumentException ("failed.");
}
I have done a service that starts other exe files. There are some things to consider: As a service you have no visual desktop -> no direct interaction is possible. Also, the program must be designed to run a long while.
Stopping an exe is to kill the process. Consider the usual side-effects.
It wouldn't be too difficult - you can just have it poll some webservice for a list of "banned" applications every so often, and then on another timer, have it check for a banned application that's running and kill it if it is. Here's some code to get a list of processes, and you can use the methods on the process class to launch or kill something:
http://www.vbdotnetheaven.com/UploadFile/prvn_131971/machineprocessvb11182005001454AM/machineprocessvb.aspx
The timers could run independently as well - it can check the webservice once/day for a new add/prohibited list, but then check the processes running on the computer every 2 minutes, for example.
I'll answer question 2 first:
To start or stop programs, you just need to use the System.Diagnostics.Process object, and the example code is right in the MSDN library.
For question 1:
You could have it poll a web service periodically if you want to do that.
As an alternate, you can set up the service for remoting and have it just listen on a tcp port for function calls, then write a Windows app that can use remoting to call the service. I've had to do this for a legitimate business app and it works very well.
However, I would be very irresponsible if I didn't include this warning:
Either way, setting up a computer so that it can execute arbitrary code is a bad idea and should be done very carefully. Just because it can be done doesn't mean it should be done. If you go the web service route, how are you going to ensure that someone doesn't tamper with the web service and get malicious code to execute? How do you know someone won't mess with your app.config and point the app to their own web service? In the remoting scenario, how to you ensure that some other .Net developer doesn't just create a proxy of their own?
In all honesty, I'm amazed that Microsoft even allows the System.Diagnostocs.Process to be used in a .Net application. You can literally do anything you want with it, including launching the cmd shell and executing system commands. Therefore, I'd urge you to seriously consider if this is truly necessary to do. You may be better served installing vnc or using remote desktop, or another app for remote access.
One other question you need to answer is "should the application be visible to the user on that remote machine?" If so, then you need to ensure that the service is set up to run under that user's context. otherwise, you could, for example, launch Word using the local system account, and it would never be visible to the person logged into that machine.

Categories