I have a .NET windows service which acts as a host for some wcf. In the OnStart method the service hosts are created and started. The service is configured to startup automatically. This works well on Windows 7 (32bit and 64bit) and it can be startet with "net start" on Windows XP Pro SP3. The service startup with "net start" command takes about 20 seconds.
But when Windows XP Pro SP3 is booting there's a timeout message in the event log. The service itself does not fail to startup, though do its dependencies. The problem can be reproduced on various XP machines. Core count and memory does not have an influence. The updates are up to date.
Now it's getting curious: I analyzed the trace and found out that the service is taking about 60 seconds for startup. Thus I've added a call to ReqestAdditionalTime(480000). But now the service takes slightly more than 480 seconds. The relation is obvious. The time is consumed in the following code section:
var asyncResults = new List<IAsyncResult>();
foreach (var host in myHosts)
asyncResults.Add(host.BeginOpen(null, host));
// wait until finished
while (asyncResults.Count != 0)
{
IAsyncResult ar = asyncResults[0];
if (!ar.IsCompleted) ar.AsyncWaitHandle.WaitOne(1000);
if (ar.IsCompleted)
{
asyncResults.Remove(ar);
var co = (ICommunicationObject)ar.AsyncState;
try
{
co.EndOpen(ar);
}
catch (Exception ex)
{
...
}
}
}
Do you have any idea what's happening here?
Hey, I found the resolution myself by doing some intensive Log-Research.
In the event log there were some services, which started AFTER the timeout of my service has been reached. As my service is running as a sepecial user, I could detect two services, which where acutally triggered by my own service. Thus I added those to the services dependencies and it works.
I wonder if there's a documentation, where the dependencies of wcf are listed.
As reference here are the services, my service is dependen on:
http
RPCSS
CryptSvc
HTTPFilter
RasMan
Latter two where those causing the deadlock.
Related
I have created a GRPC Server in C# using the example given at Link. Now I want to figure out as how should I be hosting this server so that I achieve following:
Should I make this Server a Console application or a a Windows Service. If I make it a windows Service then updating the service will be cumbersome (which is a big negative) and if I make it a console app then updating will simply need shutting down exe. But that comes with the price of closing the same by mistake. Is there any other better way?
With IIS this issue won't b there as I can simply remove the site from LB and stop the website to perform the update but since GRPC won't be a part of IIS, I am not sure what's the way to get this working.
Any references for the better architecture are welcomed.
We can use Microsoft.Extensions.Hosting pacakge to host a .net core console application by using the HostBuilder API to start building gRPC host and setting it up.
In order to run the gRPC service, we first need to start/stop Grpc.Core.Server in a hosted service. A hosted service is basically a piece of code that is run by the host when the host itself is started and the same for when it is stopped. The following code implement a GrpcHostedService to override IHostedService interface:
using System.Threading;
using System.Threading.Tasks;
using Grpc.Core;
using Microsoft.Extensions.Hosting;
namespace Grpc.Host
{
public class GrpcHostedService: IHostedService
{
private Server _server;
public GrpcHostedService(Server server)
{
_server = server;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_server.Start();
return Task.CompletedTask;
}
public async Task StopAsync(CancellationToken cancellationToken) => await _server.ShutdownAsync();
}
}
In the Program.cs, use HostBuilder API to start building our grpc host and setting it up:
public class Program
{
public static async Task Main(string[] args)
{
var hostBuilder = new HostBuilder()
// Add configuration, logging, ...
.ConfigureServices((hostContext, services) =>
{
// Better to use Dependency Injection for GreeterImpl
Server server = new Server
{
Services = {Greeter.BindService(new GreeterImpl())},
Ports = {new ServerPort("localhost", 5000, ServerCredentials.Insecure)}
};
services.AddSingleton<Server>(server);
services.AddSingleton<IHostedService, GrpcHostedService>();
});
await hostBuilder.RunConsoleAsync();
}
}
By doing this, the generic host will automatically run StartAsync on our hosted service, which in turn will call StartAsync on the Server instance, essentially start the gRPC server.
When we shut down the host with Control-C, the generic host will automatically call StopAsync on our hosted service, which again will call StopAsync on the Server instance which will do some clean up.
For other configuration in HostBuilder, you can see this blog.
I'm going to add one more option.
With dot net core, you can run this as a Linux Daemon now.
Currently gRPC doesn't support integration with ASP.Net/IIS. You would need to host the server in a console or as a Windows service.
Likely you would want this to be a Windows service to make it easier to keep the server running across reboots or crashes. If you want to easily turn your console application into a Windows service I would recommend using the excellent TopShelf Nuget.
Updating the service can be done as you would a console app.
Stop the Windows service. net stop <service-name}>
Copy the updated assemblies.
Start the Windowsservice net start <service-name>
My company (Shortbar) is building the application server for a hotel management system called HOLMS on gRPC. Our setup is as follows:
HOLMS.Application is a .NET class library (assembly) that does the actual work of the server
HOLMS.Application.ConsoleRunner is a C# console application that hosts HOLMS.Application. The console runner is used by (1) developers for convenience (mentioned in the question) as well as (2) production scenarios running inside a Docker container, where the container runtime (e.g. Amazon ECS) implements job control/scaling. It follows "12 factor app" guidelines, including running itself as a single, standalone, stateless process, fast startup/shutdown, and environment-variable config injection. The system logs to stdout which gets drained however stdout is drained in the prod environment (e.g. Sumo, logstash, etc). This is how our SaaS multi-tenant solution will go into production.
HOLMS.Application.ServiceRunner packages HOLMS.Application into a Windows service, for more traditional, on-premise situations where a customer's IT group will run the service themselves. This package uses the Windows registry for configuration and relies on Windows service job control for startup/shutdown/restarts. It logs to the Windows Event Log.
The ConsoleRunner and ServiceRunner each are only about 200 lines of code; for the most part, they just wrap the Application package, and call into it.
Hope this helps.
I am trying to use SignalR in my web application and it is hosted on IIS 7.5.
I am using Windows 7 X64 (without SP1) and .Net 4.5 with SignalR v2.0.3.0.
If I run the web application from the IIS express it works fine and nothing hangs up but as soon as I run it from IIS 7.5 it hangs up my app. I confirm that if I disable this piece of code:
$.connection.hub.start().done(function () {
});
everything works fine. Here is the screenshots which suggests the all request go in a indefinite block state after hub is started. Even hub takes around 8-9 seconds to respond but even after that if I request a simple JPG I get nothing and goes in indefinite block state.
Here is the screenshot of the application pool. I am running in Integrated mode.
This is the Hub class:
public class AppHub : Hub
{
public override Task OnConnected()
{
if (Context.User != null)
{
var email = Context.User.Identity.Name;
if (!string.IsNullOrEmpty(email))
{
UserRepository userRepository = new UserRepository();
userRepository.DoSomething(email, Context.ConnectionId);
}
}
return base.OnConnected();
}
public override Task OnDisconnected()
{
if (Context.User != null)
{
var email = Context.User.Identity.Name;
if (!string.IsNullOrEmpty(email))
{
UserRepository userRepository = new UserRepository();
userRepository.DoSomething1(email, Context.ConnectionId);
}
}
return base.OnDisconnected();
}
}
The documentation states that you should be using IIS express on your dev machine, because of the limit on the amount of concurrent requests
When SignalR is hosted in IIS, the following versions are supported.
Note that if a client operating system is used, such as for
development (Windows 8 or Windows 7), full versions of IIS or Cassini
should not be used, since there will be a limit of 10 simultaneous
connections imposed, which will be reached very quickly since
connections are transient, frequently re-established, and are not
disposed immediately upon no longer being used. IIS Express should be
used on client operating system
Same thing with Windows 10.
taken from SignalR documentation here :
http://www.asp.net/signalr/overview/signalr-20/getting-started-with-signalr-20/supported-platforms
This is probably because of request execution limits on Windows 7 which limit how many concurrent requests there can be at a time before they get queued). According to this table here, IIS 7.5 has a request execution limit of 3 or even 1 depending on your OS, which is probably used up by the SignalR connection.
I have the windows service program which runs and sometime throw exception about system null reference. After my investigation, this is due to MySQL connection cannot establish due to MySql instance not up yet when startup computer. How to solve this problems???
thanks in advance
You can set in Windows the dependencies of each service.
You can go into Control Panel -> Administrative Tools -> Services (or run 'services.msc' from the command line). By double clicking any of the services and going to the Dependencies tab you can see what each service relies on.
Your service relies on the MySql service, so MySql should be in the dependencies list for it.
You can add items to the dependencies, here is a description of how to do this:
https://serverfault.com/questions/24821/how-to-add-dependency-on-a-windows-service-after-the-service-is-installed
This article can give you a hint on how code service dependencies:
http://bloggingabout.net/blogs/jschreuder/archive/2006/12/07/How-to_3A00_-Code-Service-Dependencies.aspx
Try the following code if you want to do it by yourself without dependencies
//Check if service is running
ServiceController mysqlServiceController = new ServiceController();
mysqlServiceController.ServiceName = "MySql";
var timeout = 3000;
//Check if the service is started
if (mysqlServiceController.Status == System.ServiceProcess.ServiceControllerStatus.Stopped
|| mysqlServiceController.Status == System.ServiceProcess.ServiceControllerStatus.Paused)
{
mysqlServiceController.Start();
try
{
//Wait till the service runs mysql
ServiceController.WaitForStatus(System.ServiceProcess.ServiceControllerStatus.Running, new TimeSpan(0, timeout, 0));
}
catch (System.ServiceProcess.TimeoutException)
{
//MessageBox.Show(string.Format("Starting the service \"{0}\" has reached to a timeout of ({1}) minutes, please check the service.", mysqlServiceController.ServiceName, timeout));
}
}
I've created a small executable that can be launched either as a normal application by calling MyApp.exe or as a service by calling MyApp.exe -s. Because I'm trying to keep as simple as possible, I "install" this app by manually running
sc create MyAppService binPath= "C:\MyApp\MyApp.exe -s"
Then I start the service with net start MyAppService like normal.
On two Windows XP machines and two Windows 2000 machines, this works fine. However, on two different Windows XP Embedded machines, when I try to start the service I get the message:
System error 1083 has occurred.
The executable program that this service is configured to run in does not implement the service.
On one machine, I was able to fix this by uninstalling and reinstalling .NET 2.0, but on the second machine this did not work.
I'm not sure how to go about debugging this, and searching google only seems to turn up specific services that fail with this message such as BITS and an Exchange service.
Below are the classes MyApp, which is the startup class, and MyAppService, which is the class that extends ServiceBase. Thanks in advance for any direction on this.
MyApp.cs
static class MyApp
{
[STAThread] static void Main( string[] args )
{
....
switch ( arg1 )
{
case "-s":
ServiceBase[] ServicesToRun;
ServicesToRun = new ServiceBase[] { new MyAppService() };
ServiceBase.Run( ServicesToRun );
break;
....
}
}
}
MyAppService.cs:
class MyAppService : ServiceBase
{
static MyAppService()
{
// ...
}
protected override void OnStart( string[] args )
{
// ...
}
}
On the desktop, this can happen if the service isn't registered correctly in the Windows Registry under the account that the svchost instance is supposed to run under. I don't have experience in XPe, but try looking in HKLM\Software\Microsoft\Windows NT\CurrentVersion\Svchost and make sure that MyAppService is correctly listed for the account.
Try to check in the Event log if there is useful info including Security log.
It seems did not recognize the MyAppService as a service or MyApp.exe does not expose any services to the XPe. Focus on this thing to get the root cause.
For fast testing, you can get XPe run in your development PC by using VMWare. VMWare has the way to copy the current running XPe into image and copy to your PC but not sure if it can work properly.
It appears that I have the same problem. The ServiceController.Start() does not start service successfully. The application is in C# .NET2 and running in Window XPe. The work around is below:
TimeSpan timeout = TimeSpan.FromMilliseconds(20000);
while (true)
{
ServiceController service = new ServiceController("myservice");
service.MachineName = ".";
try
{
service.Start()
service.WaitForStatus(ServiceControllerStatus.Running, timeout);
}
catch
{
service.Stop();
continue;
}
}
after looping 2 or 3 times, the service usually get started successfully.
But 30-40 seconds has passed. This is not acceptable.
Dos anybody have experienced on this issue? Thanks!
I have encountered a strange situation where starting an application pool from a windows service (written in C#, set to "Automatic" startup) using WMI or ADSI immediately after the server reboots hangs.
I'll describe the issue:
We are developing a large application (Windows 2003 Server SP2, IIS 6.0) which contains the following main processes (these processes are invoked & initialized using a windows service startup procedure when the application is started):
1) XServer1.exe, XServer2.exe - These processes are native COM-Exe servers, contains some logics, but mainly supplies COM objects to other processes via DCOM (mainly .NET2COM interOp calls & pure COM calls). For example, some of the classic ASP "Application Scope static objects" (w3wp.exe) are COM objects which "live" inside these processes.
2) dllhost.exe - this is a COM+ application. Some of our DLLs are loaded into this process which acts as a "state server" (the same idea as the ASP.NET out-of-proc sessions server, but for classic ASP pages).
3) 3 different IIS application pools (we'll call them appPool1\2\3) - containers of our ASP pages, ASP.NET pages, WCF services etc. Code (native C++ COM dlls & C#) in these application pools (w3wp.exe's) usually makes DCOM calls to the processes described in (1) & (2). Only appPool1 can be configured as a Web Garden.
In order to Start\Stop our application we have written a windows Service (C#) which controls these procedures. Our service process is called XWinService.exe. The service depends on the following windows services (the list began with the first 4 services, ongoing tries made the list like this...):
W3SVC
aspnet_state
COMSysApp
DcomLaunch
winmgmt
lanmanserver
lanmanworkstation
seclogon
Browser
TermService
The summary of the Stop procedure of the application (implemented by the service):
1) Stop all 3 IIS application pools (appPool1\2\3) - This is done to prevent w3wp.exe processes to jump alive when the application is shut-down. This is implemented with WMI from C# (system.Management.dll)
2) Stop XServer1\2.exe
3) Stop the COM+ application (dllhost.exe).
The summary of the Start procedure of the application (implemented by the service):
1) Execute the Stop procedure - This ensures that no HTTP hits will wake a w3wp.exe process before it's time.
2) Invokes & Initializes the XServer1\2.exe COM-Exe servers - Initialization is required prior to any w3wp.exe invocation. Only after some object had been initialized, w3wp.exe's can access these servers. This is implemented by .NET2COM InterOp (eventually DCOM).
3) Invokes & initialized the dllhost.exe (COM+ application) process - This is implemented by the ComAdmin Catalog API (C#).
4) Starts our 3 application pools - This allows incoming HTTP hits to wake w3wp.exe processes and start serving requests.
This is the C# code which is responsible to start\stop application pools (WMI). This code runs in our service processes (XWinService.exe):
ConnectionOptions co = new ConnectionOptions();
ManagementScope scope = new ManagementScope(#"\\localhost\root\MicrosoftIISV2", co);
foreach (string appPool in AppPools)
{
string objPath = string.Format("IISApplicationPool.Name='W3SVC/AppPools/{0}'", appPool);
using (ManagementObject mc = new ManagementObject(objPath))
{
mc.Scope = scope;
if (Operation.ToLower() == "start")
{
mc.InvokeMethod("Start", null, null); // ### The problematic line of code ###
}
else if (Operation.ToLower() == "stop")
{
mc.InvokeMethod("Stop", null, null);
}
else if (Operation.ToLower() == "recycle")
{
mc.InvokeMethod("Recycle", null, null);
}
}
}
Now the issue:
Prior to rebooting the server, starting the service manually (from the services.msc tool) succeeds without any problems. also, stopping it is OK. We have set the service to start "Automatic", that is, will start when the server (Win2K3 SP2) starts and rebooted the server. When the server started (the login screen appeared), our service was "stuck" (status = "Starting") and will NEVER (it hang for 2 days!) start.
Analyzing the processes reveled the following:
1) The XWinService.exe process was stuck on the problematic line of code (### above ###). This hanged for 2 days until we killed the process. Please note: Shutting down the application pools (the Start procedure begins with a Stop procedure) did not hang!
2) From a DUMP file taken (with DebugDiag tool) from XWinService.exe during this "hang" we can see the thread which is waiting. This is the (native) stack trace of it:
Thread 6 - System ID 2784
Entry point mscorwks!Thread::intermediateThreadProc
Create time 11/19/2009 1:40:05 PM
Time spent in user mode 0 Days 00:00:00.078
Time spent in kernel mode 0 Days 00:00:00.781
This thread is making a COM call to multi-threaded apartment (MTA) in process 884
Function Source
ntdll!KiFastSystemCallRet
ntdll!NtRequestWaitReplyPort+c
rpcrt4!LRPC_CCALL::SendReceive+230
rpcrt4!I_RpcSendReceive+24
ole32!ThreadSendReceive+138
ole32!CRpcChannelBuffer::SwitchAptAndDispatchCall+112
ole32!CRpcChannelBuffer::SendReceive2+d3
ole32!CAptRpcChnl::SendReceive+ab
ole32!CCtxComChnl::SendReceive+1a9
rpcrt4!NdrProxySendReceive+43
rpcrt4!NdrClientCall2+206
rpcrt4!ObjectStublessClient+8b
rpcrt4!ObjectStubless+f
….
This thread is calling (via DCOM) a component in process 884, which is svchost.exe, running the following services: AeLookupSvc, AudioSrv, Browser, CryptSvc, dmserver, EventSystem, helpsvc, lanmanserver, lanmanworkstation, Schedule, seclogon, SENS, ShellHWDetection, TrkWks, winmgmt, wuauserv, WZCSVC.
As you can see the "winmgmt" service (responsible for WMI) is running in this process and our service depends on it, so our service will start after winmgmt is started (the same for IIS W3SVC service).
The svchost.exe process (884) was dumped and we can see a thread (waiting for a DCOM call to end) accessing process 2880 which is - wmiprvse.exe (I guess this is the WMI server. Don't know if it's relevent, but there were 2 instances of this process). This is the native call stack of the thread (in svchost.exe):
Thread 48 - System ID 3816
Entry point wbemcore!CCoreQueue::_ThreadEntry
Create time 11/19/2009 1:40:56 PM
Time spent in user mode 0 Days 00:00:00.00
Time spent in kernel mode 0 Days 00:00:00.00
This thread is making a COM call to multi-threaded apartment (MTA) in process 2880
Function Source
ntdll!KiFastSystemCallRet
ntdll!NtRequestWaitReplyPort+c
rpcrt4!LRPC_CCALL::SendReceive+230
rpcrt4!I_RpcSendReceive+24
ole32!ThreadSendReceive+138
ole32!CRpcChannelBuffer::SwitchAptAndDispatchCall+112
ole32!CRpcChannelBuffer::SendReceive2+d3
ole32!CAptRpcChnl::SendReceive+ab
ole32!CCtxComChnl::SendReceive+1a9
…
3) Setting our service to "Manual" and starting it (manually - after logging into the server or starting it remotely from a different server immediately after reboot) is OK - nothing hangs.
4) We deleted our service (from the registry!) and placed a batch file in the windows "startup" folder. This batch files calls the service's code, but runs it as a normal C# executable. After server reboot, it also hang on the same problematic line of code (again... for 2 days until we killed it).
5) Using ADSI (System.DirectoryServices) instead of WMI had the same results (starting the application pools hanged!).
We have been digging into this for the past 2 weeks...
My questions:
==========
1) Did anyone encounter the same issue?
2) Does anyone know why it hangs? Is there any additional service dependency we should take in mind?
3) Does anyone have a solution for this issue?
4) Why is this happening after a reboot only when the service to set to "Automatic" startup? If we do it manually - everything is Ok!
***** Small update:**
We have noticed that on VMs (VMware stations) the service hangs after reboot for an average of ~40min, until it starts (note: it never fails to start, but 40min is way too much). An event log message is recorded in the system event log stating that our service hanged for more than 16min (source: Service Control Manager, Event ID: 7044).
On "regular" machines (real metals) the average time until the service starts is ~55 hours!!! Again, an event log entry is recorded as described above.
The avergae values were calculated from 10 differens VMs & 8 different "real" servers.
I see no one had responded, but I'll post some news anyway...
We have found out, that prior to starting the application pools, setting the service status to "Started" and opening a new thread (new Thread(...)) which runs the code above (starting the app pools with WMI) solves the issue.
This is the pseudo code of the OnStart method of the service:
OnStart {
StopProcedure();
InvokeInitXServer1And2(); //COM-Exe servers
InvokeInitCOMPlusApplication(); //dllhost.exe
SetServiceStatus(SERVICE_STARTED);
Thread worker = new Thread(new threadStart(IISAppPoolStartWMI); //Calls the code
}
This is the only way the service starts in reasonable time (Max of 3 min, Avg of ~1.5 min of real machines and VMs both!) and a w3wp.exe processes is started.
If anyone has an explnation for it (MTA\STA issues?!?!?) I'll be happy to read it.