We Keep Coding

IHost not returning when Task completed

I'm writing a Windows Service (using .NET Core 3.1), using a BackgroundService and I seem to have an issue when I want to programmatically stop the service.
My main function is as follows
static async Task Main(string[] args)
{
IHost host = null;
try
{
host = CreateService(args).Build();
await host.RunAsync();
Console.WriteLine("Ending");
}
catch (Exception ex)
{
Console.WriteLine("Exception");
}
finally
{
if (host is IAsyncDisposable d) await d.DisposeAsync();
}
}
public static IHostBuilder CreateService(string[] args) =>
Host.CreateDefaultBuilder(args)
.UseWindowsService(options =>
{
options.ServiceName = "My Service";
})
.ConfigureServices((hostContext, services) =>
{
IConfiguration configuration = hostContext.Configuration;
ServiceOptions options = configuration.GetSection("ServiceOptions").Get<ServiceOptions>();
WorkerService sService = new WorkerService();
services.AddSingleton(options);
services.AddSingleton(sService);
services.AddHostedService<WindowsBackgroundService>(service => new WindowsBackgroundService(
service.GetService<WorkerService>(),
service.GetService<ILogger<WindowsBackgroundService>>(),
service.GetService<ServerOptions>()
));
});
The background service is as follows:
public sealed class WindowsBackgroundService : BackgroundService
{
private WorkerService _workerService;
private ServerOptions _options;
private ILogger<WindowsBackgroundService> _logger;
public WindowsBackgroundService(
WorkerService workerService, ILogger<WindowsBackgroundService> logger,
ServiceOptions serviceOptions) => (_workerService, _logger, _options) = (workerService, logger, serviceOptions);
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
bool allDone = false;
while (!stoppingToken.IsCancellationRequested && !allDone)
{
await Task.Delay(TimeSpan.FromSeconds(15), stoppingToken);
// Log to watchdog
// Check if we are in the run window
if (_options.InActivePeriod())
{
Console.WriteLine("Process Queue");
var processResult = _workerService.Process(_options);
if (!processResult && _workerService.FatalError)
{
_workerService = null;
allDone = true;
Console.WriteLine("Service Quitting");
}
}
}
Console.WriteLine($"Task Ending {stoppingToken.IsCancellationRequested}");
return;
}
}
}
So everything runs as it should and I run this under the debugger or from the command line (I haven't actually installed it as a Windows Service yet as I'm still writing the code).
The function Process executes correctly. If it encounters an error that cannot be recovered, it sets it's FatalError property that is supposed to be a signal that the whole service should be stopped. The Task does indeed complete (the correct lines are written to console) but the line Console.WriteLine("Ending"); is never executed. It looks like the host.RunAsync(); never returns (unless I do a CTRL+C).
I'm not entirely certain what I am doing wrong at this point. Can anyone point me in the write direction?

Based on the shown code I see nothing that would cause the host to stop.
The hosted service, once started has no bearing on the application host itself. So even when ExecuteAsync is completed, the completion of that function does not mean that the host will stop running.
You could consider injecting IHostApplicationLifetime into the hosted service and explicitly telling the application to stop programmatically;
For example
public sealed class WindowsBackgroundService : BackgroundService {
private WorkerService _workerService;
private ServerOptions _options;
private ILogger<WindowsBackgroundService> _logger;
private IHostApplicationLifetime lifetime;
public WindowsBackgroundService(
WorkerService workerService, ILogger<WindowsBackgroundService> logger,
ServiceOptions serviceOptions, IHostApplicationLifetime lifetime)
=> (_workerService, _logger, _options, this.lifetime) = (workerService, logger, serviceOptions, lifetime);
protected override async Task ExecuteAsync(CancellationToken stoppingToken) {
bool allDone = false;
while (!stoppingToken.IsCancellationRequested && !allDone) {
await Task.Delay(TimeSpan.FromSeconds(15), stoppingToken);
// Log to watchdog
// Check if we are in the run window
if (_options.InActivePeriod()) {
Console.WriteLine("Process Queue");
var processResult = _workerService.Process(_options);
if (!processResult && _workerService.FatalError) {
_workerService = null;
allDone = true;
Console.WriteLine("Service Quitting");
lifetime.StopApplication(); //SIGNAL APPLICATION TO STOP
}
}
}
Console.WriteLine($"Task Ending {stoppingToken.IsCancellationRequested}");
return;
}
}
I am also curious about how you configured your services. Why use the factory delegate when everything you resolved manually would have been injected automatically if the default registration was done?
//...
.ConfigureServices((hostContext, services) => {
IConfiguration configuration = hostContext.Configuration;
ServiceOptions options = configuration.GetSection("ServiceOptions").Get<ServiceOptions>();
services.AddSingleton(options);
services.AddSingleton<WorkerService>();
services.AddHostedService<WindowsBackgroundService>();
});
The logging and and host lifetime are added for you by the Host.CreateDefaultBuilder

How to start BackgroundService in ASP.Net core 3.1 API

I have tried to follow this tutorial on how to create a backgroundworker.
Most of the stuff wasn't useful for me, so I didn't include that. I have no need for a queue. I just need to have this backgroundworker running in the background doing stuff every X hours
My worker looks like this. Unfortunately it seems like it never calls the ExecuteAsync method
public class EnergySolutionBackgroundWorker : BackgroundService
{
private readonly ILogger<EnergySolutionBackgroundWorker> _logger;
public EnergySolutionBackgroundWorker(ILogger<EnergySolutionBackgroundWorker> logger)
{
_logger = logger;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
_logger.LogInformation("{Type} is now running in the background.", nameof(BackgroundWorker));
await BackgroundProcessing(stoppingToken);
}
public override Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogCritical("The {Type} is stopping due to a host shutdown.", nameof(BackgroundWorker));
return base.StopAsync(cancellationToken);
}
private async Task BackgroundProcessing(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
try
{
await Task.Delay(new TimeSpan(0, 0, 1), stoppingToken);
// Doing some tasks
}
catch (Exception ex)
{
_logger.LogCritical("An error occurred when publishing a book. Exception: {#Exception}", ex);
}
}
}
}
In Startup.cs I have the following:
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddHostedService<EnergySolutionBackgroundWorker>();
}
From my understanding, this should be enough for it to automatically start the backgroundworker during startup, but that is not the case. What am I doing wrong?

you can add timer in start function.
A timed background task makes use of the System.Threading.Timer class. The timer triggers the task's DoWork method. The timer is disabled on StopAsync and disposed when the service container is disposed on Dispose:
internal class TimedHostedService : IHostedService, IDisposable
{
private readonly ILogger _logger;
private Timer _timer;
public TimedHostedService(ILogger<TimedHostedService> logger)
{
_logger = logger;
}
public Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Timed Background Service is starting.");
_timer = new Timer(DoWork, null, TimeSpan.Zero,
TimeSpan.FromSeconds(5));
return Task.CompletedTask;
}
private void DoWork(object state)
{
_logger.LogInformation("Timed Background Service is working.");
}
public Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation("Timed Background Service is stopping.");
_timer?.Change(Timeout.Infinite, 0);
return Task.CompletedTask;
}
public void Dispose()
{
_timer?.Dispose();
}
}
there is a reference

I believe I found the answer! Startup.cs is not run before the first call to the API is made. After calling a simple Test method in one of my controllers, the BackgroundProcessing method was called
That's a bit annoying, as I was hoping I later could create a backgroundworker that loads a lot of data into memory instead of it happening when the first call is made

Start IHostedService after Configure()

I have an .NET Core 3.1 app that serves an endpoint that describes health of application, and an IHostedService crunching through data in database.
There's a problem though, the worker function of HostedService starts processing for a long time, and as result the Configure() method in Startup is not called and the /status endpoint is not running.
I want the /status endpoint to start running before the HostedService kicks off. How do i start the endpoint before the Hosted Service?
Sample code
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
services.AddHostedService<SomeHostedProcessDoingHeavyWork>();
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseRouting();
app.UseEndpoints(endpoints =>
{
endpoints.MapGet("/status", async context =>
{
await context.Response.WriteAsync("OK");
});
});
}
}
The HostedService
public class SomeHostedProcessDoingHeavyWork : BackgroundService
{
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await MethodThatRunsForSeveralMinutes();
await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
}
}
private async Task MethodThatRunsForSeveralMinutes()
{
// Process data from db....
return;
}
}
I tried to explore adding the HostedService in Configure(), but app.ApplicationServices is a ServiceProvider hence readonly.

I think proposed solutions are a kind of workarounds.
If you add your hosted service inside ConfigureServices(), it will be started before Kestrel because the GenericWebHostService (that in fact runs Kestrel), is added in Program.cs when you call
.ConfigureWebHostDefaults(webBuilder =>
webBuilder.UseStartup<Startup>()
)
so it's always being added as lasts.
To launch your hosted service after Kestrel, just chain another call to
.ConfigureServices(s => s.AddYourServices())
after the call to ConfigureWebHostDefaults().
Something like this:
IHostBuilder hostBuilder = Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(webBuilder => webBuilder.UseStartup<Startup>())
.ConfigureServices(s => {
s.AddHostedService<SomeHostedProcessDoingHeavyWork>();
});
and you should be done.

ExecuteAsync should return a Task and it should do so quickly. From the documentation (emphasis mine)
ExecuteAsync(CancellationToken) is called to run the background
service. The implementation returns a Task that represents the entire
lifetime of the background service. No further services are started
until ExecuteAsync becomes asynchronous, such as by calling await.
Avoid performing long, blocking initialization work in ExecuteAsync.
The host blocks in StopAsync(CancellationToken) waiting for
ExecuteAsync to complete.
You should be able to get around this by moving your logic into a seperate method and awaiting that
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await BackgroundProcessing(stoppingToken);
}
private async Task BackgroundProcessing(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
await MethodThatRunsForSeveralMinutes();
await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
}
}
Alternatively you might just be able to add an await at the start of the method:
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await Task.Yield();
while (!stoppingToken.IsCancellationRequested)
{
await MethodThatRunsForSeveralMinutes();
await Task.Delay(TimeSpan.FromMinutes(1), stoppingToken);
}
}

I ended up using Task.Yield() and implementing an abstract class to encapsulate it, with optional PreExecuteAsyncInternal hook and errorhandler ExecuteAsyncExceptionHandler
public abstract class AsyncBackgroundService : BackgroundService
{
protected ILogger _logger;
private readonly TimeSpan _delay;
protected AsyncBackgroundService(ILogger logger, TimeSpan delay)
{
_logger = logger;
_delay = delay;
}
public virtual Task PreExecuteAsyncInternal(CancellationToken stoppingToken)
{
// Override in derived class
return Task.CompletedTask;
}
public virtual void ExecuteAsyncExceptionHandler(Exception ex)
{
// Override in derived class
}
public abstract Task ExecuteAsyncInternal(CancellationToken stoppingToken);
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
// Prevent BackgroundService from locking before Startup.Configure()
await Task.Yield();
_logger.LogInformation("Running...");
await PreExecuteAsyncInternal(stoppingToken);
while (!stoppingToken.IsCancellationRequested)
{
try
{
await ExecuteAsyncInternal(stoppingToken);
await Task.Delay(_delay, stoppingToken);
}
catch (TaskCanceledException)
{
// Deliberate
break;
}
catch (Exception ex)
{
_logger.LogCritical($"Error executing {nameof(ExecuteAsyncInternal)} in {GetType().Name}", ex.InnerException);
ExecuteAsyncExceptionHandler(ex);
break;
}
}
_logger.LogInformation("Stopping...");
}
}

await Task.Yield didn't work for me.
The simplest obvious solution:
Startup.cs
public class Startup
{
public void ConfigureServices(IServiceCollection services)
{
// Implementation omitted
services.AddSingleton<ApplicationRunState>();
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
// Implementation omitted
app.MarkConfigurationAsFinished();
}
}
StartupExtensions.cs
public static void MarkConfigurationAsFinished(this IApplicationBuilder builder)
{
builder.ApplicationServices.GetRequiredService<ApplicationRunState>()
.ConfigurationIsFinished = true;
}
ExampleBackgroundService.cs
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
if (!_serviceProvider.GetRequiredService<ApplicationRunState>()
.ConfigurationIsFinished)
{
await Task.Delay(5000);
continue;
}
// Further implementation omitted
}
}

High load ASP.NET application running as Windows Service not stopping

An asp.net application (running on Kestrel) running as a Windows Service has trouble stopping during high load.
The setup:
[...]
var config = WebHostBuilderHelper.GetConfig();
var hostBuilder = new WebHostBuilder().UseKestrel(options => options.ConfigureEndpoints())
.UseConfiguration(config)
.UseStartup<Startup>()
.UseSerilog((hostingContext, loggerConfiguration) => loggerConfiguration.ReadFrom.Configuration(hostingContext.Configuration))
.UseContentRoot(AppDomain.CurrentDomain.BaseDirectory);
if (Debugger.IsAttached || args.ToList()
.Contains("console"))
{
var host = hostBuilder.Build();
host.Run();
}
else
{
var host = hostBuilder.UseContentRoot(pathToContentRoot)
.Build();
host.RunAsCustomService();
}
RunAsCustomService:
public static class WebHostServiceExtensions
{
public static void RunAsCustomService(this IWebHost host)
{
var webHostService = new CustomWebHostService(host);
ServiceBase.Run(webHostService);
}
}
internal class CustomWebHostService : WebHostService
{
private ILogger _logger;
public CustomWebHostService(IWebHost host) : base(host)
{
Console.Write("CustomWebHostService start");
_logger = host.Services
.GetRequiredService<ILogger<CustomWebHostService>>();
}
[...]
}
When stopping the service, it correctly receives the Stop-signal from Windows (can be verified by both logs and attaching debugger) -- and it correctly stops very quickly when not under load.
During OnStop(); StopAsync() is called on the host, and then, eventually, it is disposed. I'm guessing the StopAsync does not stop very well when there are a large amount of requests being made to at the same time. Some signal being lost perhaps?
During normal operation all objects are correctly cleaned up or otherwise disposed.
Best regards,

Performing a health check in .NET Core Worker Service

How can I implement health checks in a .NET Core Worker Service?
The service will be run inside Docker and needs to be able to check the health of the service.

Another way of doing this is to implement IHealthCheckPublisher.
The benefits of this approach is the ability to re-use your existing IHealthChecks or integration with 3rd party libraries that rely on IHealthCheck interface (like this one).
Though you still target Microsoft.NET.Sdk.Web as the SDK you don't need to add any asp.net specifics.
Here is an example:
public static IHostBuilder CreateHostBuilder(string[] args)
{
return Host
.CreateDefaultBuilder(args)
.ConfigureServices((hostContext, services) =>
{
services
.AddHealthChecks()
.AddCheck<RedisHealthCheck>("redis_health_check")
.AddCheck<RfaHealthCheck>("rfa_health_check");
services.AddSingleton<IHealthCheckPublisher, HealthCheckPublisher>();
services.Configure<HealthCheckPublisherOptions>(options =>
{
options.Delay = TimeSpan.FromSeconds(5);
options.Period = TimeSpan.FromSeconds(5);
});
});
}
public class HealthCheckPublisher : IHealthCheckPublisher
{
private readonly string _fileName;
private HealthStatus _prevStatus = HealthStatus.Unhealthy;
public HealthCheckPublisher()
{
_fileName = Environment.GetEnvironmentVariable(EnvVariableNames.DOCKER_HEALTHCHECK_FILEPATH) ??
Path.GetTempFileName();
}
public Task PublishAsync(HealthReport report, CancellationToken cancellationToken)
{
// AWS will check if the file exists inside of the container with the command
// test -f $DOCKER_HEALTH_CHECK_FILEPATH
var fileExists = _prevStatus == HealthStatus.Healthy;
if (report.Status == HealthStatus.Healthy)
{
if (!fileExists)
{
using var _ = File.Create(_fileName);
}
}
else if (fileExists)
{
File.Delete(_fileName);
}
_prevStatus = report.Status;
return Task.CompletedTask;
}
}

I don't think is worth it to change SDK to Microsoft.NET.Sdk.Web. You will include additional middlewares just because of one health check? No thanks ...
What you could do is to use a different protocol like TCP.
The general idea is:
Create a separate background service that creates a TCP server (take a look at TcpListener.cs)
When you receive a request you have two options: if the application is healthy accept TCP connection otherwise reject it.
If you use containers your orchestrator should have an option to call it over TCP (in k8s there is a property tcpSocket)
If you need more detailed information you may check: Monitoring Health of ASP.NET Core Background Services With TCP Probes on Kubernetes
Cheers!

I think that you should also consider to retain the Microsoft.NET.Sdk.Worker.
Don't change the whole sdk just because of the health checks.
Then you can create a backgroundservice (just like the main worker), in order to update a file to write for example the current timestamp. An example of the background health check worker would be:
public class HealthCheckWorker : BackgroundService
{
private readonly int _intervalSec;
private readonly string _healthCheckFileName;
public HealthCheckWorker(string healthCheckFileName, int intervalSec)
{
this._intervalSec = intervalSec;
this._healthCheckFileName = healthCheckFileName;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (true)
{
File.WriteAllText(this._healthCheckFileName, DateTime.UtcNow.ToString());
await Task.Delay(this._intervalSec * 1000, stoppingToken);
}
}
}
Then you can add a extension method like this:
public static class HealthCheckWorkerExtensions
{
public static void AddHealthCheck(this IServiceCollection services,
string healthCheckFileName, int intervalSec)
{
services.AddHostedService<HealthCheckWorker>(x => new HealthCheckWorker(healthCheckFileName, intervalSec));
}
}
With this you can add in services the health check support
.ConfigureServices(services =>
{
services.AddHealthCheck("hc.txt", 5);
})

Add HTTPListener and expose the health checks endpoints.
Using HTTPListener does not require adding Microsoft.NET.Sdk.Web SDK.
Program.cs
using Consumer;
IHost host = Host.CreateDefaultBuilder(args)
.ConfigureServices(services =>
{
services.AddHostedService<Worker>();
services.AddHostedService<HttpHealthcheck>();
})
.Build();
await host.RunAsync();
HttpHealthcheck.cs
using System.Net;
using System.Text;
namespace Consumer;
public class HttpHealthcheck : BackgroundService
{
private readonly ILogger<Worker> _logger;
private readonly HttpListener _httpListener;
private readonly IConfiguration _configuration;
public HealthcheckHttpListener(ILogger<Worker> logger, IConfiguration configuration)
{
_logger = logger;
_configuration = configuration;
_httpListener = new HttpListener();
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
_httpListener.Prefixes.Add($"http://*:5001/healthz/live/");
_httpListener.Prefixes.Add($"http://*:5001/healthz/ready/");
_httpListener.Start();
_logger.LogInformation($"Healthcheck listening...");
while (!stoppingToken.IsCancellationRequested)
{
HttpListenerContext ctx = null;
try
{
ctx = await _httpListener.GetContextAsync();
}
catch (HttpListenerException ex)
{
if (ex.ErrorCode == 995) return;
}
if (ctx == null) continue;
var response = ctx.Response;
response.ContentType = "text/plain";
response.Headers.Add(HttpResponseHeader.CacheControl, "no-store, no-cache");
response.StatusCode = (int)HttpStatusCode.OK;
var messageBytes = Encoding.UTF8.GetBytes("Healthy");
response.ContentLength64 = messageBytes.Length;
await response.OutputStream.WriteAsync(messageBytes, 0, messageBytes.Length);
response.OutputStream.Close();
response.Close();
}
}
}

What I've done to accomplish this is add Microsoft.NET.Sdk.Web to my Worker, and then configured a web host to run alongside the worker:
Host.CreateDefaultBuilder(args)
.ConfigureWebHostDefaults(builder =>
{
builder.UseStartup<Startup>();
})
.ConfigureServices((hostContext, services) =>
{
services.AddHostedService<Worker>();
services.AddLogging(builder =>
builder
.AddDebug()
.AddConsole()
);
});
With that done, all that's left to do is map the health check endpoint as you normally would with ASP.NET Core.