We Keep Coding

Hangfire problem when running two methods inside the Enqueue

I have an extension to enqueue my view models pointing to an implementation of an interface IBackgroundJob
this are my extensions methods
private static readonly ActivitySource activitySource = new("MC.Hangfire.Extensions");
public static string Enqueue<T>(this T job, IBackgroundJobClient client)
{
return client.Enqueue<IBackgroundJob<T>>(ps => ps.AddTelemetry(null).EnqueueJob(null, job, JobCancellationToken.Null));
}
public static IBackgroundJob<T> AddTelemetry<T>(this IBackgroundJob<T> job, PerformContext context)
{
using var activity = activitySource.StartActivity($"Start Job {typeof(T).FullName} id {context.BackgroundJob.Id}", ActivityKind.Server);
activity?.SetTag("JobId", context.BackgroundJob.Id);
activity?.SetTag("JobJson", Newtonsoft.Json.JsonConvert.SerializeObject(job));
activity?.SetTag("Job", Newtonsoft.Json.JsonConvert.SerializeObject(context.BackgroundJob.Job));
return job;
}
My problem is that the EnqueueJob method is called, but the AddTelemetry method is not called before, how can I Add the telemetry information before calling all of my jobs, but in the context of the jobs, and of course not adding this code in all of my enqueue methods?
I'm looking for the Hangfire filters, but I think that there should be a way to inject the filter with the DI of the ASP.NET core application
I created this issue on github because I think that the problem with instrumentation is a little deeper in the code
https://github.com/HangfireIO/Hangfire/issues/2017

How to avoid using using BuildServiceProvider method at multiple places?

I have a legacy Asp.net Core 3.1 application which uses Kestrel server and all our GET and POST calls works fine. We have bunch of middlewares already on my legacy application and we use each of those middlewares for different purpose depending on what is the endpoint.
This is how our legacy application is setup as shown below. I have tried to keep things simple by keeping only important things.
Below is our BaseMiddleware class which is extended by bunch of other middlewares we have. Approx we have 10+ middlewares extending BaseMiddleware class -
BaseMiddleware.cs
public abstract class BaseMiddleware {
protected static ICatalogService catalogService;
protected static ICustomerService customerService;
private static IDictionary <string, Object> requiredServices;
private readonly RequestDelegate _next;
public abstract bool IsCorrectEndpoint(HttpContext context);
public abstract string GetEndpoint(HttpContext context);
public abstract Task HandleRequest(HttpContext context);
public BaseMiddleware(RequestDelegate next) {
var builder = new StringBuilder("");
var isMissingService = false;
foreach(var service in requiredServices) {
if (service.Value == null) {
isMissingService = true;
builder.Append(service.Key).Append(", ");
}
}
if (isMissingService) {
var errorMessage = builder.Append("cannot start server.").ToString();
throw new Exception(errorMessage);
}
_next = next;
}
public async Task Invoke(HttpContext context) {
if (IsCorrectEndpoint(context)) {
try {
await HandleRequest(context);
} catch (Exception ex) {
// handle exception here
return;
}
return;
}
await _next.Invoke(context);
}
public static void InitializeDependencies(IServiceProvider provider) {
requiredServices = new Dictionary<string, Object>();
var catalogServiceTask = Task.Run(() => provider.GetService<ICatalogService>());
var customerServiceTask = Task.Run(() => provider.GetService<ICustomerService>());
// .... few other services like above approx 10+ again
Task.WhenAll(catalogServiceTask, landingServiceTask, customerServiceTask).Wait();
requiredServices[nameof(catalogService)] = catalogService = catalogServiceTask.Result;
requiredServices[nameof(customerService)] = customerService = customerServiceTask.Result;
// ....
}
}
ICatalogService and ICustomerService are normal interfaces with some methods in them that their implementation class implements.
Below is one of our middlewares example that extend BaseMiddleware. All other middlewares follow same logic as below one -
FirstServiceMiddleware.cs
public class FirstServiceMiddleware : BaseMiddleware
{
public FirstServiceMiddleware(RequestDelegate next) : base(next) { }
public override bool IsCorrectEndpoint(HttpContext context)
{
return context.Request.Path.StartsWithSegments("/first");
}
public override string GetEndpoint(HttpContext context) => "/first";
public override async Task HandleRequest(HttpContext context)
{
context.Response.StatusCode = 200;
context.Response.ContentType = "application/json";
await context.Response.WriteAsync("Hello World!");
}
}
public static class FirstServiceMiddlewareExtension
{
public static IApplicationBuilder UseFirstService(this IApplicationBuilder builder)
{
return builder.UseMiddleware<FirstServiceMiddleware>();
}
}
Below is how my Startup class is configured -
Startup.cs
private static ILoggingService _loggingService;
public Startup(IHostingEnvironment env) {
var builder = new ConfigurationBuilder()
.SetBasePath(env.ContentRootPath)
.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
.AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true)
.AddEnvironmentVariables();
Configuration = builder.Build();
}
public IConfigurationRoot Configuration { get; }
public void ConfigureServices(IServiceCollection services) {
services.AddResponseCompression(options =>
{
options.Providers.Add<GzipCompressionProvider>();
});
services.Configure<GzipCompressionProviderOptions>(options =>
{
options.Level = CompressionLevel.Fastest;
});
DependencyBootstrap.WireUpDependencies(services);
var provider = services.BuildServiceProvider();
if (_loggingService == null) _loggingService = provider.GetService<ILoggingService>();
//.. some other code here
BaseMiddleware.InitializeDependencies(provider);
}
public void Configure(IApplicationBuilder app, IHostApplicationLifetime lifetime) {
// old legacy middlewares
app.UseFirstService();
// .. few other middlewares here
}
And below is my DependencyBootstrap class -
DependencyBootstrap.cs
public static class DependencyBootstrap
{
//.. some constants here
public static void WireUpDependencies(IServiceCollection services)
{
ThreadPool.SetMinThreads(100, 100);
var provider = services.BuildServiceProvider();
var loggingService = provider.GetService<ILoggingService>();
// ... some other code here
try
{
WireUp(services, loggingService);
}
catch (Exception ex)
{
Console.WriteLine(ex);
}
}
private static void WireUp(IServiceCollection services, ILoggingService loggingService)
{
// adding services here
services.AddSingleton<....>();
services.AddSingleton<....>();
//....
var localProvider = services.BuildServiceProvider();
if (IS_DEVELOPMENT)
{
processClient = null;
}
else
{
processClient = localProvider.GetService<IProcessClient>();
}
services.AddSingleton<IData, DataImpl>();
services.AddSingleton<ICatalogService, CatalogServiceImpl>();
services.AddSingleton<ICustomerService, CustomerServiceImpl>();
//.. some other services and singleton here
}
}
Problem Statement
I have recently started working with C# and asp.net core framework. I have done my reading and it looks like -
Our legacy application doesn't use Dependency Injection properly as we have lot of places using BuildServiceProvider method which causes that warning. I am not sure why we have to do it.
Also do we really need InitializeDependencies method in BaseMiddleware class? If not then how we can initialize dependencies properly? It looks like we are trying to initialize all the dependencies during server startup so that they all are ready when the call comes for any middleware. I'd like to keep this logic as it is if possible.
Currently I am confuse what is the best way to use DI in asp.net core and if my application is doing it wrong then how can I do it the right way? Above code works fine in our application from a very long time but looks like we might be using it totally wrong way of DI.

Calling BuildServiceProvider multiple times can cause serious trouble, because each call to BuildServiceProvider results in a new container instance with its own cache. This means that registrations that are expected to have the Singleton lifestyle, suddenly are created more than once. This is a problem called Ambiguous Lifestyle.
Some Singletons are stateless and for them there is no difference in creating one or a thousand. But other components that are registered as Singleton might have state, and the working of the application might (indirectly) depend on that state not being duplicated.
To make matters worse, while your application might work correctly today, this might change any time in the future when one of the third-party or framework components you depend on makes a change to one of their components in such way that it becomes a problem when that component is created multiple times.
In your example, you are resolving both ILoggingService and IProcessClient from a service provider. If the resolved components are stateless objects without stateful dependencies, there is no real harm done. But this might change when they become stateful. Again, this can happen by a change of one of its indirect dependencies, so this is something you might not be aware of. This can cause you or your team to waste many hours; such problem will likely not be easily found.
This means that the answer "simply" is to prevent calling BuildServiceProvider() to create intermediate container instances. But this might be easier said than done. In your case, however, you seem to require a dependency on ILoggerService before all dependencies are wired. A typical way to achieve this is to split the registration phase into two separate steps:
One step where you manually create those few singletons that you need before hand
Add them to your container builder (IServiceCollection)
Add all other registrations
For instance:
private ILoggingService _loggingService;
public Startup(Confiration config)
{
_loggingService = new MySpecialLoggingService(config.LogPath);
}
public void ConfigureServices(IServiceCollection services)
{
services.AddSingleton(_loggingService);
// More stuf here.
...
}
Advantage of this structure is that when a dependency is added to the constructor of this manually built MySpecialLoggingService, your code stops compiling and you're forced to look at this code. When that constructor depends on some other framework abstraction or application abstraction that isn't yet available, you know you're in trouble and need to rethink your design.
Final note, calling BuildServiceProvider multiple times isn't a bad thing per se. It is okay when you explicitly want to have multiple isolated modules in your application that each have their own state and run independently of one another. For instance when running multiple end points for multiple bounded contexts within the same process.
UPDATE
I think I am starting to understand what it is you are trying to achieve in your BaseMiddleware. It is a 'convenient' helper class that holds all the dependencies that its derivatives might need. This is probably an old design and you might be aware of this, but this base class is quite problematic. Base classes with dependencies are hardly ever a good idea, because they tend to become big, ever changing, and obfuscate the fact that their derivatives become too complex. In your case, even, you are using the Service Locator anti-pattern which is never a good idea.
Besides this, there is a lot going on in that BaseMiddleware class that—to me—makes little sense, such as:
It contains complex logic to verify whether all dependencies exist, while there are more effective ways to do so. The most effective way is to apply Constructor Injection because it will guarantee that its necessary dependencies are always available. On top of that, you can validate the IServiceCollection on build. This gives you even greater guarantees over the correctness of your DI configuration than your BaseMiddleware currently provides.
It resolves all its services in background threads, which implies that construction of those components is either heavy on CPU or I/O, which is a problem. Instead composition should be fast, because injection constructors should be simple, which allows you to compose object graph with confidence.
You do exception handling in the base class, while it is better suited to be applied at a higher level; for instance, using an outer-most piece of middleware. For the sake of simplicity, though, my next example keeps exception handling inside the base class. That's because I have no idea what kind of things you are doing in there, that could influence my answer.
As the base class is resolving from the root container, middleware classes are only able to make use of Singleton dependencies. Connecting to the database through Entity Framework, for instance, will be a problem, because DbContext classes should not be captured in Singleton consumers.
So, with the observations and advice above, I would suggest reducing the BaseMiddleware class to the following:
// Your middleware classes should implement IMiddleware; this allows middleware
// classes to be transient and have scoped dependencies.
public abstract class ImprovedBaseMiddleware : IMiddleware
{
public abstract bool IsCorrectEndpoint(HttpContext context);
public abstract string GetEndpoint(HttpContext context);
public abstract Task HandleRequest(HttpContext context);
public async Task InvokeAsync(HttpContext context, RequestDelegate next)
{
if (IsCorrectEndpoint(context)) {
try {
await HandleRequest(context);
}
catch (Exception ex) {
// handle exception here
return;
}
return;
}
await next(context);
}
}
Now based on this new base class, create your middleware implementations similar to this next example:
public class ImprovedFirstServiceMiddleware : ImprovedBaseMiddleware
{
private readonly ICatalogService _catalogService;
// Add all dependencies required by this middleware in the constructor.
public FirstServiceMiddleware(ICatalogService catalogService)
{
_catalogService = catalogService;
}
public override bool IsCorrectEndpoint(HttpContext context) =>
context.Request.Path.StartsWithSegments("/first");
public override string GetEndpoint(HttpContext context) => "/first";
public override async Task HandleRequest(HttpContext context)
{
context.Response.StatusCode = 200;
context.Response.ContentType = "application/json";
await context.Response.WriteAsync("Hello from "
+ _catalogService.SomeValue());
}
}
In your application, you can register your middleware classes as follows:
public void ConfigureServices(IServiceCollection services) {
// When middleware implements IMiddleware, it must be registered. But
// that's okay, because it allows the middleware with its
// dependencies to be 'verified on build'.
services.AddTransient<ImprovedFirstServiceMiddleware>();
// If you have many middleware classes, you can use
// Auto-Registration instead. e.g.:
var middlewareTypes =
from type in typeof(HomeController).Assembly.GetTypes()
where !type.IsAbstract && !type.IsGenericType
where typeof(IMiddleware).IsAssignableFrom(type)
select type;
foreach (var middlewareType in middlewareTypes)
services.AddTransient(middlewareType);
...
}
public void Configure(
IApplicationBuilder app, IHostApplicationLifetime lifetime)
{
// Add your middleware in the correct order as you did previously.
builder.UseMiddleware<ImprovedFirstServiceMiddleware>();
}
TIP: If you start to notice that a middleware classes get big constructors, that's likely because such class does too much and gets too complex. This means it should be refactored into multiple smaller classes. In that case, your class is exhibiting the Constructor Over-Injection code smell. There are many possible refactoring patterns and design patterns available that can get you out of this situation.

How can I pass a SignalR hub context to a Hangfire job on ASP .NET Core 2.1?

How can I pass a SignalR hub context to a Hangfire job on ASP .NET Core 2.1?
It seems that since passing arguments to Hangfire is done via serialization/deserialization, it seems that Hangfire has hard-time reconstructing the SignalR hub context.
I schedule the job (in my controller) using :
BackgroundJob.Schedule(() => _hubContext.Clients.All.SendAsync(
"MyMessage",
"MyMessageContent",
System.Threading.CancellationToken.None),
TimeSpan.FromMinutes(2));
Then after 2 minutes, when the job tries to execute, I have the error :
Newtonsoft.Json.JsonSerializationException: Could not create an
instance of type Microsoft.AspNetCore.SignalR.IClientProxy. Type is an
interface or abstract class and cannot be instantiated.
Any idea?
Update 1
I ended up using a static context defined in Startup.cs, and assigned from Configure()
hbctx = app.ApplicationServices.GetRequiredService<IHubContext<MySignalRHub>>();
So now Hangfire schedules instead a hub helper that uses the static context :
BackgroundJob.Schedule(() => new MyHubHelper().Send(), TimeSpan.FromMinutes(2));
and the hub helper gets the context with Startup.hbctx
Even though this is working, it is a little smelly
Update 2
I tried also using the approach in Access SignalR Hub without Constructor Injection:
My background job scheduling became :
BackgroundJob.Schedule(() => Startup.GetService().SendOutAlert(2), TimeSpan.FromMinutes(2));
However this time, I have an exception when I reach the above line:
An unhandled exception has occurred while executing the request
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'IServiceProvider'.
Update 3
Thanks all. The solution was to create a helper that gets the hubcontext via its constructor via DI, and then using hangfire to schedule the helper method Send as the background job.
public interface IMyHubHelper
{
void SendOutAlert(String userId);
}
public class MyHubHelper : IMyHubHelper
{
private readonly IHubContext<MySignalRHub> _hubContext;
public MyHubHelper(IHubContext<MySignalRHub> hubContext)
{
_hubContext = hubContext;
}
public void SendOutAlert(String userId)
{
_hubContext.Clients.All.SendAsync("ReceiveMessage", userId, "msg");
}
}
Then launching the background job from anywhere with :
BackgroundJob.Schedule<MyHubHelper>( x => x.SendOutAlert(userId), TimeSpan.FromMinutes(2));

The answer from Nkosi suggesting to use Schedule<T> generics pointed me to the final solution I used:
First, my MySignalRHub is just an empty class inheriting from Hub.
public class MySignalRHub
{
}
Then, I created a hub helper which maintains a hubcontext on my MySignalRHub. The hubcontext is injected in the helper class via the ASP.Net Core built-in DI mechanism (as explained here).
The helper class:
public class MyHubHelper : IMyHubHelper
{
private readonly IHubContext<MySignalRHub> _hubContext;
public MyHubHelper(IHubContext<MySignalRHub> hubContext)
{
_hubContext = hubContext;
}
public void SendData(String data)
{
_hubContext.Clients.All.SendAsync("ReceiveMessage", data);
}
}
The helper interface:
public interface IMyHubHelper
{
void SendData(String data);
}
Finally, I can use hangfire to schedule from anywhere in the code the method SendData() of the hub helper as a background job with:
BackgroundJob.Schedule<MyHubHelper>(h => h.SendData(myData), TimeSpan.FromMinutes(2));

Using Schedule<T> generics you should be able to take advantage of the dependency injection capabilities of the framework.
BackgroundJob.Schedule<IHubContext<MySignalRHub>>(hubContext =>
hubContext.Clients.All.SendAsync(
"MyMessage",
"MyMessageContent",
System.Threading.CancellationToken.None),
TimeSpan.FromMinutes(2));

Proper way to override dependencies within a scope

I'm using Simple Injector. I have a background processor which is using DI from the start. It will pickup jobs to run, and run them. However, each job needs to run within its own scope so that I can override some contextual dependencies. For example, the job needs to run within a specific security context (the one from which it was created), so I need to start a new scope and override the ISecurityContext injection so the job will be properly secured.
To accomplish this, I was creating a new container (with the proper ISecurityContext) and starting a scope, then running the job, but I'm not sure if this is an appropriate thing to do.
RunJob
private readonly Func<ISecurityContext, Container> _containerFactory;
internal async Task RunJob(BackgroundJob job) {
var parameters = job.GetParameters();
var securityContext = parameters.SecurityContext;
using (var container = _containerFactory(securityContext))
using (AsyncScopedLifestyle.BeginScope(container)) {
// Run the job within this scope.
}
}
DI Bits
container.RegisterSingleton<Func<ISecurityContext, Container>>(() => securityContext => {
var c = new Container();
RegisterDependencies(c);
c.Options.AllowOverridingRegistrations = true;
c.Register<ISecurityContext>(() => securityContext, Lifestyle.Scoped);
return c;
});
It doesn't feel right to me, but I'm not sure what the correct solution is.

The Simple Injector documentation warns about what you are doing by stating:
Warning: Do not create an infinite number of Container instances (such as one instance per request). Doing so will drain the performance of your application. The library is optimized for using a very limited number of Container instances. Creating and initializing Container instances has a large overhead, but resolving from the Container is extremely fast once initialized.
In general, you should create only one Container instance per application. This not only holds from a performance perspective, but the creation of this sort of 'child containers' in general is littered with quirks and flaws. For instance, how to ensure that registrations are singletons across the application?
So instead, don't abuse the container for your runtime state, but store it elsewhere. You can use a Scope instance as dictionary for scoped state, but it's as easy to create a simple wrapper for ISecurityContext that is registered as Scoped instance and gets initialized directly after the scope is created as seen in the following example.
// Can be part of your Composition Root
internal sealed class SecurityContextWrapper : ISecurityContext
{
// One of the rare cases that Property Injection makes sense.
public ISecurityContext WrappedSecurityContext { get; set; }
// Implement ISecurityContext methods here that delegate to WrappedSecurityContext.
}
// Composition Root. Only have 1 container for the complete application
c = new Container();
RegisterDependencies(c);
c.Register<SecurityContextWrapper>(Lifestyle.Scoped);
c.Register<ISecurityContext, SecurityContextWrapper>(Lifestyle.Scoped);
// Job logic
private readonly Container _container;
internal async Task RunJob(BackgroundJob job) {
var parameters = job.GetParameters();
var securityContext = parameters.SecurityContext;
using (AsyncScopedLifestyle.BeginScope(_container)) {
// Resolve the wapper inside the scope
var wrapper = _container.GetInstance<SecurityContextWrapper>();
// Set it's wrapped value.
wrapper.WrappedSecurityContext = securityContext;
// Run the job within this scope.
}
}
Alternatively, if you use Scope as state, you can inject a Scope instance as constructor argument of SecurityContextWrapper. That removes the need to use Property Injection, but does make your SecurityContextWrapper dependent on Simple Injector:
// Can be part of your Composition Root
internal sealed class SecurityContextWrapper : ISecurityContext
{
ISecurityContext _wrappedSecurityContext;
public SecurityContextWrapper(Scope scope)
{
_wrappedSecurityContext= (ISecurityContext)scope.GetItem(typeof(ISecurityContext));
}
// Implement ISecurityContext methods here that delegate to WrappedSecurityContext.
}
// Composition Root. Only have 1 container for the complete application
c = new Container();
RegisterDependencies(c);
c.Register<ISecurityContext, SecurityContextWrapper>(Lifestyle.Scoped);
// Job logic
private readonly Container _container;
internal async Task RunJob(BackgroundJob job) {
var parameters = job.GetParameters();
var securityContext = parameters.SecurityContext;
using (var scope = AsyncScopedLifestyle.BeginScope(_container)) {
// Set it's wrapped value.
scope.SetItem(typeof(ISecurityContext), securityContext);
// Run the job within this scope.
}
}

AutoFac - Initialize heavy-weight singletons on app_start

Our configuration is, MVC5 C# app, using AutoFac.
We have a number of singletons, which if they're initialized with the first request, causes a bad experience for the user, because their initialization takes around 3-4 seconds in total. We're using AutoFac for Dependency injection, I'm wondering if there's any way of making sure the singletons (or these specific ones) are built on App_Start so we don't lose time when the user sends the first request? If not, what's the best way of solving this problem?

The general solution to this type of problem is to hide such heavy weight objects after a proxy implementation. This way you can trigger the initialization process directly at application startup, while the operation runs in the background without requests to be blocked (unless they require the uninitialized data during their request).
In case your code looks like this:
// The abstraction in question
public interface IMyService
{
ServiceData GetData();
}
// The heavy implementation
public class HeavyInitializationService : IMyServic {
public HeavyInitializationService() {
// Load data here
Thread.Sleep(3000);
}
public ServiceData GetData() => ...
}
A proxy can be created as follows:
public class LazyMyServiceProxy : IMyService {
private readonly Lazy<IMyService> lazyService;
public LazyMyServiceProxy(Lazy<IMyService> lazyService) {
this.lazyService = lazyService;
}
public ServiceData GetData() => this.lazyService.Value.GetData();
}
You can use this proxy as follows:
Lazy<IMyService> lazyService = new Lazy<IMyService>(() =>
new HeavyInitializationService());
container.Register<IMyService>(c => new LazyMyServiceProxy(lazyService))
.SingleInstance();
// Trigger the creation of the heavy data on a background thread:
Task.Factory.StartNew(() => {
// Triggers the creation of HeavyInitializationService on background thread.
var v = lazyService.Value;
});