We Keep Coding

MediatR library: following the DRY principle

I use library MediatR in my ASP.NET Core application.
I have the following entity Ad:
public class Ad
{
public Guid AdId { get; set; }
public AdType AdType { get; set; }
public double Cost { get; set; }
public string Content { get; set; }
// ...
}
public enum AdType
{
TextAd,
HtmlAd,
BannerAd,
VideoAd
}
I want to introduce the ability to create a new ad. To do so, I've created the following command:
public class CreateAdCommand : IRequest<Guid>
{
public AdType AdType { get; set; }
public double Cost { get; set; }
public string Content { get; set; }
public class Handler : IRequestHandler<CreateAdCommand, Guid>
{
private readonly MyDbContext _context;
public Handler(MyDbContext context)
{
_context = context;
}
public async Task<Guid> Handle(CreateAdCommand request, CancellationToken cancellationToken)
{
var ad = new Ad {AdType = request.AdType, Cost = request.Cost, Content = request.Content};
_context.Ads.Add(ad);
_context.SaveChangesAsync();
return ad.AdId;
}
}
}
This code works great. But here is a huge problem: each ad-type has some additional logic to the ad creation process (e.g., when creating the ad of type TextAd we need to find the keywords in the content of the ad). The simplest solution is:
public async Task<Guid> Handle(CreateAdCommand request, CancellationToken cancellationToken)
{
var ad = new Ad {AdType = request.AdType, Cost = request.Cost, Content = request.Content};
_context.Ads.Add(ad);
_context.SaveChangesAsync();
switch (request.AdType)
{
case AdType.TextAd:
// Some additional logic here...
break;
case AdType.HtmlAd:
// Some additional logic here...
break;
case AdType.BannerAd:
// Some additional logic here...
break;
case AdType.VideoAd:
// Some additional logic here...
break;
}
return ad.AdId;
}
This solution violates the Open Closed Principle (when I create a new ad-type, I need to create a new case inside of CreateAdCommand).
I have another idea. I can create a separate command for each ad-type (e.g., CreateTextAdCommand, CreateHtmlAdCommand, CreateBannerAdCommand, CreateVideoAdCommand). This solution follows the Open Closed Principle (when I create a new ad-type, I need to create a new command for this ad-type - I don't need to change the existing code).
public class CreateTextAdCommand : IRequest<Guid>
{
public double Cost { get; set; }
public string Content { get; set; }
public class Handler : IRequestHandler<CreateTextAdCommand, Guid>
{
private readonly MyDbContext _context;
public Handler(MyDbContext context)
{
_context = context;
}
public async Task<Guid> Handle(CreateTextAdCommand request, CancellationToken cancellationToken)
{
var ad = new Ad {AdType = AdType.TextAd, Cost = request.Cost, Content = request.Content};
_context.Ads.Add(ad);
await _context.SaveChangesAsync();
// Some additional logic here ...
return ad.AdId;
}
}
}
public class CreateHtmlAdCommand : IRequest<Guid>
{
public double Cost { get; set; }
public string Content { get; set; }
public class Handler : IRequestHandler<CreateHtmlAdCommand, Guid>
{
private readonly MyDbContext _context;
public Handler(MyDbContext context)
{
_context = context;
}
public async Task<Guid> Handle(CreateHtmlAdCommand request, CancellationToken cancellationToken)
{
var ad = new Ad {AdType = AdType.HtmlAd, Cost = request.Cost, Content = request.Content};
_context.Ads.Add(ad);
await _context.SaveChangesAsync();
// Some additional logic here ...
return ad.AdId;
}
}
}
// The same for CreateBannerAdCommand and CreateVideoAdCommand.
This solution follows the Open Closed Principle, but violates the DRY principle. How can I solve this problem?

If you stick to your second approach, you can levarage MediatR 'Behaviors' (https://github.com/jbogard/MediatR/wiki/Behaviors). They act like pipelines, where you can offload common behavior into a commonly used handler.
To do this, create a marker interface
interface ICreateAdCommand {}
Now let each concreate command inherit from it
public class CreateTextAdCommand : ICreateAdCommand
{
public readonly string AdType {get;} = AdType.Text
}
public class CreateHtmltAdCommand : ICreateAdCommand
{
public readonly string AdType {get;} = AdType.Html
}
/*...*/
You could combine this or replace this with a common abstract base class, to avoid repetition of common properties. This is up to you.
Now we create the handler for our behavior:
public class CreateAdBehavior<TRequest, TResponse> : IPipelineBehavior<TRequest, TResponse>
where TReq : ICreateAdCommand
{
public CreateAdBehavior()
{
//wire up dependencies.
}
public async Task<TResponse> Handle(TRequest request, CancellationToken cancellationToken, RequestHandlerDelegate<TResponse> next)
{
var ad = new Ad {AdType = request.AdType, Cost = request.Cost, Content = request.Content};
_context.Ads.Add(ad);
await _context.SaveChangesAsync();
//go on with the next step in the pipeline
var response = await next();
return response;
}
}
Now wire up this behavior. In asp.net core this would be in your startup.cs
services.AddTransient(typeof(IPipelineBehavior<,>), typeof(CreateAdBehavior<,>));
At this stage, everytime any of your IRequests implement ICreateAdCommand, it would automatically call the handler above and after this is done it would call the next behavior in line, or if there is none, the actual handler.
Your specific handler for, let's say a HtmlAd would now roughly look like this:
public class CreateHtmlAdCommand : IRequest<Guid>
{
public class Handler : IRequestHandler<CreateHtmlAdCommand, Guid>
{
private readonly MyDbContext _context;
public Handler(MyDbContext context)
{
_context = context;
}
public async Task<Guid> Handle(CreateHtmlAdCommand request, CancellationToken cancellationToken)
{
// Some additional logic here ...
}
}
}
** Update **
If you want to drag data across the pipeline, you can leverage the actual request object.
public abstract class IRequestWithItems
{
public IDictionary<string, object> Items {get;} = new Dictionary<string,object>();
}
Now in your CreateAdBehavior, you can create your ad and store it in the dictionary, to retrieve it in the next handler:
var ad = { ... }
await _context.SaveChangesAsync();
items["newlyCreatedAd"] = ad;
And in the actual Task<Guid> Handle() method, you have now the ad at your disposal, without looping back to your database to retrieve it again.
Details from the author: https://jimmybogard.com/sharing-context-in-mediatr-pipelines/

Worker Service dynamically changing execution interval

I have a worker service with the base class for all workers, which accepts the IOptionsMonitor constructor parameter. This monitor contains options object instance with the execution interval value. The question is how to dynamically change the interval, even when the await Task.Delay(Interval); was called? I mean if the Interval value is set to one day, and after the Task.Delay method being called it changes, for example, to one hour - I still need to wait one day and only on the next call the delay would be updated. How I can cancel the current delay and start a new one if the property Interval value was updated?
Thanks.
Please see the code attached below:
public abstract class WorkerBase<TWorker, TWorkerOptions> : BackgroundService
where TWorker : WorkerBase<TWorker, TWorkerOptions>
where TWorkerOptions : IHaveIntervalProperty
{
protected WorkerBase(IServiceProvider serviceProvider, ILogger<TWorker> logger, IOptionsMonitor<TWorkerOptions> options)
{
_logger = logger;
_serviceProvider = serviceProvider;
_workerName = typeof(TWorker).Name;
Interval = options.CurrentValue.Interval;
options.OnChange(UpdateOptions);
}
public TimeSpan Interval { get; private set; }
public virtual void UpdateOptions(TWorkerOptions options)
=> Interval = options.Interval;
public abstract Task DoWork(IServiceProvider provider);
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
_logger.LogInformation(Logs.InformationWorkerRunning, _workerName, DateTime.UtcNow);
try
{
using var scope = _serviceProvider.CreateScope();
await DoWork(scope.ServiceProvider);
}
catch (Exception e)
{
_logger.LogCritical(e, e.Message);
}
finally
{
await Task.Delay(Interval, stoppingToken);
}
}
}
}

Okay, so based on #Panagiotis Kanavos comment I came up with the following code:
public abstract class RepeatableWorker<TWorker, TOptions> : IHostedService, IDisposable
where TWorker : RepeatableWorker<TWorker, TOptions>
where TOptions : IHaveIntervalProperty
{
#region Fields
private readonly IServiceProvider _serviceProvider;
private protected readonly ILogger<TWorker> _logger;
private readonly string _workerName;
private Timer? _executionTimer;
private TimeSpan _interval;
#endregion
#region Constructors
protected RepeatableWorker(IServiceProvider serviceProvider,
ILogger<TWorker> logger,
IOptionsMonitor<TOptions> options)
{
_serviceProvider = serviceProvider;
_logger = logger;
_workerName = typeof(TWorker).Name;
_interval = options.CurrentValue.Interval;
options.OnChange(UpdateOptions);
}
#endregion
#region Properties
public TimeSpan Interval
{
get => _interval;
private set
{
if (value != _interval)
{
_executionTimer?.Change(TimeSpan.Zero, value);
_interval = value;
}
}
}
#endregion
#region Public methods
public virtual void UpdateOptions(TOptions options)
=> Interval = options.Interval;
public abstract void DoWork(IServiceProvider serviceProvider);
public Task StartAsync(CancellationToken cancellationToken)
{
_logger.LogInformation(Logs.InformationWorkerStarting, _workerName, DateTime.UtcNow);
_executionTimer = new(DoWorkInternal, null, TimeSpan.Zero, Interval);
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
_logger.LogInformation(Logs.InformationWorkerStopping, _workerName, DateTime.UtcNow);
_executionTimer?.Change(Timeout.Infinite, 0);
return Task.CompletedTask;
}
public void Dispose()
{
GC.SuppressFinalize(this);
_executionTimer?.Dispose();
}
#endregion
#region Private methods
private void DoWorkInternal(object? state)
{
try
{
_logger.LogInformation("Worker {0} running at {1}.", _workerName, DateTime.UtcNow);
using var scope = _serviceProvider.CreateScope();
DoWork(scope.ServiceProvider);
}
catch (Exception e)
{
_logger.LogCritical(e, e.Message);
}
}
#endregion
}
And the IHaveIntervalProperty interface:
public interface IHaveIntervalProperty
{
TimeSpan Interval { get; set; }
}
Just in case someone will need such a solution.

Integration between SignalR and WebAPI

I have a WebAPI server with integrated SignalR Hubs.
The problem it is in the integration between both components and efficiently call the clients interested on a given item that was updated through REST with the least overhead possible on the Controller side.
I have read about Background tasks with hosted services in ASP.NET Core, or Publish Subscriber Patterns but they don't seem the right fit for this problem.
From the documentation examples, the background tasks seem to atempt to preserve order which is not required, in fact, it is desired to allow multiple requests to be handled concurrently, as efficiently as possible.
With this in mind, I created this third component called MappingComponent that is being called through a new Task.
It is important to design the Controller in a way that he spends the least amount of work "raising the events" possible. Exceptions should be (i believe) handled within the MappingComponent.
What would be a better approach/design pattern that the following implementation, to avoid using Task.Run?
ApiController
[Route("api/[controller]")]
[ApiController]
public class ItemController : ControllerBase
{
private readonly MappingComponent mappingComponent;
private readonly IDataContext dataContext;
[HttpPost]
public async Task<ActionResult<Item>> PostItem(ItemDTO itemDTO)
{
await dataContext.Items.AddAsync(item);
(...)
_ = Task.Run(async () =>
{
try
{
await mappingComponent.NotifyOnItemAdd(item);
}
catch (Exception e)
{
Console.WriteLine(e);
}
});
return CreatedAtAction("Get", new { id = item.Id }, item);
}
[HttpDelete("{id}", Name = "Delete")]
public async Task<IActionResult> Delete(int id)
{
var item = await dataContext.Items.FindAsync(id);
(...)
_ = Task.Run(async () =>
{
try
{
await mappingComponent.NotifyOnItemDelete(item);
}
catch (Exception e)
{
Console.WriteLine(e);
}
});
return NoContent();
}
[HttpPatch("{id}", Name = "Patch")]
public async Task<IActionResult> Patch(int id,
[FromBody] JsonPatchDocument<Item> itemToPatch)
{
var item = await dataContext.Items.FindAsync(id);
(...)
_ = Task.Run(async () =>
{
try
{
await mappingComponent.NotifyOnItemEdit(item);
}
catch (Exception e)
{
Console.WriteLine(e);
}
});
return StatusCode(StatusCodes.Status202Accepted);
}
}
SignalR Hub
public class BroadcastHub : Hub<IHubClient>
{
private readonly MappingComponent mappingComponent;
public BroadcastHub(MappingComponent mappingComponent)
{
this.mappingComponent = mappingComponent;
}
public override Task OnConnectedAsync()
{
mappingComponent.OnConnected(Context.User.Identity.Name, Context.ConnectionId));
return base.OnConnectedAsync();
}
public override Task OnDisconnectedAsync()
{
mappingComponent.OnDisconnected(Context.User.Identity.Name, Context.ConnectionId));
return base.OnDisconnectedAsync();
}
public void Subscribe(string itemQuery)
{
mappingComponent.SubscribeConnection(Context.User.Identity.Name, Context.ConnectionId, itemQuery));
}
public void Unsubscribe()
{
mappingComponent.UnsubscribeConnection(Context.ConnectionId));
}
}
"MappingComponent" being registered as singleton on startup
public class MappingComponent
{
private readonly IServiceScopeFactory scopeFactory;
private readonly IHubContext<BroadcastHub, IHubClient> _hubContext;
private static readonly ConcurrentDictionary<string, User> Users = new(StringComparer.InvariantCultureIgnoreCase);
private static readonly ConcurrentDictionary<int, List<string>> ItemConnection = new();
private static readonly ConcurrentDictionary<string, List<int>> ConnectionItem = new();
public MappingComponent(IServiceScopeFactory scopeFactory, IHubContext<BroadcastHub, IHubClient> hubContext)
{
//this.dataContext = dataContext;
this._hubContext = hubContext;
this.scopeFactory = scopeFactory;
}
internal void OnConnected(string userName, string connectionId){(...)}
internal void OnDisconnected(string userName, string connectionId){(...)}
internal void SubscribeConnection(string userName, string connectionId, string query){(...)}
internal void UnsubscribeConnection(string connectionId){(...)}
internal async Task NotifyOnItemAdd(Item item)
{
List<string> interestedConnections = new();
(...)
//Example containing locks
lock()
{
//There is a need to acess EF database
using (var scope = scopeFactory.CreateScope())
{
var dataContext = scope.ServiceProvider.GetRequiredService<IDataContext>()
await dataContext.Items.(...)
interestedConnections = ...
}
}
await _hubContext.Clients.Clients(interestedConnections).BroadcastItem(item);
}
internal async Task NotifyOnItemEdit(Item item)
{
List<string> interestedConnections = new();
(...)
await _hubContext.Clients.Clients(interestedConnections).BroadcastItem(item);
}
internal async Task NotifyOnItemDelete(Item item)
{
List<string> interestedConnections = new();
(...)
await _hubContext.Clients.Clients(interestedConnections).BroadcastAllItems();
}
}

Implement HealthChecks for long running process .Net Core

I have a long-running process: IHostedService. It runs all day long calling different external apis (services) to pull in data. I would like to get notified if during this process any of these external services failed/exceptions got thrown, the process is taking longer than 30 min etc. How would I set it up?
After some research, I ended up with this:
https://learn.microsoft.com/en-us/aspnet/core/host-and-deploy/health-checks?view=aspnetcore-5.0#separate-readiness-and-liveness-probes (StartupHostedServiceHealthCheck section)
How do I implement .NET Core Health Checks on a Hosted Service?
This is what I have so far:
// Registered as Singleton
public interface IHostedServiceStatus
{
bool IsHostedServiceRunning { get; set; }
DateTime RunTime { get; }
string Message { get; }
void Setup(string name, DateTime runTime, string message = null);
}
public class HostedServiceStatus : IHostedServiceStatus
{
private string _message;
private string _name;
public string Message => $"HostedService {_name} started on: {RunTime} failed to complete. {_message}";
public bool IsHostedServiceRunning { get; set; }
public DateTime RunTime { get; private set; }
public void Setup(string name, DateTime runTime, string message = null)
{
_name = name;
RunTime = runTime;
IsHostedServiceRunning = true;
if (!string.IsNullOrEmpty(message))
_message = message;
}
}
// HealthCheck
public class HostedServiceHealthCheck : IHealthCheck
{
private readonly IHostedServiceStatus _hostedServiceStatus;
private readonly TimeSpan _duration = TimeSpan.FromMinutes(30);
public HostedServiceHealthCheck(IHostedServiceStatus hostedServiceStatus)
{
_hostedServiceStatus = hostedServiceStatus;
}
public Task<HealthCheckResult> CheckHealthAsync(
HealthCheckContext context,
CancellationToken cancellationToken = default)
{
if (_hostedServiceStatus.IsHostedServiceRunning)
{
if (_hostedServiceStatus.RunTime.Subtract(DateTime.Now).Duration() >= _duration)
return Task.FromResult(
HealthCheckResult.Unhealthy(_hostedServiceStatus.Message));
}
return Task.FromResult(
HealthCheckResult.Healthy("Task is finished."));
}
}
// Long running process
public async void Process(object state)
{
// each service runs about 10 min
foreach (var externalService in externalServices)
{
try
{
_hostedServiceStatus.Setup(externalService.Name, DateTime.Now); // setup healthcheckStatus - injected
...
// calls externalService gets data and saves to db
_dataMinerStatus.IsHostedServiceRunning = false; // udpate Healthcheck - finished successfully
}
catch (Exception ex)
{
// set MInDateTime so that it becamse UnHealthy
_hostedServiceStatus.Setup(externalService.Name, DateTime.MinValue);
// HealthCheck injected
await _healthCheck.CheckHealthAsync(new HealthCheckContext()); // send notification? (webhook setup) - will this work?
}
}

Adding multiple queues in hosted service

I am implementing queues using https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services?view=aspnetcore-3.1&tabs=visual-studio example.
this is how my code looks:
in startup.cs I am adding my hosted service and background queue
services.AddHostedService<QueuedHostedService>();
services.AddSingleton<IBackgroundTaskQueue, BackgroundTaskQueue>();
then I implement scoped service, hosted service and background queue as following:
namespace Services.Services {
public class QueuedHostedService: BackgroundService {
private readonly ILogger _logger;
private readonly IServiceProvider _serviceProvider;
public QueuedHostedService(IServiceProvider serviceProvider, IBackgroundTaskQueue taskQueue, ILoggerFactory loggerFactory) {
_serviceProvider = serviceProvider;
TaskQueue = taskQueue;
_logger = loggerFactory.CreateLogger < QueuedHostedService > ();
}
public IBackgroundTaskQueue TaskQueue {
get;
}
protected override async Task ExecuteAsync(CancellationToken cancellationToken) {
while (!cancellationToken.IsCancellationRequested) {
var workItem = await TaskQueue.DequeueAsync(cancellationToken);
try {
await workItem(cancellationToken);
} catch (Exception ex) {
}
}
}
}
}
public interface IBackgroundTaskQueue {
void QueueBackgroundWorkItem(Func < CancellationToken, Task > workItem);
Task < Func < CancellationToken, Task >> DequeueAsync(CancellationToken cancellationToken);
}
namespace Services.Services {
public class BackgroundTaskQueue: IBackgroundTaskQueue {
private ConcurrentQueue < Func < CancellationToken, Task >> _workItems = new ConcurrentQueue < Func < CancellationToken, Task >> ();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
public void QueueBackgroundWorkItem(Func < CancellationToken, Task > workItem) {
if (workItem == null) {
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
public async Task < Func < CancellationToken, Task >> DequeueAsync(CancellationToken cancellationToken) {
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out
var workItem);
return workItem;
}
}
}
// scoped service
namespace Services.Services {
public class ImportService: BaseService, IImportService {
private readonly IFileProcessingService _scopedProcessingService;
private readonly ConfigurationSettings _configurationSettings;
public IBackgroundTaskQueue Queue {
get;
}
private
const string AZURE_BLOB_CONTAINER = "blobcontainer";
public IServiceProvider Services {
get;
}
public ImportService(IServiceProvider services, IBackgroundTaskQueue queue): base(services) {
Services = services;
_configurationSettings = services.GetService < ConfigurationSettings > ();
_scopedProcessingService = services.GetProcessingService();
Queue = queue;
}
// ---- Main file
public async Task ImportFile(string filePath, long fileSize, int userId, FileFormatType fileFormat, TransactionsDataHeadersMap dataHeadersMap, string delimiter, string dateFormat) {
await _scopedProcessingService.ImportFile(filePath, fileSize, userId, fileFormat, dataHeadersMap, delimiter, dateFormat);
}
public async Task UploadToBlobStorage(IFormFile file, int userId, TransactionalDataFileType type) {
var fileFormat = GetFileFormat(file);
var tempFilePath = await GetTemporaryPath(file);
var fileName = userId.ToString() + "-" + DateTime.Now + "." + fileFormat;
// ....... //
ProcessFile(tempFilePath, fileFormat, file, type, userId);
}
private void ProcessFile(string tempFilePath, FileFormatType fileFormat, IFormFile file, Tyoe type, int userId) {
var delimiter = ",";
Queue.QueueBackgroundWorkItem(async token => {
using(var scope = Services.CreateScope()) {
var scopedProcessingService =
scope.ServiceProvider
.GetRequiredService < IFileProcessingService > ();
// do the processing
switch (type) {
case "csv":
await scopedProcessingService.ImportFile(tempFilePath, file.Length, userId, fileFormat, new Headers(), delimiter ? ? ",", "yyyy-MM-dd");
break;
}
}
});
}
}
}
I am adding elemeents to queue on request in controller. Now I want to add another queue for pocessing other requests. Is it possible to use another queue using same Hosted service? I have trouble finding examples how to do that. Should I just add another scoped servide and another background queue?

The first option is the most straightforward - you just create bunch of classes and interfaces QueuedHostedServiceA, QueuedHostedServiceB, IBackgroundTaskQueueA.. (you can use inheritance to reduce code duplication)
Also you can introduce concept of "handler" and make all this stuff generic:
interface IHandler<T> { Task Handle(T msg, CancelationToken ...)}
interface IBackgroundMessageQueue<T> {...} // same impl but with T instead of Func<CancellationToken,Task>
class IBackgroundMessageQueue<T> {...} // same impl but with T instead of Func<CancellationToken,Task>
class QueuedHostedService<T>
{
public QueuedHostedService(..., IBackgroundMessageQueue<T> queue, IHandler<T> h) {... }
protected override async Task ExecuteAsync(CancellationToken cancellationToken) {
while (!cancellationToken.IsCancellationRequested) {
T message = await queue.DequeueAsync(cancellationToken);
try {
using(var scp = serviceProvider.CreateScope())
{
var handler = ServiceProvider.GetRequiredService<IHandler<T>>;
await handler.Handle(message, cancellationToken);
}
} catch (Exception ex) {
}
}
}
}
And for each message type you create your own handler:
class ProcessFile(string tempFilePath, FileFormatType fileFormat, IFormFile file, Tyoe type, int userId){}
FileProcessor: IHandler<ProcessFile> {implement your logic from ImportService.ProcessFile}
Then you register everything:
services.AddScoped<IHandler<ProcessFile>, FileProcessor>()
services.AddSingleton<IBackgroundTaskQueue<ProcessFile>, BackgroundTaskQueue<ProcessFile>>();
services.AddHostedService<QueuedHostedService<ProcessFile>>();
and in your ImportService you resolve typed queue:
public ImportService(IBackgroundMessageQueue<ProcessFile> queue)
and enqueue message in it when needed.