I am building a microservice which is supposed to send periodical notifications through various means.
To process the notifications and triggers I intend to use a BackgroundService which will look into database, call appropriate service based on notification type and mark into the database the notification as being sent.
How can I access the database from the background service in a safe way, not having concurrency issues?
Is it enough to inject IServiceProvider and create a scope?
public class MyBackgroundService : BackgroundService
{
private readonly IServiceProvider _serviceProvider;
public MyBackgroundService(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
await DoWorkAsync(stoppingToken);
}
private async Task DoWorkAsync(CancellationToken stoppingToken)
{
using (IServiceScope scope = _serviceProvider.CreateScope())
{
IRepository<Notification> notificationRepository =
scope.ServiceProvider.GetRequiredService<IRepository<Notification>>();
IRepository<NotificationLog> notificationLogRepository =
scope.ServiceProvider.GetRequiredService<IRepository<NotificationLog>>();
IUnitOfWork unitOfWork =
scope.ServiceProvider.GetRequiredService<IUnitOfWork>();
while(true)
{
if(stoppingToken.IsCancellationRequested)
{
return;
}
var list = await notificationRepository.GetAll();
.....................................
await notificationLogRepository.Add(...);
await unitOfWork.Commit();
}
}
}
public override async Task StopAsync(CancellationToken stoppingToken)
{
await base.StopAsync(stoppingToken);
}
}
Is it enough to inject IServiceProvider and create a scope?
Yes.
While you can create a single scope for your entire background worker's lifetime, you also can create a scope per "invocation" - in this case, you could create a new scope each time the timer goes off. That way, if there's ever a situation where the next one starts before the current one completes, the two invocations will be guaranteed to have different scopes.
If your "timer" is just doing an await Task.Delay, though, then there's no possibility of overlapping invocations and separate scopes aren't necessary. Some people prefer them anyway, since "invocation" and "scope" conceptually go well together.
Related
in my rested-web-service project, i need to check database in one hour intervals, this method should continue to work all the time, even if there is not user, and this method is single per hosted server.(some thing like cron in linux but i will host is iis)
i have created a hostedService using microsft docs,
with merely modifying the execute intervals (from 5 sec to 1 hours),
**but problem occure after i deploy webservice solution in iis,this hosted servicerun if swagger page is open when i close browser or not calling rest-service, there is no logs in database
another problem is if a open if i open swagger ui and call webservices from mobile-client there is multiple logs instead of only one per hour.**
i bellive there is two problem here, first TimedHostedService scope is not singleton and it is created per request(scoped) and second problem is application is not kept alive where there is not any session in swagger-ui (or there is no user for werb services).
i hava tried creating Async Method for this purpose that method call-it self with task.delay(onHour)
but this didnt work either.
public class TimedHostedService : IHostedService, IDisposable
{
private int executionCount = 0;
private readonly ILogger<TimedHostedService> _logger;
private Timer? _timer = null;
public TimedHostedService(ILogger<TimedHostedService> logger)
{
_logger = logger;
}
public Task StartAsync(CancellationToken stoppingToken)
{
_timer = new Timer(DoWork, null, TimeSpan.Zero,
TimeSpan.FromHours(1));
return Task.CompletedTask;
}
private void DoWork(object? state)
{
var count = Interlocked.Increment(ref executionCount);
_logger.LogInformation(
"Timed Hosted Service is working. Count: {Count}", count);
}
public Task StopAsync(CancellationToken stoppingToken)
{
_timer?.Change(Timeout.Infinite, 0);
return Task.CompletedTask;
}
public void Dispose()
{
_timer?.Dispose();
}
}
I'm having issues finding a solution to my problem and mostly because i don't understand them completely so, here I am asking your support.
I need to fire and forget a method that selects and updates records from database, the problem is I have a 15seconds range between record creation and its appearance in my database (synchronization issue not fixable by me, so i have to accept it) without freezing user's interface and meanwhile letting it create other records.
I tried to simply Task.Run(method) it but every time it's fired the dbContext it's refreshed so nothing is done.
Googling around I found a lot of IHostedService and BackgroundService solutions but i really can't get to the point in them: if I understand what i'm trying to do, I need to call an Hosted Service and passing them a scoped version of my dbContext so every fired method will have it's own dbContext and they could work simultaneously. But can't really get HOW TO DO that.
I managed my code in various layers and repositories, so I'll try to be as clear as possible.
Controller:
public class RMAController : BaseController
{
private readonly ApplicationServiceRecords applicationServiceRecords;
public RMAController(ApplicationServiceRecords
applicationServiceRecords,
IConfiguration configuration,
AuthenticationService authenticationService,
AuthorizationService authorizationService)
: base(
configuration,
authenticationService,
authorizationService)
{
this.applicationService = applicationService;
this.applicationServiceRecords= applicationServiceRecords;
}
[HttpPost]
[Authorize]
public async Task<IActionResult>CreateRecord(
ResponseCreateRecord viewModel)
{
if (!ModelState.IsValid)
return Content("Error X");
//this is the method i want to fire and forget
await ApplicationServiceRecords.CreateRecordAsync(viewModel);
return RedirectToAction("TestPage");
}
}
Inside "CreateRecordAsync" i call other method's from Domain Layer that create, waits and update the record (again, can't get rid of waits nor create it without the need to update it immediately)
I tried using BackgroundService, this way:
public class BackgroundWorkerQueue
{
private ConcurrentQueue <Func< CancellationToken,Task >> _workItems = new ConcurrentQueue < Func < CancellationToken,Task >> ();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
public async Task < Func < CancellationToken,
Task >> DequeueAsync(CancellationToken cancellationToken) {
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out
var workItem);
return workItem;
}
public void QueueBackgroundWorkItem(Func < CancellationToken, Task > workItem) {
if (workItem == null) {
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
}
public class LongRunningService: BackgroundService {
private readonly BackgroundWorkerQueue queue;
private readonly ILogger < LongRunningService > _logger;
private readonly MyContext _dbcontext;
public LongRunningService(BackgroundWorkerQueue queue, ILogger < LongRunningService > logger, IServiceProvider serviceProvider) {
_logger = logger;
_dbcontext = serviceProvider.CreateScope().ServiceProvider.GetRequiredService <MyContext > ();//thought thiw could be the solution, yet nope (probably can't get how to use it)
this.queue = queue;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var workItem = await queue.DequeueAsync(stoppingToken);
await workItem(stoppingToken);
}
}
}
Added them in startup:
services.AddHostedService<LongRunningService>();
services.AddSingleton<BackgroundWorkerQueue>();
And Fired the method using(?) this from controller :
_backgroundWorkerQueue.QueueBackgroundWorkItem(async token =>{
ApplicationServiceRecords.CreateRecordAsync(viewModel); });
But I got "Invalid attempt to call ReadAsync when reader is closed." on first attemp using DB in a simple select client = await repository.GetClienteByIdAsync(client.Id);
And that's all.
I'm sorry for bad english/ bad programming/bad explanation, and thank you in advance to everyone'll help.
I have created a Background Service in my Server Solution
public class PurgeService : IHostedService, IDisposable
{
private readonly IServiceProvider _provider;
private Timer timer;
public PurgeService(IServiceProvider serviceProvider)
{
using (IServiceScope scope = serviceProvider.CreateScope())
{
_provider = scope.ServiceProvider;
}
}
public void Dispose()
{
timer?.Dispose();
}
public Task Purge(IServiceProvider serviceProvider)
{
var dbcontext = serviceProvider.GetRequiredService<ApplicationDBContext>();
var setting = dbcontext.AppSet.First();
double deletetime = setting.PurgeTimer *(1);
DateTime deletedate = DateTime.Now.AddHours(deletetime);
string deleteSQL = $"DELETE FROM Notifications WHERE CreatedDate > {deletedate}"
}
public Task StartAsync(CancellationToken cancellationToken)
{
timer = new Timer(x => Purge(_provider), null, TimeSpan.Zero, TimeSpan.FromSeconds(10));
return Task.CompletedTask;
}
public Task StopAsync(CancellationToken cancellationToken)
{
return Task.CompletedTask;
}
}
And added it to the Startup.cs
services.AddHostedService<PurgeService>();
My goal is to have a background service that checks every 10 seconds if there are notifications older than allowed and deletes them.
But when I run the app an error occurs
"System.ObjectDisposedException: "Cannot access a disposed object."
How can I implement this correctly?
Your constructor seems to be establishing a scope, and immediately disposing the scope. Your provider is tied to that scope so gets nuked immediately in the constructor when Dispose() is called on the scope due to using.
Background services are usually singletons with a lifetime equivalent to the lifetime of the the application. It seems to me you should register your service as a singleton outside of the service class itself. Something like
public class PurgeService : IHostedService
{
// no weird constructor taking in IServiceProvider
}
Also, using an IoC inside of a class is a bit of an antipattern. Inject a DB context factory instead of trying to resolve through a container inside your class.
The problem was that the database table was empty, but the null exception threw a completely different exception.
I've implemented the BackgroundQueue as explained here, and as shown:
public ActionResult SomeAction()
{
backgroundQueue.QueueBackgroundWorkItem(async ct =>
{
//Do some work...
});
return Ok();
}
I registered the BackgroundQueue with Autofac as:
builder.RegisterType<BackgroundQueue>()
.As<IBackgroundQueue>()
.SingleInstance();
So far so good. I call my controller action and the task is added to the queue. And there it stays without being executed.
So how do I get the task to execute?
The BackgroundQueue implementation that you took from the documentation is only one part to the solution: The background queue will just keep track of the jobs that you want to be executed.
What you will also need is right below that in the docs: The QueuedHostedService. This is a background service that gets registered with the DI container and is started when the application starts. From then on, it will monitor your BackgroundQueue and work off jobs as they get queued.
A simplified example implementation of this background service, without logging or error handling, could look like this:
public class QueuedHostedService : BackgroundService
{
private readonly IBackgroundQueue _backgroundQueue;
public QueuedHostedService(IBackgroundQueue backgroundQueue)
{
_backgroundQueue = backgroundQueue;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var workItem = await _backgroundQueue.DequeueAsync(stoppingToken);
await workItem(stoppingToken);
}
}
}
I'm developing a web api with asp.net core 2.2 and ef core 2.2.1. The api, besides handling the restful requests made by an angular app, is in charge of processing some xml files that are used as an interface with other software. Files are local to the application server and detected through a FileWatcher.
I've noticed during my tests that when I reprocess multiple times an xml test file, starting from the second time the file is being reprocessed, I obtain the Exception:
System.InvalidOperationException: The instance of entity type
'QualityLot' cannot be tracked because another instance with the key
value '{QualityLotID: ...}' is already being tracked. When
attaching existing entities, ensure that only one entity instance with
a given key value is attached.
when I call the method DbContext.QualityLot.Update(qualityLot);
The "processing file" service and the service it's using are configured into the Startup.cs file as following:
services.AddHostedService<InterfaceDownloadService>();
services.AddTransient<IQLDwnldService, QLDwnldService>();
the db context is configued like this:
services.AddDbContext<MyDbContext>(cfg =>
{
cfg.UseSqlServer(_config.GetConnectionString("LIMSConnectionString"));
});
and the class looks like:
public class InterfaceDownloadService : BackgroundServiceBase
{
[...]
public InterfaceDownloadService(IHostingEnvironment env,
ILogger<InterfaceDownloadService> logger,
IServiceProvider serviceProvider)
{
_ServiceProvider = serviceProvider;
}
[...]
private void processFiles()
{
[...]
_ServiceProvider.GetService<IQLDwnldService>().QLDownloadAsync(ev);
}
}
public abstract class BackgroundServiceBase : IHostedService, IDisposable
{
private Task _executingTask;
private readonly CancellationTokenSource _stoppingCts =
new CancellationTokenSource();
protected abstract Task ExecuteAsync(CancellationToken stoppingToken);
public virtual Task StartAsync(CancellationToken cancellationToken)
{
// Store the task we're executing
_executingTask = ExecuteAsync(_stoppingCts.Token);
// If the task is completed then return it,
// this will bubble cancellation and failure to the caller
if (_executingTask.IsCompleted)
{
return _executingTask;
}
// Otherwise it's running
return Task.CompletedTask;
}
public virtual async Task StopAsync(CancellationToken cancellationToken)
{
// Stop called without start
if (_executingTask == null)
{
return;
}
try
{
// Signal cancellation to the executing method
_stoppingCts.Cancel();
}
finally
{
// Wait until the task completes or the stop token triggers
await Task.WhenAny(_executingTask, Task.Delay(Timeout.Infinite,
cancellationToken));
}
}
public virtual void Dispose()
{
_stoppingCts.Cancel();
}
}
Here the critical point, where I have the exception:
public async Task QLDownloadAsync(FileReceivedEvent fileReceivedEvent)
{
Logger.LogInformation($"QLDwnld file {fileReceivedEvent.Event.FullPath} received for Processing");
try
{
QualityLotDownload qualityRoutingDwnld = deserializeObject<QualityLotDownload>(fileReceivedEvent.XsltPath, fileReceivedEvent.Event.FullPath);
Logger.LogDebug($"QLDwnld file {fileReceivedEvent.Event.FullPath} deserialized correctly. Need to determinate whether Insert or Update QualityLot {qualityRoutingDwnld.QualityLots.QualityLot.QualityLotID}");
for (int remainingRetries = fileReceivedEvent.MaxRetries; remainingRetries > 0; remainingRetries--)
{
using (var transaction = await DbContext.Database.BeginTransactionAsync())
{
try
{
var qualityLotDeserialized = qualityRoutingDwnld.QualityLots.QualityLot;
// insert the object into the database
var qualityLot = await DbContext.QualityLot.Where(x => x.QualityLotID == qualityLotDeserialized.QualityLotID).FirstOrDefaultAsync();
if (qualityLot == null) // INSERT QL
{
await InsertQualityLot(qualityLotDeserialized);
}
else // UPDATE QL
{
await UpdateQualityLot(qualityLot, qualityLotDeserialized);
}
[...]
transaction.Commit();
}
catch (Exception ex)
{
Logger.LogError(ex, $"Retry {fileReceivedEvent.MaxRetries - remainingRetries +1}: Exception processing QLDwnld file {fileReceivedEvent.Event.FullPath}.");
transaction.Rollback();
if (remainingRetries == 1)
{
return;
}
}
The method UpdateQualityLot(qualityLot, qualityLotDeserialized); is invoked because the entity already exists in the db
private async Task UpdateQualityLot(QualityLot qualityLot, QualityLotDownloadQualityLotsQualityLot qualityLotDeserialized)
{
[fields update]
DbContext.QualityLot.Update(qualityLot);
await DbContext.SaveChangesAsync();
}
The call to DbContext.QualityLot.Update(qualityLot); fails.
From what I can see the instance of QLDwnldService is new for every file being processed, in other words the following method returns every time a new object (as configured into Startup.cs)
_ServiceProvider.GetService<IQLDwnldService>().QLDownloadAsync(ev);
, while the DbContext is reused and that's probably the reason why the entity results already tracked.
I also tride to setup the non-tracking option in the DbContext OnConfiguring()
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
base.OnConfiguring(optionsBuilder);
optionsBuilder
.UseQueryTrackingBehavior(QueryTrackingBehavior.NoTracking);
}
So my question is. What's wrong here? Maybe an architecture problematic or maybe a misleading configuration of e core? Thanks in advance for any support.
To be honest I could not figure out where your DBContext is actually injected from your code.
But from the error message I'd say your context is reused in a place where it should not be. So it's injected once and then used over and over and over.
You have registered your service as "Scoped" (because that's the default).
You should register it as "Transient" to ensure you will get a new instance on every call to your service provider:
services.AddDbContext<MyDbContext>(cfg =>
{
cfg.UseSqlServer(_config.GetConnectionString("LIMSConnectionString"));
},
ServiceLifetime.Transient);
Brad mentioned that this will have consequences for the rest of your application and he's right.
The better option might be to leave your DbContext scoped and inject the IServiceScopeFactory into your hosted service. Then create a new scope where you need it:
using(var scope = injectedServiceScopeFactory.CreateScope())
{
var dbContext = scope.ServiceProvider.GetService<DbContext>();
// do your processing with context
} // this will end the scope, the scoped dbcontext will be disposed here
Please note that this still does not mean that you should access the DbContext in parallel. I don't know why your calls are all async. If you are actually doing parallel work, make sure you create one DbContext per thread.