Fire and Forget database-related method in Hosted Service - c#

I'm having issues finding a solution to my problem and mostly because i don't understand them completely so, here I am asking your support.
I need to fire and forget a method that selects and updates records from database, the problem is I have a 15seconds range between record creation and its appearance in my database (synchronization issue not fixable by me, so i have to accept it) without freezing user's interface and meanwhile letting it create other records.
I tried to simply Task.Run(method) it but every time it's fired the dbContext it's refreshed so nothing is done.
Googling around I found a lot of IHostedService and BackgroundService solutions but i really can't get to the point in them: if I understand what i'm trying to do, I need to call an Hosted Service and passing them a scoped version of my dbContext so every fired method will have it's own dbContext and they could work simultaneously. But can't really get HOW TO DO that.
I managed my code in various layers and repositories, so I'll try to be as clear as possible.
Controller:
public class RMAController : BaseController
{
private readonly ApplicationServiceRecords applicationServiceRecords;
public RMAController(ApplicationServiceRecords
applicationServiceRecords,
IConfiguration configuration,
AuthenticationService authenticationService,
AuthorizationService authorizationService)
: base(
configuration,
authenticationService,
authorizationService)
{
this.applicationService = applicationService;
this.applicationServiceRecords= applicationServiceRecords;
}
[HttpPost]
[Authorize]
public async Task<IActionResult>CreateRecord(
ResponseCreateRecord viewModel)
{
if (!ModelState.IsValid)
return Content("Error X");
//this is the method i want to fire and forget
await ApplicationServiceRecords.CreateRecordAsync(viewModel);
return RedirectToAction("TestPage");
}
}
Inside "CreateRecordAsync" i call other method's from Domain Layer that create, waits and update the record (again, can't get rid of waits nor create it without the need to update it immediately)
I tried using BackgroundService, this way:
public class BackgroundWorkerQueue
{
private ConcurrentQueue <Func< CancellationToken,Task >> _workItems = new ConcurrentQueue < Func < CancellationToken,Task >> ();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
public async Task < Func < CancellationToken,
Task >> DequeueAsync(CancellationToken cancellationToken) {
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out
var workItem);
return workItem;
}
public void QueueBackgroundWorkItem(Func < CancellationToken, Task > workItem) {
if (workItem == null) {
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
}
public class LongRunningService: BackgroundService {
private readonly BackgroundWorkerQueue queue;
private readonly ILogger < LongRunningService > _logger;
private readonly MyContext _dbcontext;
public LongRunningService(BackgroundWorkerQueue queue, ILogger < LongRunningService > logger, IServiceProvider serviceProvider) {
_logger = logger;
_dbcontext = serviceProvider.CreateScope().ServiceProvider.GetRequiredService <MyContext > ();//thought thiw could be the solution, yet nope (probably can't get how to use it)
this.queue = queue;
}
protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
while (!stoppingToken.IsCancellationRequested)
{
var workItem = await queue.DequeueAsync(stoppingToken);
await workItem(stoppingToken);
}
}
}
Added them in startup:
services.AddHostedService<LongRunningService>();
services.AddSingleton<BackgroundWorkerQueue>();
And Fired the method using(?) this from controller :
_backgroundWorkerQueue.QueueBackgroundWorkItem(async token =>{
ApplicationServiceRecords.CreateRecordAsync(viewModel); });
But I got "Invalid attempt to call ReadAsync when reader is closed." on first attemp using DB in a simple select client = await repository.GetClienteByIdAsync(client.Id);
And that's all.
I'm sorry for bad english/ bad programming/bad explanation, and thank you in advance to everyone'll help.

Related

Async load data in BaseController, how to make controller constructor async

I run .net core 6 version, and I have no idea how to make constructor of my controller (BaseController) async, because i am calling async service to load items in Menu. Menu is on every page, so any other of my controllers (AccountController, OrderController) inherits from my BaseController.
It must be loaded in constructor, just on creating the controller, I cant hang on it on another action. Right now, when i just make it sync, on first page call there are no preloaded data, because it's not waiting on completition.
public class BaseController : Controller
{
private readonly ICategoryService _categoryService;
protected readonly IMapper _mapper;
protected LayoutViewModel _layoutViewModel = new LayoutViewModel();
public BaseController(ICategoryService categoryService, IMapper mapper)
{
_categoryService = categoryService;
_mapper = mapper;
LoadRankedCategories();
}
public void LoadCategories()
{
var categories = _categoryService.GetCategoriesAsync();
_layoutViewModel.Categories = _mapper.Map<IEnumerable<MenuCategoryViewModel>>(categories);
}
}
Ok, on controller construction start the asynchronous task and save said task in a field. Then make sure you await said task whenever you need the results.
public class BaseController : Controller
{
private Task _categoriesTask;
public BaseController()
{
_categoriesTask = LoadCategoriesAsync();
}
private async Task LoadCategoriesAsync()
{
var rawCategories = await _categoryService.GetCategoriesAsync();
return _mapper.Map<IEnumerable<MenuCategoryViewModel>>(rawCategories);
}
[HttpGet]
public async Task<IActionResult> SomeGet()
{
// Await the class-level task.
var categories = await _categoriesTask;
...
}
}
NOTE: As I see the sample here, however, you should do this in the static constructor or use some caching mechanism like memcached (if microservices or distributed) or Redis, assuming the categories never change. So take the solution as a purely academic response on how to overcome the problem. In reality, this doesn't feel right for the stated reason.
Finally, I wouldn't recommend this at all unless there is absolutely no other way: You can spawn a new thread and block it with Result. This might carry undesired issues, so use it at your own risk.
public BaseController()
{
// This will get you the actual categories.
_categories = Task.Run(() => LoadCategoriesAsync()).Result();
}

Memory leak in Xamarin Forms app when using DI in a Task

I am creating a Xamarin Forms application, and I am using the Xamarin Profiler to show that I have a memory leak. I have tracked the memory leak down to where it is happening, but I can't understand WHY it is happening.
I have a class (we will call it MyClass for now). And that class is using a Timer to call a service once every second. That service makes a REST call to retrieve a bunch of information, and then serializes the results back into an object....
MyClass:
public class MyClass : ContentPage
{
private readonly IMyService myService;
public MyClass() : base()
{
}
protected override async void OnAppearing()
{
StartTimer();
}
private void StartTimer()
{
Task.Run(async() =>
{
while(true)
{
myService = ((App)App.Current)
.serviceProvider
.GetRequiredService<IMyService>();
//--- everytime I call myService.GetSystemStatus(), my allocated memory continues to rise
MyResponse response = await myService.GetSystemStatus();
Device.BeginInvokeOnMainThread(() =>
{
// update the UI here...
});
await Task.Delay(1000);
}
});
}
}
MyService (Singleton):
public class MyService : IMyService
{
private readonly IMyHttpClientFactory httpClientFactory;
public MyService(IMyHttpClientFactory httpClientFactory)
{
this.httpClientFactory = httpClientFactory;
}
public async Task<MyResponse> GetSystemStatus()
{
return await httpClientFactory.Create().GetAsync<MyResponse>(
"http://example.com/api/status"
);
}
}
MyHttpClientFactory (Singleton):
public class MyHttpClientFactory : IMyHttpClientFactory
{
private readonly IServiceProvider _serviceProvider;
public MyHttpClientFactory(IServiceProvider serviceProvider)
{
_serviceProvider = serviceProvider;
}
public MyHttpClient Create()
{
return _serviceProvider.GetRequiredService<MyHttpClient>();
}
}
MyHttpClient:
public class MyHttpClient : IDisposable
{
private HttpClient _httpClient;
public MyHttpClient ()
{
_httpClient = new HttpClient();
_httpClient.Timeout = TimeSpan.FromSeconds(10);
}
public async Task<T> GetAsync<T>(string url) where T : new()
{
string s = await GetStringAsync(url);
return JsonConvert.DeserializeObject<T>(s);
}
public async Task<string> GetStringAsync(string url)
{
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
My services are defined as follows:
public partial class App : Application
public ServiceProvider serviceProvider;
public App()
{
IServiceCollection services = new ServiceCollection();
ConfigureServices(services);
serviceProvider = services.BuildServiceProvider();
InitializeComponent();
}
private void ConfigureServices(IServiceCollection services)
{
services.AddHttpClient<MyHttpClient>("MyHttpClient", x =>
{
x.Timeout = TimeSpan.FromSeconds(5);
});
services.AddSingleton<IMyHttpClientFactory, MyHttpClientFactory>();
services.AddSingleton<IMyService, MyService>();
}
}
Best I can tell, the memory is going up because I am referencing the DI MyService inside a separate thread. But I am not sure if this is the reason or if there is something else that would be causing the leak?
Any advice would be greatly appreciated!!!
Thanks!
From what I understand from your code and your comments, it looks like you're looping by calling StartTimer() inside the Device.StartTimer() method.
According to the documentation, Device.StartTimer() is recurring and will occur every X seconds, depending of your interval parameter.
By removing the call to StartTimer() (the one between t.Dispose() and return false of MyClass.StartTimer, your code should work as expected and you will not create a new timer every x seconds
What could be the cause of the leak:
Your MyHttpClient class implements the IDisposable interface, yet the code to use an instance of this class is not leveraging the disposable nature of the object.
Even though the internal HttpClient instance is wrapped in a using statement, the MyHttpClient instance will not be disposed of as you would expect.
// from MyHttpClient class
public async Task<MyResponse> GetSystemStatus()
{
// no using statement here
return await httpClientFactory.Create().GetAsync<MyResponse>(
"http://example.com/api/status"
);
}
// should be:
public async Task<MyResponse> GetSystemStatus()
{
using (var client = await httpClientFactory.Create())
{
return await client.GetAsync<MyResponse>("http://example.com/api/status");
}
}
Another thing to try is to change the location of the resolution of the MyService instance to inside the Task since this is where it is used. This will allow the task to own the resource, and allow it to be collected when the task is complete.
private void StartTimer()
{
Device.StartTimer(TimeSpan.FromSeconds(1), () =>
{
Task t = Task.Run(async() =>
{
// resolve the service here
myService = ((App)App.Current)
.serviceProvider
.GetRequiredService<IMyService>();
MyResponse response = await myService.GetSystemStatus();
Device.BeginInvokeOnMainThread(() =>
{
// update the UI here...
});
});
t.Wait();
t.Dispose();
StartTimer();
return false;
});
}
A couple of additional observations of your code:
In your HttpClientFactory's Create() method, you are resolving an instance of your client from the DI container.
Your MyHttpClient class has a default constructor which means the resolution is not needed since there are no additional dependencies requiring DI support.
Your code could simply return a new MyHttpClient() instance from the Create() method without the need for DI.
Your MyHttpClient also implements the IMyHttpClient interface, but your factory returns the concrete type. This means you need to either remove the interface as unnecessary or change the return type to be the interface type since the interface is redundant unless it is used.
Thank you all for your answers....
I finally figured out the source of the memory leak.
The problem was that I was referencing "MyService" like this:
myService = ((App)App.Current)
.serviceProvider
.GetRequiredService<IMyService>();
The problem was that the serviceProvider object was a public property on my App. So each time I referenced the provider inside my loop, it was creating the leak.
To get around this, I added an abstract method to each of my pages that implemented MyClass to return the service correctly using DI. This has corrected my memory leak issue....
Thanks all for the help!
I don't think that your timer logic is the cause of the leak.
But in case it is useful to you, here is a clean way to do work periodically, yet if work takes a long time, avoid events "piling up".
Given await/async, no Timer is needed.
(There is an alternative solution that starts/stops a single System.Timers.Timer, but I won't go into that here.)
Replace StartTimer() declaration with the following:
/// <summary> Runs until keepRunning() returns false.
/// Delays by "msecDelay" AFTER finishing the previous loop's non-UI work. </summary>
private void StartTaskLoopWhileKeepRunning(Func<bool> keepRunning, int msecDelay = 250)
{
Task.Run(async () =>
{
while (keepRunning())
{
// Do non-UI work here.
// ... possibly slow work ...
Device.BeginInvokeOnMainThread(() =>
{
// NOTE: This work will run in parallel with the next delay.
// ... Do UI work here. ...
});
// Non-UI thread sleeps for msec.
await Task.Delay(msecDelay);
}
});
}

How to save workitem of type Func<CancellationToken, Task> in DB in BackgroundService to call it again

I am trying to implement a separate service for running a heavy time taking method run in background. I am kind of trying to do something like Hangfire but I can't use Hangfire because of some reasons.
So I have implement a package which takes in the delegate of the method and queues it and run it in background and also retries for some count if any exceptions. So I want to add one more functionality to it is if while retrying application is stop. It should continue it's from that count.
For above this to work I need to save this Func<CancellationToken, Task> delegate to DB. So that whenever again when we start I can get it from DB and run with all the details like exact arguments value.
Below is the code for better understanding and it is to understand the flow it does not contain retry logic.
This is BackgroundTaskQueue.cs
public class BackgroundTaskQueue : IBackgroundTaskQueue
{
private ConcurrentQueue<Func<CancellationToken, Task>> _workItems =
new ConcurrentQueue<Func<CancellationToken, Task>>();
private SemaphoreSlim _signal = new SemaphoreSlim(0);
private readonly ILogger<BackgroundTaskQueue> _logger;
public async Task QueueBackgroundWorkItemAsync(
Func<CancellationToken, Task> workItem, string jobId)
{
try{
if (workItem == null)
{
throw new ArgumentNullException(nameof(workItem));
}
_workItems.Enqueue(workItem);
_signal.Release();
}
catch (Exception ex) {
_logger.LogError(ex.Message);
}
}
public async Task<Func<CancellationToken, Task>> DequeueAsync(
CancellationToken cancellationToken)
{
await _signal.WaitAsync(cancellationToken);
_workItems.TryDequeue(out var workItem);
return workItem;
}
}
QueuedHostedService.cs
public class QueuedHostedService : BackgroundService
{
private readonly ILogger _logger;
private readonly IApplicationDbContext _dbContext;
private string jobId;
private Func<CancellationToken, Task>? workItem;
private CancellationToken cancellationToken;
public QueuedHostedService(IApplicationDbContext dbContext,
IBackgroundTaskQueue taskQueue,
ILoggerFactory loggerFactory)
{
TaskQueue = taskQueue;
_logger = loggerFactory.CreateLogger<QueuedHostedService>();
_dbContext = dbContext;
}
public IBackgroundTaskQueue TaskQueue { get; }
protected async override Task ExecuteAsync(
CancellationToken cancellationToken)
{
_logger.LogInformation("Queued Hosted Service is starting.");
this.cancellationToken = cancellationToken;
while (!this.cancellationToken.IsCancellationRequested)
{
workItem = await TaskQueue.DequeueAsync(cancellationToken);
await workItem(this.cancellationToken);
}
_logger.LogInformation("Queued Hosted Service is stopping.");
}
}
To make any method run in background we simply have to inject IBackgroundTaskQueue and call the method QueueBackgroundWorkItemAsync like below.
_queue.QueueBackgroundWorkItemAsync(async token => SomeMethod(arg1, arg2));
I want to make it same generic as Hangfire but I am unable to figure out what all fields I need to save to db from the Func<CancellationToken, Task>() and how do I get those fields. And another thing is how do I construct back that same Func<CancellationToken, Task>().
For above this to work I need to save this Func<CancellationToken, Task> delegate to DB.
This is how Hangfire works. It is one of the trickiest parts, and (IMO) it is also the worst part of the design. In particular, handling application upgrades can be tricky. It wasn't that long ago that Hangfire just flat-out didn't support rolling upgrades at all - now they do, but because of the delegate-serialization logic, they can only handle some types of application updates.
I recommend instead of trying to save a delegate, you have your code save a logical representation of the work to do. E.g.:
public interface IBackgroundWork { Task ExecuteAsync(CancellationToken token); }
public sealed class SomeBackgroundWork : IBackgroundWork
{
private readonly string _arg1;
private readonly string _arg2;
public SomeBackgroundWork(string arg1, string arg2)
{
_arg1 = arg1;
_arg2 = arg2;
}
public Task ExecuteAsync(CancellationToken token) =>
SomeMethodAsync(_arg1, _arg2);
}
Then serialize it appropriately (i.e., with a type field).
The disadvantage of this approach is that you need to define actual type representations for all of your background work item types. The advantage is that it fully captures all the logic without being obfuscated by delegate serialization, and it also is fully upgrade-safe, including rolling upgrades.

OnActionExecutionAsync in IAsyncActionFilter only hit sometimes

I have made an Api Key authorization attribute, and it works... sometimes, which is quite strange.
The attribute code:
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method)]
public class ApiKeyAuthAttribute: Attribute, IAsyncActionFilter
{
private const string ApiKeyHeadername = "ApiKey";
public async Task OnActionExecutionAsync(ActionExecutingContext context, ActionExecutionDelegate next)
{
DataAcces dataAcces = context.HttpContext.RequestServices.GetService(typeof(DataAcces)) as DataAcces;
if (!context.HttpContext.Request.Headers.TryGetValue(ApiKeyHeadername, out var apiKey) ||
!dataAcces.TryAuthenticate(apiKey, out NsInvoiceRecognitionCredentials credentials))
{
context.Result = new UnauthorizedResult();
return;
}
((JobController)context.Controller).Credentials = credentials;
await next();
}
}
The controller code:
[ApiController]
[Route("jobs")]
[ApiKeyAuth]
public class JobController : ControllerBase
{
private readonly DataAcces _dataAcces;
private readonly ILogger<JobController> _logger;
public NsInvoiceRecognitionCredentials Credentials;
public JobController(ILogger<JobController> logger, DataAcces dataAcces)
{
_logger = logger;
_dataAcces = dataAcces;
}
[HttpPost, Route("add")]
public IActionResult Add(IFormFile document)
{
//Do stuff
return Ok();
}
}
DataAcces code:
public bool TryAuthenticate(string apiKeyString, out NsInvoiceRecognitionCredentials credentials)
{
if (Guid.TryParse(apiKeyString, out Guid apiKey))
{
credentials = _dbContext.Set<NsInvoiceRecognitionCredentials>().FirstOrDefault(x => x.ApiKey == apiKey);
return credentials != null;
}
credentials = default;
return false;
}
Every request I send ends up in the controller constructor, which is expected. After that it should go to OnActionExecutionAsync in the ApiKeyAut hAttribute, but this does not happen 100% of the time, only about 50% of the time. The other 50% of the time it just does nothing and eventually time out.
I don't think the problem is in my filter or the dataAcces code, just because it doesnt even hit my breakpoint on the first line of OnActionExecutionAsync in my filter, so those parts are not even executed.
If I place a breakpoint in the first line of my controller constructor I can step over the next few lines, and about 50% of the time it just stops after those lines are executed, no exception, nothing.
EDIT:
I figured out what is going wrong, I didnt find a proper solution yet. I'll answer my question if I do.
There was nothing going wrong in the filter, the problems all occured due to the fact that I am working with files in the request body. These files apparently made the request too heavy (strange because its only 100kb). When I send requests without any files, it all works fine.
I think the problem might be the way I am sending and receiving files (IFormFiles)
If anyone knows why, or how to fix this I would be most grateful
Try to change it OnActionExecutingAsync and you have to use it like this
[ServiceFilter(typeof(ApiKeyAuthAttribute))]
public class JobController : ControllerBase
And before using it should be registered in startup
services.AddScoped<ApiKeyAuthAttribute>();
bur it usually makes sense to use filter only for the action where it is needed
[ServiceFilter(typeof(ApiKeyAuthAttribute))]
public async Task<IActionResult> GetUData()
and maybe you will have to remove all extra attributes. And I don't see any async tasks in your controller. Maybe you can try just ActionFilter
public class ApiKeyAuthAttribute: IActionFilter
{
public void OnActionExecuting(ActionExecutingContext context)
{
or you can try to merge both
public class ApiKeyAuthAttribute: IActionFilter, IAsyncActionFilter
{
public void OnActionExecuting(ActionExecutingContext context)
{
......
}
public async Task OnActionExecutingAsync(ActionExecutingContext context,
ActionExecutionDelegate next)
{
OnActionExecuting(context);
await next();
}
}

Delaying a task in C# - db context disposed

I have a situation where I need certain code to execute at a certain time (in my ASP .NET Core project).
I know that delaying a task is not a great solution, but this is what I have and I'd like to know how to make it work:
async Task MyMethod()
{
// do something
// Create a new thread that waits for the appropriate time
TimeSpan time = dbAppointment.ScheduledOn - TimeSpan.FromMinutes(5.0) - DateTime.UtcNow;
_ = Task.Delay(time).ContinueWith(async x => await
notificationManager.CreateReminder());
// continue doing something
}
When I try to run the code, it enters the method that is supposed to be executed, at the right time:
public async Task CreateReminder() {}
but fails when it tries to use my dbContext which I injected using DI into the NotificationManager constructor, stating that it was disposed.
This is the "flow" of the dependencies:
public class MyClass
{
private readonly MyContext dbContext;
private readonly INotificationManager notificationManager;
public MyClass(MyContext context, INotificationManager nm)
{
dbContext = context;
notificationManager = nm;
}
public async Task MyMethod() // the method shown in the snippet above
{
// method does something using the dbContext
_ = Task.Delay(time).ContinueWith(async x => await
notificationManager.CreateReminder());
}
}
public class NotificationManager: INotificationManager
{
private readonly MyContext dbContext;
public NotificationManager(MyContext context) { dbContext = context;}
public async Task CreateReminder() { // this method uses the dbContext}
}
DI setup in startup.cs:
services.AddDbContext<MyContext>();
services.AddScoped<INotificationManager, NotificationManager>();
Options
Use a job scheduler (Like Hangfire, Quartz.Net, Jobbr, ...)
Use a background service if your .net core version is >= 2
In both cases you'll need to inject the DatabaseContext in the job class otherwise you'll receive an ObjectDisposedException.
When you need to scale-out to multiple machines you'll need a job server with a state store like SQL Server, MSMQ, RabbitMQ, Redis,...
Sample with Hangfire
public class MyDelayJob
{
private readonly MyContext dbContext;
private readonly INotificationManager notificationManager;
public MyDelayJob(MyContext context, INotificationManager nm)
{
dbContext= context;
notificationManager = nm;
}
public async Task Run(/*parameters*/)
{
await notificationManager.CreateReminder()
}
}
/*Shedule code in MyMethod
IBackgroundJobClient can be injected
you need to register MyDelayJob with your IOC container.
*/
backgroundJobClient.Schedule<MyDelayJob>(j => j.Run(), TimeSpan.FromSeconds(60))
See the docs for IBackgroundJobClient

Categories