We Keep Coding

Run heavy task that uses dbContext on the API background without await

my C# .Net 6 API project has a reporting requirement: Convert any query or class to CSV file.
The way that I've got the tasks is as follows:
in a [POST] request endpoint named export create the CSV file from a query and upload it to blob storage, without making the user wait for the task to finish.
once the controller gets the requests start the task and return 200 immediately.
later on frontEnd will make a get request and ask for the document, if the document is done, return the document URL.
This is the endpoint code that I have so far:
[HttpPost("export")]
public virtual async Task<IActionResult> Export([FromQuery] UrlRequestBase? urlRequestBase,
[FromBody] BodyRequestBase? bodyRequestBase)
{
object? response;
int status = 200;
try
{
await urlRequestBase.Parse(this);
await bodyRequestBase.Parse();
//Run the export creation in another thread
Task.Run(() => _repositoryBase.CreateExport(urlRequestBase, bodyRequestBase));
return StatusCode(status);
}
catch (ExceptionBase ex)
{
return StatusCode(ex.CodeResult, ex.CreateResponseFromException());
}
}
The problem is that when I try to make a query inside the repository the dbContext is
disposed of because of the lifetime of the DI container, so I get the following error:
Cannot access a disposed context instance. A common cause of this error is disposing a
context instance that was resolved from dependency injection and then later trying to use
the same context instance elsewhere in your application. This may occur if you are calling
'Dispose' on the context instance, or wrapping it in a using statement. If you are using
dependency injection, you should let the dependency injection container take care of
disposing context instances.
it only works when I add await operator but is intended to not wait this time.
How can I run this type of heavy task without await operator and still use the dbContext?
Exists a better way to do it?

You can use the HostingEnvironment.QueueBackgroundWorkItem functionality of ASP.net
There are lots of docs on that, but here's the one that looks good to me:
https://www.c-sharpcorner.com/article/run-background-task-using-hostingenvironment-queuebackgroundworkitem-net-framew/
And steps that might work:
design your worker class with public properties for the query and for a task ID.
Generate a unique task ID somehow, e.g., GUID or counter
'new up' your workout BEFORE the call to QueueBackgroundWorker... so you can set the task ID and query
Queue the item for work
Insert the unique task ID into a table (or equivalent) with a blank URL and/or status or progress fields
Return the task ID to the client, end the web request. This is so the client can save it to ask for the correct document.
Worker logic
Worker will run the query and store the results in the blob storage.
Worker updates row for task ID and status/progress and fills in the URL
Now client behavior:
calls a different web API method with the task ID
server reads the data table for that ID, says 'not found', 'in progress', 'errored', 'done-here's the URL', etc.
While I was typing, 'tia' made a comment about a similar service, I don't think it's exactly the same thing and if it isn't, you could use the same design as these bullets as it appears to offer similar functionality.
Study the docs and see which might be better for you, asp.net has some cool toys as you can see!

The FastAI response was so helpful but unfortunately, I had almost no time to implement it.
DavidG recommended me to use HangFire and now my issue is finally solved
[HttpPost("export")]
public virtual async Task<IActionResult> Export([FromQuery] UrlRequestBase? urlRequestBase,
[FromBody] BodyRequestBase? bodyRequestBase)
{
object? response;
int status = 200;
try
{
await urlRequestBase.Parse(this);
await bodyRequestBase.Parse();
//HangFire solution
_backgroundJobClient.Enqueue(() => _repositoryBase.CreateExport(urlRequestBase, bodyRequestBase));
return StatusCode(status);
}
catch (ExceptionBase ex)
{
return StatusCode(ex.CodeResult, ex.CreateResponseFromException());
}
return StatusCode(status, await ResponseBase.Response(response));
}
Thank you, for taking the time to help me!

Enable ASP.Net Core Session Locking?

According to the ASP.Net Core docs, the behaviour of the session state has changed in that it is now non-locking:
Session state is non-locking. If two requests simultaneously attempt to modify the contents of a session, the last request overrides the first. Session is implemented as a coherent session, which means that all the contents are stored together. When two requests seek to modify different session values, the last request may override session changes made by the first.
My understanding is that this is different to the behaviour of the session in the .Net Framework, where the user's session was locked per request so that whenever you read from/wrote to it, you weren't overwriting another request's data or reading stale data, for that user.
My question(s):
Is there a way to re-enable this per-request locking of the user's session in .Net Core?
If not, is there a reliable way to use the session to prevent duplicate submission of data for a given user? To give a specific example, we have a payment process that involves the user returning from an externally hosted ThreeDSecure (3DS) iFrame (payment card security process). We are noticing that sometimes (somehow) the user is submitting the form within the iFrame multiple times, which we have no control over. As a result this triggers multiple callbacks to our application. In our previous .Net Framework app, we used the session to indicate if a payment was in progress. If this flag was set in the session and you hit the 3DS callback again, the app would stop you proceeding. However, now it seems that because the session isn't locked, when these near simultaneous, duplicate callbacks occur, thread 'A' sets 'payment in progress = true' but thread 'B' doesn't see that in time, it's snapshot of the session is still seeing 'payment in progress = false' and the callback logic is processed twice.
What are some good approaches to handling simultaneous requests accessing the same session, now that the way the session works has changed?

The problem that you have faced with is called Race Condition (stackoverflow, wiki). To cut-through, you'd like to get exclusive access to the session state, you can achieve that in several ways and they highly depend on your architecture.
In-process synchronization
If you have a single machine with a single process handling all requests (for example you use a self-hosted server, Kestrel), you may use lock. Just do it correctly and not how #TMG suggested.
Here is an implementation reference:
Use single global object to lock all threads:
private static object s_locker = new object();
public bool Process(string transaction) {
lock (s_locker) {
if(!HttpContext.Session.TryGetValue("TransactionId", out _)) {
... handle transaction
}
}
}
Pros: a simple solution
Cons: all requests from all users will wait on this lock
use per-session lock object. Idea is similar, but instead of a single object you just use a dictionary:
internal class LockTracker : IDisposable
{
private static Dictionary<string, LockTracker> _locks = new Dictionary<string, LockTracker>();
private int _activeUses = 0;
private readonly string _id;
private LockTracker(string id) => _id = id;
public static LockTracker Get(string id)
{
lock(_locks)
{
if(!_locks.ContainsKey(id))
_locks.Add(id, new LockTracker(id));
var res = _locks[id];
res._activeUses += 1;
return res;
}
}
void IDisposable.Dispose()
{
lock(_locks)
{
_activeUses--;
if(_activeUses == 0)
_locks.Remove(_id);
}
}
}
public bool Process(string transaction)
{
var session = HttpContext.Session;
var locker = LockTracker.Get(session.Id);
using(locker) // remove object after execution if no other sessions use it
lock (locker) // synchronize threads on session specific object
{
// check if current session has already transaction in progress
var transactionInProgress = session.TryGetValue("TransactionId", out _);
if (!transactionInProgress)
{
// if there is no transaction, set and handle it
HttpContext.Session.Set("TransactionId", System.Text.Encoding.UTF8.GetBytes(transaction));
HttpContext.Session.Set("StartTransaction", BitConverter.GetBytes(DateTimeOffset.UtcNow.ToUnixTimeSeconds()));
// handle transaction here
}
// return whatever you need, here is just a boolean.
return transactionInProgress;
}
}
Pros: manages concurrency on the session level
Cons: more complex solution
Remember that lock-based option will work only when the same process on the webserver handling all user's requests - lock is intra-process synchronization mechanism! Depending on what you use as a persistent layer for sessions (like NCache or Redis), this option might be the most performant though.
Cross-process synchronization
If there are several processes on the machine (for example you have IIS and apppool is configured to run multiple worker processes), then you need to use kernel-level synchronization primitive, like Mutex.
Cross-machine synchronization
If you have a load balancer (LB) in front of your webfarm so that any of N machines can handle user's request, then getting exclusive access is not so trivial.
One option here is to simplify the problem by enabling the 'sticky session' option in your LB so that all requests from the same user (session) will be routed to the same machine. In this case, you are fine to use any cross-process or in-process synchronization option (depends on what you have running there).
Another option is to externalize synchronization, for example, move it to the transactional DB, something similar to what #HoomanBahreini suggested. Beware that you need to be very cautious on handling failure scenarios: you may mark your session as in progress and then your webserver which handled it crashed leaving it locked in DB.
Important
In all of these options you must ensure that you obtain lock before reading the state and hold it until you update the state.
Please clarify what option is the closest to your case and I can provide more technical details.

Session is designed to store temporary user data among multiple requests, a good example is login-state... without session you would have to login to stackoverflow.com every time you open a new question... but the website remembers you, because your send them your session state inside a cookie. According to Microsoft:
The session data is backed by a cache and considered ephemeral data.
The site should continue to function without the session data.
Critical application data should be stored in the user database and
cached in session only as a performance optimization.
It is quite simple to implement a locking mechanism to solve your mutex issue, however the session itself is not a reliable storage and you may loose its content at any time.
How to identify duplicate payments?
The problem is you are getting multiple payment requests and you want to discard the duplicate ones... what's your definition of a duplicate payment?
Your current solution discard the second payment while a first one is in progress... let's say your payment takes 2 seconds to complete... what will happen if you receive the duplicate payment after 3 seconds?
Every reliable payment system includes a unique PaymentId in their request... what you need to do is to mark this PaymentId as processed in your DB. This way you won't process the same payment twice, no matter when the duplicate request arrives.
You can use a Unique Constraint on PaymentId to prevent duplicate payments:
public bool ProcessPayment(Payment payment) {
bool res = InsertIntoDb(payment);
if (res == false) {
return false; // <-- insert has failed because PaymentId is not unique
}
Process(payment);
return true;
}
Same example using lock:
public class SafePayment {
private static readonly Object lockObject = new Object();
public bool ProcessPayment(Payment payment) {
lock (lockObject) {
var duplicatePayment = ReadFromDb(payment.Id);
if (duplicatePayment != null) {
return false; // <-- duplicate
}
Process(payment);
WriteToDb(payment);
return true;
}
}
}

Async/await and resource access

I’m wondering how to implement resource access nicely using async/await. I have singleton service in web application that is acting as a proxy to LDAP and have to buffer all data on first access – all invocation after that is done via cache but after some time cache is invalidated and data should be get again. Now my implementation looks like this but it is not meet my requirements
public async Task<string> GetUserDisplayName(string username)
{
var users = await GetCachedUsers();
// code using users from cache
}
private async Task<IEnumerable<LdapUser>> GetCachedUsers()
{
var users = _Cache.Get<IEnumerable<LdapUser>>();
if (users == null)
{
users = await _Connector.GetAllUsers();
_Cache.Add(users, TimeSpan.FromHours(USER_CACHE_VALID_HOURS));
}
return users;
}
I’m wondering how to implement this in this way that when couple request go to the service first time they should be awaited on the same task but not blocked and download from LDAP should go only once. I could do this traditionally and lock the resource but that threads will be blocked and I want them to back to threadpool in async way like in async/wait pattern.

SemaphoreSlim has a WaitAsync method that will let you create a critical section in asynchronous code. You can use that semaphore to prevent multiple invocations of the method from generating the value together without actually blocking any of the threads.

Worker queue and user context

We have a worker queue that a user can add work to. When the worker item is added the context is the users (HttpContext). But its a background thread that polls the queue and executes the items one by one in order.
I cant just store the User because when the HttpContext is disposed so will the Principal object
The code that can run in the worker needs the Principal to be correct for stuff like PrincipalPermissions etc.
Also, Lifetime management (IoC) uses the HttpContext for InRequest scopes, is it possible to recreate a HttpContext with the correct principal etc.
edit:
Faking HttpContext is just a nice to have feature for Life time management, this I can work around.
But our backend code heavily depends on having the correct user principal for the thread since we use this to validate if user has access to that part of the system. I would mark as answer if someone can answer how to store a user principal with identity, roles and IsAuthenticated state and later use that on another thread

Your best practice for consuming stateful data from the HttpContext is to create your own application specific context which accepts an HttpContext in the constructor (Dependency Injected).
Your business logic should never be dependent on an HttpContext but rather your new application specific context (which may have been created using info from an HttpContext).
This will not only solve your above problems, but also increase testability of your code.
Example:
public class MyApplicationContext
{
public IPrincipal ContextPrincipal { get; set; }
public MyApplicationContext(HttpContext httpContext)
{
// Store the current user principal & identity
ContextPrincipal = httpContext.User;
// Need to grab anything else from the HttpContext? Do it here!
// That could be cookies, Http request header values, query string
// parameters, session state variables, etc.
//
// Once you gather up any other stateful data, store it here in
// your application context object as the HttpRequest can't be passed
// to another thread.
}
}
public class MyHttpHandler : IHttpHandler
{
#region IHttpHandler Members
public bool IsReusable
{
// Return false in case your Managed Handler cannot be reused for another request.
// Usually this would be false in case you have some state information preserved per request.
get { return true; }
}
public void ProcessRequest(HttpContext context)
{
// Do some work on another thread using the ThreadPool
ThreadPool.QueueUserWorkItem(new WaitCallback(DoWork), new MyApplicationContext(context));
}
public void DoWork(object state)
{
// Grab our state info which should be an instance of an
// MyApplicationContext.
MyApplicationContext context = (MyApplicationContext) state;
// Assign this ThreadPool thread's current principal according
// to our passed in application context.
Thread.CurrentPrincipal = context.ContextPrincipal;
// Check if this user is authenticated.
if (context.ContextPrincipal.Identity.IsAuthenticated)
{
var userName = context.ContextPrincipal.Identity.Name;
}
// Check if this user is an administrator.
if (context.ContextPrincipal.IsInRole("Administrator"))
{
}
// Do some long-ish process that we need to do on the threadpool
// after the HttpRequest has already been responded to earlier.
//
// This would normally be some fancy calculation/math, data
// operation or file routines.
for (int i = 0; i < 30; i++)
{
Thread.Sleep(1000);
}
}
#endregion
}
Neither the IPrincipal nor IIdentity interface explicitly offer a dispose method. So they should both be ok to keep a reference to them. However, I haven't tested the above code, I wrote it just for this question.
If by some poor design they actually do depend on an underlying database connection to query the roles membership, you'd simply have to evaluate that earlier in the constructor of your application context while the HttpContext and asp.net forms authentication provider are still non disposed/closed.
You can always take apart the principal and identity and recreate a new instance of GenericPrincipal and GenericIdentity or even create your application Identity class which implements IIdentity. There is lots of room for customization/extension here.

public void TestMethod1()
{
System.Net.WebClient client = new System.Net.WebClient();
client.BaseAddress = "http://www.teejoo.com";
//Invoke your function here
client.OpenReadAsync(new Uri("http://www.teejoo.com/YourLogicalPage.aspx"));
//Pur your logical in your page, so you can use httpContext
client.OpenReadCompleted += new System.Net.OpenReadCompletedEventHandler(client_OpenReadCompleted);
}
void client_OpenReadCompleted(object sender, System.Net.OpenReadCompletedEventArgs e)
{
//to Check the response HERE
}

Why don't you use an auxiliar class to hold the information you need? You can create it during the web request with the appropriate values and pass it down as an argument to the background worker.
Cloning the HTTPContext object is not possible because of the internal server session state. Even if it were possible, using it outside of a real HTTP request just to check for values doesn't seem like a good solution.

HTTPContext across threads

I need to instantiate a singleton object per web request, so that the data is processed once and is valid throughout the request, I was using HttpContext.Current.Items to share data during HTTP request, everything was fine until we needed the singleton object instance across multiple threads, the first thing that I came up with was to pass the HttpContext instance to the new thread:
HttpContext context = HttpContext.Current;
ThreadPool.QueueUserWorkItem(callback =>
{
HttpContext.Current = context;
// blah blah
});
Which I don't think is a thread-safe approach as noted here.
Using Reflector I figured HttpContext.Current.Items actually uses CallContext to store objects in each logical thread. So I changed the singleton interface to this:
public static SingletonType SingletonInstance
{
get { return CallContext.GetData(key) as SingletonType; }
set { CallContext.SetData(key, value); }
}
And simply overwrite SingletonInstance when starting any new thread! The code works fine, however it seems that somehow under heavy load, CallContext.GetData(key) returns null and the application crashes with with a null reference exception!
I was thinking, if CallContext.GetData is atomic? But it just doesn't seem right, the CallContext is thread specific data storage and must be atomic or I am missing the point!
My other guess is that setting the SingletonInstance (CallContext.SetData) happens in one thread while CallContext.GetData executes in another as noted here but I don't know how/why?
update:
We are keeping an instance of each online user in an array on the server. The singleton object is actually a reference to the object representing current user. Current user must be unique and available in each thread for database querying, logging, error handling and more, this is how it is done:
public static ApplicationUser CurrentUser
{
get { return CallContext.GetData("ApplicationUser") as ApplicationUser ; }
set { CallContext.SetData("ApplicationUser", value); }
}

ASP.NET may migrate request between threads if it's under load. Once request is received page constructor may execute on one thread and page load on another. In this thread switch CallContext and ThreadStatic are not migrated, but luckaly HttpContext is.
This may be misleading as HttpContext is call context, but this is a little quirk in ASP.NET, probably due to cutting corners to improve performance.
You'll have to remove dependencies to CallContext and use HttpContext entire way through.
You can read more details in this terrific blog post by Piers7.

This was resolved during a chat session.
In essence it involves long-running tasks and a suggestion of using an external service (Web, or regular Windows Service) was decided as the best solution to the problem.

Thread-safing your second method is the best approach.
This is thread-safe version of your singletone:
public sealed class SingletonType
{
#region thread-safe singletone
private static object _lock = new object();
private SingletonType() { }
public static SingletonType SingletonInstance
{
get
{
if (CallContext.GetData(key) == null)
{
lock (_lock)
{
if (CallContext.GetData(key) == null)
CallContext.SetData(key, new SingletonType());
}
}
return CallContext.GetData(key) as SingletonType;
}
}
#endregion
//
//
// SingletoneType members
//
//
}
NOTE : using a lock { } block is the key.