We Keep Coding

Thread Static, ASP.NET and Async handlers

Please consider these sceanrios:
An async .ashx handler
A async .asmx web-service method
A sync MVC 5 controller action method
I am trying to figure out a way to set "logical thread" specific data that can be accessed consistently during a "logical" http request, i.e. if the data was set on the thread in the "BeginExecute" part of which-ever async handler you would consider, that data is available in the "EndExecute" part of that asnc handler even if ASP.NET executes the "EndExecute" part on a different OS/.Net thread.
Moreover, I am expecting that the data set in the "BeginExecute" part on whatever OS/.Net thread it was on is NOT available on a subsequent http request if the second request is assigned the thread that was earlier assigned to first http request when it was in "BeginExecute" portion but this thread freed up as the first http request went in its async operation (and its possibly still completing its async operation).
I believe the word "logical thread" or "logical thread context" in .Net actually means the same "logical" flow of operation that I have mentioned (and not the underlying OS/.Net thread that keeps getting re-assigned). If you look at it from a workflow perspective, each http request is a new "logical" operation (even if multiple users invoke the same web-service sequentially or in parallel, each request is a new and separte logical operation), and in this meaning, the "logical" operation is one-time and cannot repeat. However the same underlying OS/.Net threads can be mapped to "logical" operations as they arrive based on their availability.
Additionally I want to expose this data as HttpContext.Current sort of static property. To some people this may come as a surprise, but HttpContext.Current does not work correctly if you are using for example async .asmx web-service methods. I am sure I have read content on web which says HttpContext.Current should always return correct HttpContext, but I have seen it as null in EndExecuteMethod of .asmx web-methods. It would be great if somecan can confirm if I am right in making my last statement, but this statement is not the overall question I am trying to ask here.
After reading a good amount of literature (e.g. What is the difference between log4net.ThreadContext and log4net.LogicalThreadContext?, http://msmvps.com/blogs/jon_skeet/archive/2010/11/08/the-importance-of-context-and-a-question-of-explicitness.aspx, http://blog.stephencleary.com/2013/04/implicit-async-context-asynclocal.html and more including MSDN docs), here are my inferences:
ThreadStatic is local to underlying OS/.Net thread and not to the "logical" operation, hence in my example; data set on first http request in "BeginExecute" would be visible in next http request if the second http request gets assigned the same thread as "BeginExecute" for first thread. And this data won't be available in "EndExecute" if it happens to be re-assigned to another thread by .Net (which would happen in vast majority of the cases).
Thread.SetData is even more problematic for my use-case. It needs data slots to be passed in and if I were to pass in a data slot from a return value of Thread.GetNamedDataSlot, the information is available across the app domain; as named data slots are shared between threads.
CallContext.SetData is like ThreadStatic (which means its not shared by app domain but different http requests would see the same data if they get assgined to the same underlying OS/.Net thread). CallContext.SetData provides an additional ability to marshal the context data for RPC calls which is irrelevant to the current question being asked.
Then there's the ThreadLocal class (.Net 4/.Net 4.5). It could have solved one part of my problem it seems, I could have passed it inside stateObject of BeingExecute operation, and extract from the same stateObject parameter of endExecute operation. From this perspective, ThreadLocal seems to be written for .Net's async support. But it won't work when I need to access it like HttpContext.Current as there's no way I can see to preserve the "logical thread static" instance ofit (unless I have said something incorrectly in my previous 3 points).
And finally it seems CallContext.LogicalSetData does what I intend to achive. Using the set of CallContext.LogicalSetData and CallContext.LogicalGetData methods, I should be able to achieve the HttpContext.Current like impact which works correctly for "logical task executions".
Now come the questions:
Is everything I have said above correct. Please correct any and all incorrect claims I have made.
Are there any other options available for thread static kind of feature in .Net that I missed.
Does CallContext.LogicalSetData/LogicalGetData pass on the context data to RPC calls (the msdn page does not mention clearly, http://msdn.microsoft.com/en-us/library/system.runtime.remoting.messaging.callcontext.logicalsetdata(v=vs.110).aspx).
Are there any downsides (performance wise or otherwise) of using CallContext.LogicalSetData/LogicalGetData.
This page says something about copy-on-write behavior for LogicalSetData: http://blog.stephencleary.com/2013/04/implicit-async-context-asynclocal.html. In the context of async handlers/async MVC 5 action methods, what's the impact if I save a reference type using logicalsetdata and later change the state of the reference type. What are the repuccursions.
For mutation/logicalsetdata/async, I still can't see what's the problem by mutating the object. When the async method starts, the copy-on-write behavior would trigger a copy of context data the next time logicalsetdata is called. This is a shallow copy, so my reference object is now actually shared by 2 logical contexts and the changes in one context are visible in the other context which is what I would normally expect from a reference type.
A long question with lots of references, but hopefully I did my research well and the answers would benefit other people too.

I am trying to figure out a way to set "logical thread" specific data that can be accessed consistently during a "logical" http request
The only possible options are HttpContext.Current.Items and the logical CallContext.
Moreover, I am expecting that the data set in the "BeginExecute" part on whatever OS/.Net thread it was on is NOT available on a subsequent http request
HttpContext.Current.Items will always be cleared on a new request, but you'll have to clear the logical CallContext data yourself.
HttpContext.Current does not work correctly if you are using for example async .asmx web-service methods.
I find this surprising. I haven't tried it, but it should work - if you are running on .NET 4.5, targeting .NET 4.5 (i.e., have targetFramework set to 4.5 in your web.config), and aren't using async void.
[ThreadStatic], thread-local data slots, (non-logical) CallContext, and ThreadLocal are all thread-specific data, and will not work for asynchronous code.
Is everything I have said above correct. Please correct any and all incorrect claims I have made.
There is really way too much text in your question. Stack Overflow is a Q&A site, not a mentoring site.
Are there any other options available for thread static kind of feature in .Net that I missed.
No.
Does CallContext.LogicalSetData/LogicalGetData pass on the context data to RPC calls
I have no idea. Try it and see.
Are there any downsides (performance wise or otherwise) of using CallContext.LogicalSetData/LogicalGetData.
There's a definite performance hit. The .NET framework is highly optimized for the common case (no logical call context data).
what's the impact if I save a reference type using logicalsetdata and later change the state of the reference type.
The logical CallContext has shallow-copy-on-write behavior. So, any kind of asynchronous fork/join concurrency (i.e., Task.WhenAll) will end up sharing that state. If you use ConfigureAwait(false), you could also end up with race conditions.
To actually solve your problem, I recommend you first look into why HttpContext.Current doesn't work as expected; my guess (without seeing the project) is that targetFramework is set to 4.0 instead of 4.5. HttpContext.Current.Items is the most performant choice if you can get it working.

Maintain a reference to HttpContext on different thread after request has completed

I've been trying to determine how safe it is to maintain a read-only reference to the HttpContext after a request has completed. Specifically, I'm looking to grab a reference to HttpContext.Current when a request is executing, and then place a reference to the context into an in-memory queue for later evaluation (possibly up to a few seconds later) on a different thread.
Obviously, trying to work with and attempting to write to the context.Response won't work and I'm not particularly interested in that. Instead, I'm interested in reading a few values on the context.Request property and a few other values from the context itself, e.g. context.Timestamp, context.Items, and maybe a few others.
Preliminary tests indicate I can do this, but I'm trying to program intentionally rather than by coincidence. Can anyone point to any documentation (or perhaps a blog post or something from Jon Skeet) that indicates that references can be maintained after the HTTP request is complete?

No, this isn't safe, but it has little to do with threading.
It's HttpContext.Current. That's context as in "context of the current request".
Don't use this after the request is over!
The various IDisposable resources in the context will have been disposed of, so don't use them. Any number of other things that were valid during the request will not be valid because the request is over.
Once you decide what parts of it you need, copy those parts and save them.

Associate data with a thread: How do you do that?

While writing some aysnc controllers in ASP.NET MVC2, I ran up against a situation where I had to call AsyncManager.Sync. It got me wondering:
When I retrieve HttpContext.Current, what's going on? How does ASP.NET know which HttpContext I'm after? How is the current context associated with this thread, and how is it retrieved?

Thread scoped storage is used, this is called Thread Local Storage.
This mechanism allows data to be affinitized with a thread i.e. only the thread that allocates the data sees the data. This is useful for creating so called ambient programming models such as HttpContext.Current and TransactionScope. The mechanism allows data to be accessible at any time on the executing thread without having to "tramp" data through method parameters for instance. It's an elegant solution for certain context\orthogonal problems.
There are number of ways of using TLS including the ThreadStaticAttribute and Thread.SetData\GetData.

Singleton pattern in web applications

I'm using a singleton pattern for the datacontext in my web application so that I dont have to instantiate it every time, however I'm not sure how web applications work, does IIS open a thread for every user connected? if so, what would happend if my singleton is not thread safe? Also, is it OK to use a singleton pattern for the datacontext? Thanks.

I'm using a singleton pattern for the datacontext in my web application
"Singleton" can mean many different things in this context. Is it single-instance per request? Per session? Per thread? Per AppDomain (static instance)? The implications of all of these are drastically different.
A "singleton" per request (stored in the HttpContext) is fine. A singleton per session is discouraged, but can be made to work. A singleton per thread may appear to work but is likely to result in unexpected and difficult-to-debug behaviour. A singleton per Application or AppDomain is a disaster waiting to happen.
so that I dont have to instantiate it every time
Creating a DataContext is very, very cheap. The metadata is globally cached, and connections aren't created until you actually execute a query. There is no reason to try to optimize away the construction of a DataContext instance.
however I'm not sure how web applications work, does IIS open a thread for every user connected?
IIS uses a different thread for every request, but a single request may use multiple threads, and the threads are taken from the Thread Pool, which means that ultimately the same user will have requests on many different threads, and conversely, different users will share the same thread over multiple requests and an extended period of time. That is why I mention above that you cannot rely on a Thread-Local Singleton.
if so, what would happend if my singleton is not thread safe?
Very bad things. Anything that you cache globally in an ASP.NET application either needs to be made thread safe or needs to be locked while it is in use.
Also, is it OK to use a singleton pattern for the datacontext? Thanks.
A DataContext is not thread-safe, and in this case, even if you lock the DataContext while it is in use (which is already a poor idea), you can still run into cross-thread/cross-request race conditions. Don't do this.
DataContext instances should be confined to the scope of a single method when possible, using the using clause. The next best thing is to store them in the HttpContext. If you must, you can store one in the Session, but there are many things you need to be aware of (see this question I answered recently on the ObjectContext - almost all of the same principles apply to a DataContext).
But above all, do not create "global" singleton instances of a DataContext in an ASP.NET application. You will deeply regret it later.

Many people keep the DataContext around for the duration of the request by keeping it in the HttpContext.Current.Items Thereby it is also private to the request.
Have a look at this blogpost by Steve Sanderson, and the UnitOfWork pattern.

Static variables are visible to all users on the per app domain, not per session. Once created, the variable will sit in memory for the lifetime of the app domain, even if there are no active references to the object.
So if you have some sort of stateful information in a web app that shouldn't be visible to other users, it should absolutely not be static. Store that sort of information in the users session instead, or convert your static var to something like this:
public static Data SomeData
{
get
{
if (HttpContext.Session["SomeData"] == null)
HttpContext.Session["SomeData"] = new Data();
return (Data)HttpContext.Session["SomeData"];
}
}
It looks like a static variable, but its session specific, so the data gets garbage collected when the session dies and its totally invisible to other users. There safety is not guaranteed.
Additionally, if you have stateful information in a static variable, you need some sort of syncronization to modify it, otherwise you'll have a nightmare of race conditions to untangle.

#ryudice the web server creates a new thread for each request. I think the best approach is to have a datacontext bound to each request, meaning that you should create a new datacontext every time you serve a request. A good way of achieving this is by using a DI tool, such as StructureMap. These kind of tools allow you to setup the lifecycle of the instances you configure, so for example in your case you would configure your XDataContext class to be HttpContext scoped.
Regards.

here are Microsoft's examples on how to do multi-tier with LINQ-To-SQL.
http://code.msdn.microsoft.com/multitierlinqtosql

Server-side equivalent of HttpContext?

I have a web app that currently uses the current HttpContext to store a LINQ Data Context. The context is persisted for the current request, on a per user basis, per Rick Strahl's blog:
string ocKey = "ocm_" + HttpContext.Current.GetHashCode().ToString("x")
Thread.CurrentContext.ContextID.ToString();
if (!HttpContext.Current.Items.Contains(ocKey))
{
// Get new Data Context and store it in the HTTP Context
}
However, I have some scripts that execute from the global.asax file, that don't have an HttpContext. The HttpContext.Current is NULL, because the server is the one making the "request".
Is there an equivalent object that I can use to store the Data Context? So I don't have to worry about re-creating it, and attaching/detaching objects? I only want to persist the context for the lifetime of my processes.
UPDATED:
I am currently trying to use a static variable in my DAL helper class. on the first call to one of the methods in the class the DataContext is instantiated, and stored in the static variable. At the end of my process, I call another method that calls Dispose on the DataContext, and sets the static variable to NULL.

Can you not just use a static variable specifically for those scripts? That will have the same life-time as the AppDomain. You should probably think carefully about any concurrency concerns, but it sounds like the simplest way to keep a value around.
(I've just checked, and although one instance of HttpApplication can be used to service multiple requests, each one only serves one request at a time - which suggests that multiple instances are created for concurrent request processing. I haven't validated this, but it does sound like it wouldn't be safe to keep it in an instance variable.)
EDIT: Josh's answer suggests that you want this to be per-thread. That sounds slightly odd to me, as unless you've got a lot of these events occurring, you're quite likely to only ever see them execute on different threads, making the whole sharing business pointless. If you really do want that sort of thing, I'd suggest just using an instance variable in the HttpApplication-derived class - for exactly the reason described in the paragraph above :)

Why not use the current HttpContext? The scripts in your global.asax file are all the result of a request coming into the server, so there should be a context associated with that request which you can grab.
I don't understand the need for generating the key based on the hashcode or the thread. There is going to be a separate instance of HttpContext for each request that comes in, and that instance is going to be specific to the thread that is processing the request. Because of that, the key is pretty much worthless when it's based on the instance of HttpContext and the thread.
Also, how do you dispose of the DataContext when you are done? It implements IDisposable for a reason, so I would recommend against a shared instance like this.
UPDATE
In the comments, it indicates that there is a timer that is running that is executing the scripts. Instead of the timer, I would recommend setting up a Scheduled Task which will call a webservice or predetermined page on the site which will perform the task. Then you will always have an HttpContext to work with.

HttpContext.Current is a static method and should be available from anywhere as long as the code is executing within the context of a request.
In your case your not executing within the context of a request, You could look at using Application.Cache but I would caution against holding a DataContext open. I am not very famillar with linq to entities, so I could be wrong, but generally caching data base related items such as connections is bad.
I would also recommend that you consider moving the logic out of your global.asax and to a windows service. This would let you have more control over these tasks, for example you can shut them down seperatley of the web site.
Edit
As JS points out you could use a static variable. You could also define an instance variable marked with ThreadLocal attribute. This will give each thread its own copy of the variable, and can eliminate contention. Since you want each thread to have its own copy anyways.

Is there a reason why these need to be handled the same way as the other DataContexts? It seems to me that if the context is only needed inside the event handling routine, you shouldn't need to keep it around. Especially if it is in Application_Start (as per your comment), I wouldn't bother caching it anywhere -- just use it locally and pass it to the other methods as needed.

Set the DataContext as the state parameter when creating the timer. Based on the info you posted on the comments, it seems to me that your DataContext is more related to the timers than anything else.
Also avoid using the same DataContext for different timers, because you would end up with mixed modifications from the different timers. Also make sure your same timer logic isn't run twice, since it would cause the same i.e. too short period with no control.