This question already has answers here:
Locking pattern for proper use of .NET MemoryCache
(10 answers)
Closed 6 years ago.
I'm trying to implement following scenario and unable to come up with a solution.
In my web service I've cache object (contains static data) based on the session id. Once request is received it checks whether cache contains any key for the session id.
If not available, it will load it from DB and stores it in cache.
If available it uses that cache and continues with further processing.
Now, with multithreading enabled in this service and when multiple requests (with same session id) are sent to service, all of them are trying to load the data into cache as none of them find that key initially.
Question is: I wanted to stop all the other threads till the first thread loads static data into cache and once first thread is done with loading data in to cache, other threads should use that cache instead of trying to load again.
Looks trivial but somehow not able to think of any multi threading feature which can solve this.
My code looks something like below:
somemethod()
{
if(cache.Contains(someKey)
{
// use cache and do further processing
}
else
{
cache.add(someKey)
}
}
You can try following logic
1) Thread1 for comes and finds that object is not present in cache
2) Puts a wait command object in cache for this session Id. This object tells any other threads to wait till further notice.
3) Thread1 fetches the data from DB and put it backs into cache.
4) Thread1 notifies other threads that they can proceed since data is now available.
Classic remedy againsnt the race condition is mutual exclusion. Locking is the simplest solution providing such capability.
public class Cache
{
private object _locker = new object();
private SessionDataCollection _cache;
public SessionData Get(SessionId id)
{
lock (_locker)
{
if (!Contains(id))
Fetch(id);
return Retrieve(id);
}
}
private bool Contains(SessionId id)
{
//check if present in _cache
}
private void Fetch(SessionId id)
{
//get from db and store in _cache
}
private SessionData Retrieve(SessionId id)
{
//retrvieve from _cache
}
}
I've got a scenario where I require to cache information from a webapi temporarily when it is first called. With the same parameters this API can be called a few times a second.
Due to performance restrictions I don't want each call fetching the data and putting it into the memory cache so I've implemented a system with Semaphores to try and allow one thread to initialize the cache and then allow the rest to just query that cache.
I've stripped down the code to show an example of what i'm doing currently.
private static MemoryCacher memCacher = new MemoryCacher();
private static ConcurrentDictionary<string, Semaphore> dictionary = new ConcurrentDictionary<string, Semaphore>();
private async Task<int[]> DoAThing(string requestHash)
{
// check for an existing cached result before hitting the dictionary
var cacheValue = memCacher.GetValue(requestHash);
if (cacheValue != null)
{
return ((CachedResult)cacheValue).CheeseBurgers;
}
Semaphore semi;
semi = dictionary.GetOrAdd(requestHash, new Semaphore(1, 1, requestHash));
semi.WaitOne();
//It's possible a previous thread has now filled up the cache. Have a squiz.
cacheValue = memCacher.GetValue(requestHash);
if (cacheValue != null)
{
dictionary.TryRemove(requestHash);
semi.Release();
return ((CachedResult)cacheValue).CheeseBurgers;
}
// fetch the latest data from the relevant web api
var response = await httpClient.PostAsync(url, content);
// add the result to the cache
memCacher.Add(requestHash, new CachedResult() { CheeseBurgers = response.returnArray }, DateTime.Now.AddSeconds(30));
// We have added everything to the cacher so we don't need this semaphore in the dictonary anymore:
dictionary.TryRemove(requestHash);
//Open the floodgates
semi.Release()
return response.returnArray;
}
Unfortunately there are many weird issues where more than one thread at a time manages to get through the WaitOne() call and then when released manages to break due to the count restriction on the semaphore. (to make sure only one semaphore is working at a time)
I've tried using Mutexes and Monitors, but since IIS doesn't guarantee that an API call will always run on the same thread this causes it to fail regularly when the mutex is attempted to be released in a different thread.
Any suggestions on other ways to implement this would be welcome as well!
We're using the following pattern to handle caching of universal objects for our asp.net application.
private object SystemConfigurationCacheLock = new object();
public SystemConfiguration SystemConfiguration
{
get
{
if (HttpContext.Current.Cache["SystemConfiguration"] == null)
lock (SystemConfigurationCacheLock)
{
if (HttpContext.Current.Cache["SystemConfiguration"] == null)
HttpContext.Current.Cache.Insert("SystemConfiguration", GetSystemConfiguration(), null, DateTime.Now.AddMinutes(1), Cache.NoSlidingExpiration, new CacheItemUpdateCallback(SystemConfigurationCacheItemUpdateCallback));
}
return HttpContext.Current.Cache["SystemConfiguration"] as SystemConfiguration;
}
}
private void SystemConfigurationCacheItemUpdateCallback(string key, CacheItemUpdateReason reason, out object expensiveObject, out CacheDependency dependency, out DateTime absoluteExpiration, out TimeSpan slidingExpiration)
{
dependency = null;
absoluteExpiration = DateTime.Now.AddMinutes(1);
slidingExpiration = Cache.NoSlidingExpiration;
expensiveObject = GetSystemConfiguration();
}
private SystemConfiguration GetSystemConfiguration()
{
//Load system configuration
}
The problem is that when under load (~100,000 users) we see a huge jump in TTFB as the CacheItemUpdateCallback blocks all the other threads from executing until it has finished refreshing the cache from the database.
So what I figured we needed is solution that when the first thread after an expiry of the cache attempts to access it, an asynchronous thread is fired off to update the cache but still allows all other executing threads to read from the old cache until it has sucessfully updated.
Is there anything built into the .NET framework that can natively handle what I'm asking, or will I have to write it from scratch? Your thoughts please...
A couple of things...
The use of the HttpContext.Current.Cache is incidental and not necessarily essential as we've got no problem using private members on a singleton to hold the cached data.
Please don't comment on the cache times, SPROC effeciency, why we're caching in the first place etc as it's not relevent. Thanks!
AppFabric might be a good fit for what you're looking for.
http://msdn.microsoft.com/en-us/windowsserver/ee695849
http://msdn.microsoft.com/en-us/library/ff383731.aspx
So it turns out after several hours of investigation that the problem is not the CacheItemUpdateCallback blocking other threads as I originally thought, in fact it did exactly what I wanted it to asynchronously but it was the garbage collector stopping everything to clean up the LOH.
I have the following code, in which I’m trying to process a large amount of data, and update the UI. I’ve tried the same thing using a background worker, but I get a similar issue. The problem seems to be that I’m trying to use a class that was not instantiated on the new thread (the actual error is that the current thread doesn't "own" the instance). My question is, is there a way that I can pass this instance between threads to avoid this error?
DataInterfaceClass dataInterfaceClass = new DataInterfaceClass();
private void OutputData(List<MyResult> Data)
{
progressBar1.Maximum = Data.Count;
progressBar1.Minimum = 1;
progressBar1.Value = 1;
foreach (MyResult res in Data)
{
// Add data to listview
UpdateStatus("Processing", res.Name);
foreach (KeyValuePair<int, string> dets in res.Details)
{
ThreadPool.QueueUserWorkItem((o) =>
{
// Get large amount of data from DB based on key
// – gives error because DataInterfaceClass was
// created in different thread.
MyResult tmpResult = dataInterfaceClass
.GetInfo(dets.DataKey);
if (tmpResult == null)
{
// Updates listview
UpdateStatus("Could not get details",
dets.DataKey);
}
else
{
UpdateStatus("Got Details", dets.DataKey);
}
progressBar1.Dispatcher.BeginInvoke(
(Action)(() => progressBar1.Value++));
});
}
}
}
EDIT:
DataInterfaceClass is actually definated and created outside of the function that it is used in, but it is an instance and not static.
UPDATE:
You seem to have modified the posted source code, so...
You should create an instance of the DataInterfaceClass exclusively for each background thread or task. Provide your task with enough input to create its own instance.
That being said, if you try to access data in a single database in a highly parallel way, this might result in database timeouts. Even if you can get your data access to work in a multithreaded way, I would recommend limiting the number of simultaneous background tasks to prevent this from occurring.
You could use a Semaphore (or similar) to ensure that no more than a certain amount of tasks are running at the same time.
Create a global instance for DataInterfaceClass inside the class that has OutputData method defined, that way you would be able to use it within the method.
However, you would need to be cautious in using it. If all the threads would use the same instance to read from the database, it would result in errors.
You should either create a new instance of the DataInterfaceClass in each thread, or have some lock implemented inside your GetInfo method to avoid multiple access issues.
I've got a web application that controls which web applications get served traffic from our load balancer. The web application runs on each individual server.
It keeps track of the "in or out" state for each application in an object in the ASP.NET application state, and the object is serialized to a file on the disk whenever the state is changed. The state is deserialized from the file when the web application starts.
While the site itself only gets a couple requests a second tops, and the file it rarely accessed, I've found that it was extremely easy for some reason to get collisions while attempting to read from or write to the file. This mechanism needs to be extremely reliable, because we have an automated system that regularly does rolling deployments to the server.
Before anyone makes any comments questioning the prudence of any of the above, allow me to simply say that explaining the reasoning behind it would make this post much longer than it already is, so I'd like to avoid moving mountains.
That said, the code that I use to control access to the file looks like this:
internal static Mutex _lock = null;
/// <summary>Executes the specified <see cref="Func{FileStream, Object}" /> delegate on
/// the filesystem copy of the <see cref="ServerState" />.
/// The work done on the file is wrapped in a lock statement to ensure there are no
/// locking collisions caused by attempting to save and load the file simultaneously
/// from separate requests.
/// </summary>
/// <param name="action">The logic to be executed on the
/// <see cref="ServerState" /> file.</param>
/// <returns>An object containing any result data returned by <param name="func" />.
///</returns>
private static Boolean InvokeOnFile(Func<FileStream, Object> func, out Object result)
{
var l = new Logger();
if (ServerState._lock.WaitOne(1500, false))
{
l.LogInformation( "Got lock to read/write file-based server state."
, (Int32)VipEvent.GotStateLock);
var fileStream = File.Open( ServerState.PATH, FileMode.OpenOrCreate
, FileAccess.ReadWrite, FileShare.None);
result = func.Invoke(fileStream);
fileStream.Close();
fileStream.Dispose();
fileStream = null;
ServerState._lock.ReleaseMutex();
l.LogInformation( "Released state file lock."
, (Int32)VipEvent.ReleasedStateLock);
return true;
}
else
{
l.LogWarning( "Could not get a lock to access the file-based server state."
, (Int32)VipEvent.CouldNotGetStateLock);
result = null;
return false;
}
}
This usually works, but occasionally I cannot get access to the mutex (I see the "Could not get a lock" event in the log). I cannot reproduce this locally - it only happens on my production servers (Win Server 2k3/IIS 6). If I remove the timeout, the application hangs indefinitely (race condition??), including on subsequent requests.
When I do get the errors, looking at the event log tells me that the mutex lock was achieved and released by the previous request before the error was logged.
The mutex is instantiated in the Application_Start event. I get the same results when it is instantiated statically in the declaration.
Excuses, excuses: threading/locking is not my forté, as I generally don't have to worry about it.
Any suggestions as to why it randomly would fail to get a signal?
Update:
I've added proper error handling (how embarrassing!), but I am still getting the same errors - and for the record, unhandled exceptions were never the problem.
Only one process would ever be accessing the file - I don't use a web garden for this application's web pool, and no other applications use the file. The only exception I can think of would be when the app pool recycles, and the old WP is still open when the new one is created - but I can tell from watching the task manager that the issue occurs while there is only one worker process.
#mmr: How is using Monitor any different from using a Mutex? Based on the MSDN documentation, it looks like it is effectively doing the same thing - if and I can't get the lock with my Mutex, it does fail gracefully by just returning false.
Another thing to note: The issues I'm having seem to be completely random - if it fails on one request, it might work fine on the next. There doesn't seem to be a pattern, either (certainly no every other, at least).
Update 2:
This lock is not used for any other call. The only time _lock is referenced outside the InvokeOnFile method is when it is instantiated.
The Func that is invoked is either reading from the file and deserializing into an object, or serializing an object and writing it to the file. Neither operation is done on a separate thread.
ServerState.PATH is a static readonly field, which I don't expect would cause any concurrency problems.
I'd also like to re-iterate my earlier point that I cannot reproduce this locally (in Cassini).
Lessons learned:
Use proper error handling (duh!)
Use the right tool for the job (and have a basic understanding of what/how that tool does). As sambo points out, using a Mutex apparently has a lot of overhead, which was causing issues in my application, whereas Monitor is designed specifically for .NET.
You should only be using Mutexes if you need cross-process synchronization.
Although a mutex can be used for
intra-process thread synchronization,
using Monitor is generally preferred,
because monitors were designed
specifically for the .NET Framework
and therefore make better use of
resources. In contrast, the Mutex
class is a wrapper to a Win32
construct. While it is more powerful
than a monitor, a mutex requires
interop transitions that are more
computationally expensive than those
required by the Monitor class.
If you need to support inter-process locking you need a Global mutex.
The pattern being used is incredibly fragile, there is no exception handling and you are not ensuring that your Mutex is released. That is really risky code and most likely the reason you see these hangs when there is no timeout.
Also, if your file operation ever takes longer than 1.5 seconds then there is a chance concurrent Mutexes will not be able to grab it. I would recommend getting the locking right and avoiding the timeout.
I think its best to re-write this to use a lock. Also, it looks like you are calling out to another method, if this take forever, the lock will be held forever. That's pretty risky.
This is both shorter and much safer:
// if you want timeout support use
// try{var success=Monitor.TryEnter(m_syncObj, 2000);}
// finally{Monitor.Exit(m_syncObj)}
lock(m_syncObj)
{
l.LogInformation( "Got lock to read/write file-based server state."
, (Int32)VipEvent.GotStateLock);
using (var fileStream = File.Open( ServerState.PATH, FileMode.OpenOrCreate
, FileAccess.ReadWrite, FileShare.None))
{
// the line below is risky, what will happen if the call to invoke
// never returns?
result = func.Invoke(fileStream);
}
}
l.LogInformation("Released state file lock.", (Int32)VipEvent.ReleasedStateLock);
return true;
// note exceptions may leak out of this method. either handle them here.
// or in the calling method.
// For example the file access may fail of func.Invoke may fail
If some of the file operations fail, the lock will not be released. Most probably that is the case. Put the file operations in try/catch block, and release the lock in the finally block.
Anyway, if you read the file in your Global.asax Application_Start method, this will ensure that noone else is working on it (you said that the file is read on application start, right?). To avoid collisions on application pool restaring, etc., you just can try to read the file (assuming that the write operation takes an exclusive lock), and then wait 1 second and retry if exception is thrown.
Now, you have the problem of synchronizing the writes. Whatever method decides to change the file should take care to not invoke a write operation if another one is in progress with simple lock statement.
I see a couple of potential issues here.
Edit for Update 2: If the function is a simple serialize/deserialize combination, I'd separate the two out into two different functions, one into a 'serialize' function, and one into a 'deserialize' function. They really are two different tasks. You can then have different, lock-specific tasks. Invoke is nifty, but I've gotten into lots of trouble myself going for 'nifty' over 'working'.
1) Is your LogInformation function locking? Because you call it inside the mutex first, and then once you release the mutex. So if there's a lock to write to the log file/structure, then you can end up with your race condition there. To avoid that, put the log inside the lock.
2) Check out using the Monitor class, which I know works in C# and I'd assume works in ASP.NET. For that, you can just simply try to get the lock, and fail gracefully otherwise. One way to use this is to just keep trying to get the lock. (Edit for why: see here; basically, a mutex is across processes, the Monitor is in just one process, but was designed for .NET and so is preferred. No other real explanation is given by the docs.)
3) What happens if the filestream opening fails, because someone else has the lock? That would throw an exception, and that could cause this code to behave badly (ie, the lock is still held by the thread that has the exception, and another thread can get at it).
4) What about the func itself? Does that start another thread, or is it entirely within the one thread? What about accessing ServerState.PATH?
5) What other functions can access ServerState._lock? I prefer to have each function that requires a lock get its own lock, to avoid race/deadlock conditions. If you have many many threads, and each of them try to lock on the same object but for totally different tasks, then you could end up with deadlocks and races without any really easily understandable reason. I've changed to code to reflect that idea, rather than using some global lock. (I realize other people suggest a global lock; I really don't like that idea, because of the possibility of other things grabbing it for some task that is not this task).
Object MyLock = new Object();
private static Boolean InvokeOnFile(Func<FileStream, Object> func, out Object result)
{
var l = null;
var filestream = null;
Boolean success = false;
if (Monitor.TryEnter(MyLock, 1500))
try {
l = new Logger();
l.LogInformation("Got lock to read/write file-based server state.", (Int32)VipEvent.GotStateLock);
using (fileStream = File.Open(ServerState.PATH, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None)){
result = func.Invoke(fileStream);
} //'using' means avoiding the dispose/close requirements
success = true;
}
catch {//your filestream access failed
l.LogInformation("File access failed.", (Int32)VipEvent.ReleasedStateLock);
} finally {
l.LogInformation("About to released state file lock.", (Int32)VipEvent.ReleasedStateLock);
Monitor.Exit(MyLock);//gets you out of the lock you've got
}
} else {
result = null;
//l.LogWarning("Could not get a lock to access the file-based server state.", (Int32)VipEvent.CouldNotGetStateLock);//if the lock doesn't show in the log, then it wasn't gotten; again, if your logger is locking, then you could have some issues here
}
return Success;
}