I saw a code where they have the data access layer like this:
public class CustomerDA{
private static readonly object _sync = new object();
private static readonly CustomerDA _mutex = new CustomerDA();
private CustomerDA(){
}
public CustomerDA GetInstance(){
lock(_sync){
return _mutex;
}
}
public DataSet GetCustomers(){
//database SELECT
//return a DataSet
}
public int UpdateCustomer(some parameters){
//update some user
}
}
public class CustomerBO{
public DataSet GetCustomers(){
//some bussiness logic
return CustomerDA.GetInstance().GetCustomers();
}
}
I was using it, but start thinking... "and what if had to build a facebook like application where there are hundreds of thousands of concurrent users? would I be blocking each user from doing his things until the previous user ends his database stuff? and for the Update method, is it useful to LOCK THREADS in the app when database engines already manage concurrency at database server level?"
Then I started to think about moving the lock to the GetCustomers and UpdateCustomer methods, but think again: "is it useful at all?"
Edit on January 03:
you're all right, I missed the "static" keyword in the "GetInstance" method.
Antoher thing: I was in the idea that no thread could access the _mutex variable if there was another thread working in the same data access class. I mean, I thought that since the _mutex variable is being returned from inside the lock statement, no thread could access the _mutex until the ";" was reached in the following sentence:
return CustomerDA.GetInstance().GetCustomer();
After doing some tracing, I realize I was making the wrong assumption. Could you please confirm that I was making the wrong assumption?
So... Can I say for sure that my Data Access layer does not need any lock statement (even on INSERT, UPDATE, DELETE) and that it does not matter if methods in my DataAccess are static or instance methods?
Thanks again... your comments are so useful to me
The lock in that code is completely pointless. It locks around code that returns a value that never changes, so there is no reason to have a lock there. The purpose of the lock in the code is to make the object a singleton, but as it's not using lazy initialisation, the lock is not needed at all.
Making the data access layer a singleton is a really bad idea, that means that only one thread at a time can access the database. It also means that the methods in the class have to use locks to make sure that only one thread at a time accesses the database, or the code won't work properly.
Instead, each thread should get their own instance of the data access layer, with their own connection to the database. That way the database takes care of the concurrency issues, and the theads doesn't have to do any locking at all.
Set your lock where it is needed, so where concurrent accesses happen. Put in only as much code inside lock/critical section as much really need.
That GetInstance shouldn't be static ?
the following pseudo code explains how GetInstance operates:
LOCK
rval = _mutex
UNLOCK
Return rval
_mutex is readonly, refers to a non-null object, so it can't be changed, why lock ?
If your database provides concurrency management, but in your program you create two thread writing the same data in the same time in your own domain while waiting for the data,
how could your database help ?
Related
I'm exploring the source code for a class that was mentioned(for educational purposes) but stuck in one place:
Each instance of a lock has a unique _lockID assigned.
There is an internal ReaderWriterCount helper defined to store thread specific data per lock.
There is a pool of records defined for a thread in the thread-static field:
[ThreadStatic]
private static ReaderWriterCount? t_rwc;
The GetThreadRWCount methods returns a first empty record from the pool or append a new one if needed. Record is updated with _lockId reference:
empty.lockID = _lockID;
return empty;
And one more utility method to check if the record is referencing the current lock:
private bool IsRwHashEntryChanged(ReaderWriterCount lrwc)
{
return lrwc.lockID != _lockID;
}
And, finally, the logic I'm struggling to understand:
_spinLock.Enter(EnterSpinLockReason.EnterAnyRead);
lrwc = GetThreadRWCount(dontAllocate: false)!;
// ********* some other code ********** //
_spinLock.Exit();
spinCount++;
SpinWait(spinCount);
_spinLock.Enter(EnterSpinLockReason.EnterAnyRead);
// The per-thread structure may have been recycled as the lock is acquired (due to message pumping), load again.
if (IsRwHashEntryChanged(lrwc))
lrwc = GetThreadRWCount(dontAllocate: false)!;
continue;
The question is why this lrwc re-check is needed. The fact that it's placed within the lock and re-validated after the lock is acquired again after pause signalizes that the record(its owner) could be changed in between, but that means the same thread must request an access to a different lock at the same. And even if that's a case(have no idea how it is possible) how the spin-lock, that is local to a current instance, can help to solve the issue. There seems to be some flow in my understanding of multithreading. I would be very grateful if you could help me figure it out.
I've seen many examples of the lock usage, and it's usually something like this:
private static readonly object obj = new object();
lock (obj)
{
// code here
}
Is it possible to lock based on a property of a class? I didn't want to lock globally for any calls to the method with the lock statement, I'd like to lock only if the object passed as argument had the same property value as another object which was being processed prior to that.
Is that possible? Does that make sense at all?
This is what I had in mind:
public class GmailController : Controller
{
private static readonly ConcurrentQueue<PushRequest> queue = new ConcurrentQueue<PushRequest>();
[HttpPost]
public IActionResult ProcessPushNotification(PushRequest push)
{
var existingPush = queue.FirstOrDefault(q => q.Matches(push));
if (existingPush == null)
{
queue.Enqueue(push);
existingPush = push;
}
try
{
// lock if there is an existing push in the
// queue that matches the requested one
lock (existingPush)
{
// process the push notification
}
}
finally
{
queue.TryDequeue(out existingPush);
}
}
}
Background: I have an API where I receive push notifications from Gmail's API when our users send/receive emails. However, if someone sends a message to two users at the same time, I get two push notifications. My first idea was querying the database before inserting (based on subject, sender, etc). In some rare cases, the query of the second call is made before the SaveChanges of the previous call, so I end up having duplicates.
I know that if I ever wanted to scale out, lock would become useless. I also know I could just create a job to check recent entries and eliminate duplicates, but I was trying something different. Any suggestions are welcome.
Let me first make sure I understand the proposal. The problem given is that we have some resource shared to multiple threads, call it database, and it admits two operations: Read(Context) and Write(Context). The proposal is to have lock granularity based on a property of the context. That is:
void MyRead(Context c)
{
lock(c.P) { database.Read(c); }
}
void MyWrite(Context c)
{
lock(c.P) { database.Write(c); }
}
So now if we have a call to MyRead where the context property has value X, and a call to MyWrite where the context property has value Y, and the two calls are racing on two different threads, they are not serialized. However, if we have, say, two calls to MyWrite and a call to MyRead, and in all of them the context property has value Z, those calls are serialized.
Is this possible? Yes. That doesn't make it a good idea. As implemented above, this is a bad idea and you shouldn't do it.
It is instructive to learn why it is a bad idea.
First, this simply fails if the property is a value type, like an integer. You might think, well, my context is an ID number, that's an integer, and I want to serialize all accesses to the database using ID number 123, and serialize all accesses using ID number 345, but not serialize those accesses with respect to each other. Locks only work on reference types, and boxing a value type always gives you a freshly allocated box, so the lock would never be contested even if the ids were the same. It would be completely broken.
Second, it fails badly if the property is a string. Locks are logically "compared" by reference, not by value. With boxed integers, you always get different references. With strings, you sometimes get different references! (Because of interning being applied inconsistently.) You could be in a situation where you are locking on "ABC" and sometimes another lock on "ABC" waits, and sometimes it does not!
But the fundamental rule that is broken is: you must never lock on an object unless that object has been specifically designed to be a lock object, and the same code which controls access to the locked resource controls access to the lock object.
The problem here is not "local" to the lock but rather global. Suppose your property is a Frob where Frob is a reference type. You don't know if any other code in your process is also locking on that same Frob, and therefore you don't know what lock ordering constraints are necessary to prevent deadlocks. Whether a program deadlocks or not is a global property of a program. Just like you can build a hollow house out of solid bricks, you can build a deadlocking program out of a collection of locks that are individually correct. By ensuring that every lock is only taken out on a private object that you control, you ensure that no one else is ever locking on one of your objects, and therefore the analysis of whether your program contains a deadlock becomes simpler.
Note that I said "simpler" and not "simple". It reduces it to almost impossible to get correct, from literally impossible to get correct.
So if you were hell bent on doing this, what would be the right way to do it?
The right way would be to implement a new service: a lock object provider. LockProvider<T> needs to be able to hash and compare for equality two Ts. The service it provides is: you tell it that you want a lock object for a particular value of T, and it gives you back the canonical lock object for that T. When you're done, you say you're done. The provider keeps a reference count of how many times it has handed out a lock object and how many times it got it back, and deletes it from its dictionary when the count goes to zero, so that we don't have a memory leak.
Obviously the lock provider needs to be threadsafe and needs to be extremely low contention, because it is a mechanism designed to prevent contention, so it had better not cause any! If this is the road you intend to go down, you need to get an expert on C# threading to design and implement this object. It is very easy to get this wrong. As I have noted in comments to your post, you are attempting to use a concurrent queue as a sort of poor lock provider and it is a mass of race condition bugs.
This is some of the hardest code to get correct in all of .NET programming. I have been a .NET programmer for almost 20 years and implemented parts of the compiler and I do not consider myself competent to get this stuff right. Seek the help of an actual expert.
Although I find Eric Lippert's answer fantastic and marked it as the correct one (and I won't change that), his thoughts made me think and I wanted to share an alternative solution I found to this problem (and I'd appreciate any feedbacks), even though I'm not going to use it as I ended up using Azure functions with my code (so this wouldn't make sense), and a cron job to detected and eliminate possible duplicates.
public class UserScopeLocker : IDisposable
{
private static readonly object _obj = new object();
private static ICollection<string> UserQueue = new HashSet<string>();
private readonly string _userId;
protected UserScopeLocker(string userId)
{
this._userId = userId;
}
public static UserScopeLocker Acquire(string userId)
{
while (true)
{
lock (_obj)
{
if (UserQueue.Contains(userId))
{
continue;
}
UserQueue.Add(userId);
return new UserScopeLocker(userId);
}
}
}
public void Dispose()
{
lock (_obj)
{
UserQueue.Remove(this._userId);
}
}
}
...then you would use it like this:
[HttpPost]
public IActionResult ProcessPushNotification(PushRequest push)
{
using(var scope = UserScopeLocker.Acquire(push.UserId))
{
// process the push notification
// two threads can't enter here for the same UserId
// the second one will be blocked until the first disposes
}
}
The idea is:
UserScopeLocker has a protected constructor, ensuring you call Acquire.
_obj is private static readonly, only the UserScopeLocker can lock this object.
_userId is a private readonly field, ensuring even its own class can't change its value.
lock is done when checking, adding and removing, so two threads can't compete on these actions.
Possible flaws I detected:
Since UserScopeLocker relies on IDisposable to release some UserId, I can't guarantee the caller will properly use using statement (or manually dispose the scope object).
I can't guarantee the scope won't be used in a recursive function (thus possibly causing a deadlock).
I can't guarantee the code inside the using statement won't call another function which also tries to acquire a scope to the user (this would also cause a deadlock).
Ok, I've used locks quite a bit, but I've never had this scenario before. I have two different classes that contain code used to modify the same MSAccess database:
public class DatabaseNinja
{
public void UseSQLKatana
{
//Code to execute queries against db.TableAwesome
}
}
public class DatabasePirate
{
public void UseSQLCutlass
{
//Code to execute queries against db.TableAwesome
}
}
This is a problem, because transactions to the database cannot be executed in parallel, and these methods (UseSQLKatana and UseSQLCutlass) are called by different threads.
In my research, I see that it is bad practice to use a public object as a lock object so how do I lock these methods so that they don't run in tandem? Is the answer simply to have these methods in the same class? (That is actually not so simple in my real code)
Well, first off, you could create a third class:
internal class ImplementationDetail
{
private static readonly object lockme = new object();
public static void DoDatabaseQuery(whatever)
{
lock(lockme)
ReallyDoQuery(whatever);
}
}
and now UseSQLKatana and UseSQLCutlass call ImplementationDetail.DoDatabaseQuery.
Second, you could decide to not worry about it, and lock an object that is visible to both types. The primary reason to avoid that is because it becomes difficult to reason about who is locking the object, and difficult to protect against hostile partially trusted code locking the object maliciously. If you don't care about either downside then you don't have to blindly follow the guideline.
The reason it's bad practice to lock on a public object is that you can never be sure who ELSE is locking on that object. Although unlikely, someone else someday can decide that they want to grab your lock object, and do some process that ends up calling your code, where you lock onto that same lock object, and now you have an impossible deadlock to figure out. (It's the same issue for using 'this').
A better way to do this would be to use a public Mutex object. These are much more heavyweight, but it's much easier to debug the issue.
Use a Mutex.
You can create mutex in main class and call Wait method at the beginning of each class (method); then set mutex so when the other method is called it gonna wait for first class to finish.
Ah, remember to release mutex exiting from those methods...
I see two differing questions here:
Why is it a bad idea to lock on a public object?
The idea is that locking on an object restricts access while the lock is maintained - this means none of its members can be accessed, and other sources may not be aware of the lock and attempt to utilise the instance, even trying to acquire a lock themselves, hence causing problems.
For this reason, use a dedicated object instance to lock onto.
How do I lock these methods so that they don't run in tandem?
You could consider the Mutex class; creating a 'global' mutex will allow your classes to operate on the basis of knowing the state of the lock throughout the application. Or, you could use a shared ReaderWriterLockSlim instance, but I wouldn't really recommend the cross-class sharing of it.
You can use a public LOCK object as a lock object. You'll just have to specify that the object you're creating is a Lock object solely used for locking the Ninja and Pirate class.
In a c# program, I have 2 threads which launches a stored procedure.
This store procedure reads and writes data in some tables.
When I start my program, I have sometimes a SQL server exception (lock trouble).
To avoid deadlock, I tried to add a lock(this){ ... } in my program to avoid simultaneous calls of this stored procedure but without success (same exception)
How can fix that ?
lock(this) will not solve your concurrency problems, if more than one instance of the class is running, as the locks will refer to different this references, i.e.
public class Locker
{
public void Work()
{
lock (this)
{
//do something
}
}
}
used as (assume these codes are run in parallel)
Locker first = new Locker(); Locker second = new Locker();
first.Work() // <-- locks on first second.Work() // <-- locks on second
will lock on different objects and not really lock at all.
Using this pattern
public class Locker
{
private static object lockObject = new object();
// a static doodad for locking
public void Work()
{
lock (lockObject)
{
//do something
}
}
}
will lock on the same thing in both cases, and make the second call wait.
However, in most cases from my experience, lock problems in SQL Server procedures were the fault of the procedure itself, holding transactions open longer than neccessary, opening unneeded transactions, having suboptimal queries, etc. Making your sp calls wait in line in the C# code, instead of waiting in line at the SQL Server, does not solve those problems.
Also, deadlocks are a specific category of concurency issues that almost always can be solved by refactoring the solution with data access in mind. Give us more info about the problem, there might be a solution that does not need application-level locks at all.
As explained by #SWeko, C#'s lock will only resolve concurrency issue among threads of the current AppDomain, so if more than one AppDomains are running, let us say two desktop clients for simplicity, then they will run into deadlock. See Cross-Process Locking in C# and What is the difference between lock and Mutex? for more details.
It would be much better, even in case of desktop application, that you deal with deadlock issue within your stored procedure. The default behavior would be that your second request will wait till timeout for the first to finish and if you don't want to wait then use WITH(NOWAIT). Explore more
I have a class used to cache access to a database resource. It looks something like this:
//gets registered as a singleton
class DataCacher<T>
{
IDictionary<string, T> items = GetDataFromDb();
//Get is called all the time from zillions of threads
internal T Get(string key)
{
return items[key];
}
IDictionary<string, T> GetDataFromDb() { ...expensive slow SQL access... }
//this gets called every 5 minutes
internal void Reset()
{
items.Clear();
}
}
I've simplified this code somewhat, but the gist of it is that there is a potential concurrency issue, in that while the items are being cleared, if Get is called things may go awry.
Now I can just bung lock blocks into Get and Reset, but I'm worried that the locks on the Get will reduce performance of the site, as Get is called by every request thread in the web app many many times.
I can do something with a doubly checked locks I think, but I suspect there is a cleaner way to do this using something smarter than the lock{} block. What to do?
edit: Sorry all, I didn't make this explicit earlier, but the items.Clear() implementation I am using is not in fact a straight dictionary. Its a wrapper around a ResourceProvider which requires that the dictionary implementation calls .ReleaseAllResources() on each of the items as they get removed. This means that calling code doesn't want to run against an old version that is in the midst of disposal. Given this, is the Interlocked.Exchange method the correct one?
I would start by testing it with just a lock; locks are very cheap when not contested. However - a simpler scheme is to rely on the atomic nature of reference updates:
public void Clear() {
var tmp = GetDataFromDb(); // or new Dictionary<...> for an empty one
items = tmp; // this is atomic; subsequent get/set will use this one
}
You might also want to make items a volatile field, just to be sure it isn't held in the registers anywhere.
This still has the problem that anyone expecting there to be a given key may get disappointed (via an exception), but that is a separate issue.
The more granular option might be a ReaderWriterLockSlim.
One option is to completely replace the IDictionary instance instead of Clearing it. You can do this in a thread-safe way using the Exchange method on the Interlocked class.
See if the database will tell you what data has change. You could use
Trigger to write changes to a history table
Query Notifications (SqlServer and Oracle has these, other must do as well)
Etc
So you don’t have to reload all the data based on a timer.
Failing this.
I would make the Clear method create a new IDictionary by calling GetDataFromDB(), then once the data has been loaded set the “items” field to point to the new Dictionary. (The garbage collector will clean up the old dictionary once no threads are accessing it.)
I don’t think you care if some threads
get “old” results while reloading the
data – (if you do then you will just
have to block all threads on a lock –
painful!)
If you need all thread to swap over to the new dictionary at the same time, then you need to declare the “items” field to be volatile and use the Exchange method on the Interlocked class. However it is unlikely you need this in real life.