We Keep Coding

C# lock based on class property

I've seen many examples of the lock usage, and it's usually something like this:
private static readonly object obj = new object();
lock (obj)
{
// code here
}
Is it possible to lock based on a property of a class? I didn't want to lock globally for any calls to the method with the lock statement, I'd like to lock only if the object passed as argument had the same property value as another object which was being processed prior to that.
Is that possible? Does that make sense at all?
This is what I had in mind:
public class GmailController : Controller
{
private static readonly ConcurrentQueue<PushRequest> queue = new ConcurrentQueue<PushRequest>();
[HttpPost]
public IActionResult ProcessPushNotification(PushRequest push)
{
var existingPush = queue.FirstOrDefault(q => q.Matches(push));
if (existingPush == null)
{
queue.Enqueue(push);
existingPush = push;
}
try
{
// lock if there is an existing push in the
// queue that matches the requested one
lock (existingPush)
{
// process the push notification
}
}
finally
{
queue.TryDequeue(out existingPush);
}
}
}
Background: I have an API where I receive push notifications from Gmail's API when our users send/receive emails. However, if someone sends a message to two users at the same time, I get two push notifications. My first idea was querying the database before inserting (based on subject, sender, etc). In some rare cases, the query of the second call is made before the SaveChanges of the previous call, so I end up having duplicates.
I know that if I ever wanted to scale out, lock would become useless. I also know I could just create a job to check recent entries and eliminate duplicates, but I was trying something different. Any suggestions are welcome.

Let me first make sure I understand the proposal. The problem given is that we have some resource shared to multiple threads, call it database, and it admits two operations: Read(Context) and Write(Context). The proposal is to have lock granularity based on a property of the context. That is:
void MyRead(Context c)
{
lock(c.P) { database.Read(c); }
}
void MyWrite(Context c)
{
lock(c.P) { database.Write(c); }
}
So now if we have a call to MyRead where the context property has value X, and a call to MyWrite where the context property has value Y, and the two calls are racing on two different threads, they are not serialized. However, if we have, say, two calls to MyWrite and a call to MyRead, and in all of them the context property has value Z, those calls are serialized.
Is this possible? Yes. That doesn't make it a good idea. As implemented above, this is a bad idea and you shouldn't do it.
It is instructive to learn why it is a bad idea.
First, this simply fails if the property is a value type, like an integer. You might think, well, my context is an ID number, that's an integer, and I want to serialize all accesses to the database using ID number 123, and serialize all accesses using ID number 345, but not serialize those accesses with respect to each other. Locks only work on reference types, and boxing a value type always gives you a freshly allocated box, so the lock would never be contested even if the ids were the same. It would be completely broken.
Second, it fails badly if the property is a string. Locks are logically "compared" by reference, not by value. With boxed integers, you always get different references. With strings, you sometimes get different references! (Because of interning being applied inconsistently.) You could be in a situation where you are locking on "ABC" and sometimes another lock on "ABC" waits, and sometimes it does not!
But the fundamental rule that is broken is: you must never lock on an object unless that object has been specifically designed to be a lock object, and the same code which controls access to the locked resource controls access to the lock object.
The problem here is not "local" to the lock but rather global. Suppose your property is a Frob where Frob is a reference type. You don't know if any other code in your process is also locking on that same Frob, and therefore you don't know what lock ordering constraints are necessary to prevent deadlocks. Whether a program deadlocks or not is a global property of a program. Just like you can build a hollow house out of solid bricks, you can build a deadlocking program out of a collection of locks that are individually correct. By ensuring that every lock is only taken out on a private object that you control, you ensure that no one else is ever locking on one of your objects, and therefore the analysis of whether your program contains a deadlock becomes simpler.
Note that I said "simpler" and not "simple". It reduces it to almost impossible to get correct, from literally impossible to get correct.
So if you were hell bent on doing this, what would be the right way to do it?
The right way would be to implement a new service: a lock object provider. LockProvider<T> needs to be able to hash and compare for equality two Ts. The service it provides is: you tell it that you want a lock object for a particular value of T, and it gives you back the canonical lock object for that T. When you're done, you say you're done. The provider keeps a reference count of how many times it has handed out a lock object and how many times it got it back, and deletes it from its dictionary when the count goes to zero, so that we don't have a memory leak.
Obviously the lock provider needs to be threadsafe and needs to be extremely low contention, because it is a mechanism designed to prevent contention, so it had better not cause any! If this is the road you intend to go down, you need to get an expert on C# threading to design and implement this object. It is very easy to get this wrong. As I have noted in comments to your post, you are attempting to use a concurrent queue as a sort of poor lock provider and it is a mass of race condition bugs.
This is some of the hardest code to get correct in all of .NET programming. I have been a .NET programmer for almost 20 years and implemented parts of the compiler and I do not consider myself competent to get this stuff right. Seek the help of an actual expert.

Although I find Eric Lippert's answer fantastic and marked it as the correct one (and I won't change that), his thoughts made me think and I wanted to share an alternative solution I found to this problem (and I'd appreciate any feedbacks), even though I'm not going to use it as I ended up using Azure functions with my code (so this wouldn't make sense), and a cron job to detected and eliminate possible duplicates.
public class UserScopeLocker : IDisposable
{
private static readonly object _obj = new object();
private static ICollection<string> UserQueue = new HashSet<string>();
private readonly string _userId;
protected UserScopeLocker(string userId)
{
this._userId = userId;
}
public static UserScopeLocker Acquire(string userId)
{
while (true)
{
lock (_obj)
{
if (UserQueue.Contains(userId))
{
continue;
}
UserQueue.Add(userId);
return new UserScopeLocker(userId);
}
}
}
public void Dispose()
{
lock (_obj)
{
UserQueue.Remove(this._userId);
}
}
}
...then you would use it like this:
[HttpPost]
public IActionResult ProcessPushNotification(PushRequest push)
{
using(var scope = UserScopeLocker.Acquire(push.UserId))
{
// process the push notification
// two threads can't enter here for the same UserId
// the second one will be blocked until the first disposes
}
}
The idea is:
UserScopeLocker has a protected constructor, ensuring you call Acquire.
_obj is private static readonly, only the UserScopeLocker can lock this object.
_userId is a private readonly field, ensuring even its own class can't change its value.
lock is done when checking, adding and removing, so two threads can't compete on these actions.
Possible flaws I detected:
Since UserScopeLocker relies on IDisposable to release some UserId, I can't guarantee the caller will properly use using statement (or manually dispose the scope object).
I can't guarantee the scope won't be used in a recursive function (thus possibly causing a deadlock).
I can't guarantee the code inside the using statement won't call another function which also tries to acquire a scope to the user (this would also cause a deadlock).

Volatile for structs and collections of structs

I would like to use net wisdom to clarify some moments regarding multi-threading in .net. There are a lot of stuff in the internet about it however I was not able to find a good answer to my question.
Let say we want to maintain a state of something in our class with safety for concurrent threads. Easy case is when state is int:
class Class1
{
volatile int state = 0;
public int State
{
get
{
return state;
}
}
public Action<int> StateUpdated;
public void UpdateState(int newState)
{
state = newState;
if (StateUpdated != null)
StateUpdated(newState);
}
}
'volatile' should be enough in this case. Whatever thread needs to get current state it can use 'State' property which will never be cached. Whatever thread wants to update state it can do it safely using 'UpdateState'.
However, what to do if state is a structure? Is a complete 'lock' the only way? Side question: can a variable still be cached inside the lock?
struct StateData
{
//some fields
}
class Class1
{
StateData state;
public StateData State
{
get
{
return state;
}
}
public Action<StateData> StateUpdated;
public void UpdateState(StateData newState)
{
state = newState;
if (StateUpdated != null)
StateUpdated(newState);
}
}
And eventually the main question: will this code be sufficient for managing a collection of state objects in multi-threading environment? Or there might be some hidden problems.
public struct StateData
{
//some fields
}
public delegate void StateChangedHandler(StateData oldState, StateData newState);
class Class1
{
ConcurrentDictionary<string, StateData> stateCollection = new ConcurrentDictionary<string, StateData>();
public StateData? GetState(string key)
{
StateData o;
if (stateCollection.TryGetValue(key, out o))
return o;
else
return null;
}
public StateChangedHandler StateUpdated;
void UpdateState(string key, StateData o)
{
StateData? prev = null;
stateCollection.AddOrUpdate(key, o,
(id, old) =>
{
prev = old;
return o;
}
);
if (prev != null && StateUpdated != null)
StateUpdated(prev.Value, o);
}
}
Thanks for your answers.

However, what to do if state is a structure? Is a complete 'lock' the only way?
The volatile keyword is applicable only to types that can be updated atomically, such as reference type variables, pointers, and primitive types.
However, note that volatile provides some guarantees besides just access to the marked variable. In particular, all writes to memory that occur before a write to a volatile-marked memory location will be seen by any code that reads from memory after reading from that same volatile-marked memory location.
In theory, this means you can use a volatile field to provide volatile-like behavior for other memory locations.
In practice though, there is still a problem: new writes to the same memory locations may or may not be visible, and of course not all writes can be completed atomically. So this use of volatile is really only good for other memory locations that could be marked volatile, and even then doesn't ensure you won't get newer values than the volatile-marked memory location would otherwise indicate.
Which is a long way of saying, you should just use lock and be done with it. :)
Side question: can a variable still be cached inside the lock?
I'm not entirely sure what you mean by this question. But in general: the compilers are permitted to make optimizations as long as those optimizations don't affect the observed behavior of the code in a single thread. If the type of caching you are thinking of would not violate this rule, it might be allowed. Otherwise it wouldn't be.
will this code be sufficient for managing a collection of state objects in multi-threading environment?
The code as posted seems fine, at least as far as ensuring that the dictionary is always providing coherent state values.

volatile ensures that you get the latest value of a variable from any thread. This works only with primitive types such as int, short etc. and only with value-types.
A volatile reference type will only ensure that you get the latest reference of the declaration and not the value of where it points to.
You should only use volatile for immutable data types.
Some thread-safe ways of juggling with collections are: using lock, Mutex, and one of the latest goodies of .NET 4.5 ConcurrentCollections (which unfortunately proved to be slower than the generic bag-lock combo).
If performance and freedom is what you want go with lock
If you are managing a very extensive application go with Mutex
Depending on how long (&how many) operations take place in a collection you could go with ConcurrentCollections
Here's a nice comparison between locks and thread-safe collections.

Making Guid properties threadsafe

One of my classes has a property of type Guid. This property can read and written simultaneously by more than one thread. I'm under the impression that reads and writes to a Guid are NOT atomic, therefore I should lock them.
I've chosen to do it like this:
public Guid TestKey
{
get
{
lock (_testKeyLock)
{
return _testKey;
}
}
set
{
lock (_testKeyLock)
{
_testKey = value;
}
}
}
(Inside my class, all access to the Guid is also done through that property rather than accessing _testKey directly.)
I have two questions:
(1) Is it really necessary to lock the Guid like that to prevent torn reads? (I'm pretty sure it is.)
(2) Is that a reasonable way to do the locking? Or do I need to do it like the following:
get
{
Guid result;
lock (_testKeyLock)
{
result = _testKey;
}
return result;
}
[EDIT] This article does confirm that Guids will suffer from torn reads: http://msdn.microsoft.com/en-us/magazine/jj863136.aspx

1: yes; to protect from torn values if you have one thread reading and one writing; Guid is not guaranteed to be atomic
2: "like the following": they are effectively the same; at the IL level you cannot ret from a try/catch block, so the compiler implements your first example by introducing a local variable, just like in your second example.
Another approach might be to box it; a reference is atomic:
object boxedTestKey;
public Guid TestKey
{
get { return (Guid)boxedTestKey; }
set { boxedTestKey = value; }
}
No locking required, but a small overhead from the box.

1) Is it really necessary to lock the Guid like that to prevent torn reads? (I'm pretty sure it is.)
Yes it is.
2) Is that a reasonable way to do the locking?
Again: Yes.
If there had existed an Interlocked method for Guid then that would have been better (faster).
For double (another non-atomic struct) there is support from Interlocked and for references it is not needed.
So as a pattern this is only required for larger structs that are not supported by Interlocked.

locking the object inside a property, c#

public ArrayList InputBuffer
{
get { lock (this.in_buffer) { return this.in_buffer; } }
}
is this.in_buffer locked during a call to InputBuffer.Clear?
or does the property simply lock the in_buffer object while it's getting the reference to it; the lock exits, and then that reference is used to Clear?

No, the property locks the reference while it's getting that reference. Pretty pointless, to be honest... this is more common:
private readonly object mutex = new object();
private Foo foo = ...;
public Foo Foo
{
get
{
lock(mutex)
{
return foo;
}
}
}
That lock would only cover the property access itself, and wouldn't provide any protection for operations performed with the Foo. However, it's not the same as not having the lock at all, because so long as the variable is only written while holding the same lock, it ensures that any time you read the Foo property, you're accessing the most recent value of the property... without the lock, there's no memory barrier and you could get a "stale" result.
This is pretty weak, but worth knowing about.
Personally I try to make very few types thread-safe, and those tend to have more appropriate operations... but if you wanted to write code which did modify and read properties from multiple threads, this is one way of doing so. Using volatile can help too, but the semantics of it are hideously subtle.

The object is locked inside the braces of the lock call, and then it is unlocked.
In this case the only code in the lock call is return this.in_buffer;.
So in this case, the in_buffer is not locked during a call to InputBuffer.Clear.
One solution to your problem, using extension methods, is as follows.
private readonly object _bufLock;
class EMClass{
public static void LockedClear(this ArrayList a){
lock(_bufLock){
a.Clear();
}
}
}
Now when you do:
a.LockedClear();
The Clear call will be done in a lock.
You must ensure that the buffer is only accessed inside _bufLocks.

In addition to what others have said about the scope of the lock, remember that you aren't locking the object, you are only locking based on the object instance named.
Common practice is to have a separate lock mutex as Jon Skeet exemplifies.
If you must guarantee synchronized execution while the collection is being cleared, expose a method that clears the collection, have clients call that, and don't expose your underlying implementation details. (Which is good practice anyway - look up encapsulation.)

Making locking easier in MTAs

In multi-threaded code, when an instance may be read or written by multiple threads, they need to be locked on to perform these operations safely.
To avoid the repetition of creating an object to lock on and writing a bunch of lock statements through code, I've created a generic class to handle the locking.
Am I missing anything, conceptually? This should work, right?
public class Locked<T> where T : new()
{
private readonly object locker = new object();
private T value;
public Locked()
: this(default(T))
{ }
public Locked(T value)
{
this.value = value;
}
public T Get()
{
lock (this.locker)
{
return this.value;
}
}
public void Set(T value)
{
lock (this.locker)
{
this.value = value;
}
}
}
And an example of it being used in a class:
private Locked<bool> stopWorkerThread = new Locked<bool>();
public void WorkerThreadEntryPoint()
{
while (true)
{
if (this.stopWorkerThread.Get())
{
break;
}
Also, how would I test something like this, in an automated way (e.g. create a unit test)?
Lastly, what can I do to implement a ++ and -- operator, to avoid this:
this.runningThreads.Set(this.runningThreads.Get() + 1);

That only locks for the duration of the get/set; of course, in many common cases this will be atomic anyway, simply due to to data size.
However, in reality most locks need to span more than this, in the same way that collections locking over just the Add etc don't help much - a caller typically needs a single lock to span the "is it there? if so update, else add" sequence.
For something as simple as a bool, "volatile" might solve the problem a lot more simply - especially if it is just for a loop exit.
You might also want to consider [MethodImpl(MethodImplOptions.Synchronized)] - although personally I prefer a private lock object (like you have used) to prevent issues with external people locking on the object (the above uses "this" as the lock).
For unit testing this, you'd need something to prove it broken first - which would be hard since the operations are so small (and already atomic for most data types). One of the other things it avoids (that volatile also fixes) is caching in a register, but again that is an optimisation and hard to force to prove it is broken.
If you are interested in a lock-wrapper, you might consider existing code like this.

Your code above has quite a few potential and real multi-threading issues, and I wouldn't use something like it in a real-world situation. For example:
this.runningThreads.Set(this.runningThreads.Get() + 1);
There is a pretty obvious race condition here. When the Get() call returns, the object is no longer locked. To do a real post or pre increment, the counter would need to be locked from before the Get to after the Set.
Also you don't always need to do a full lock if all you are doing is synchronous reads.
A better lock interface would (I think) require you to explicitly lock the instance where you need to do it. My experience is mainly with C++ so I can't recommend a full implementation, but my preferred syntax might look something like this:
using (Locked<T> lock = Locked<T>(instance))
{
// write value
instance++;
}
// read value
print instance;