I have a Queue<T> field that is accessed by various threads. Enequeue() is called from multiple threads many times per second, while there is a single thread that performs the Dequeue() and Count operations.
I havent been thinking much about this until now, since I played it "safe" and used lock on a static object before any operations with this queue. While there currently aren't any performance issue, I would like to get rid of the locks if they are redundant. My questions are:
since I never iterate through the queue, are locks really needed in this situation? I mean, will the program crash when it happens that one thread enqueues and the second thread dequeues elements at exactly the same time?
should I perhaps use Queue.Synchronized() to get a wrapper, and if so: will that impact performance compared to the original queue?
1: yes they are necessary; both enqueue and dequeue mutate state; a standard queue is not thread safe
2: ConcurrentQueue<T> would work nicely; personally I use a version I wrote here on SO, Creating a blocking Queue<T> in .NET? - it makes it easy to throttle the size of the queue, and do efficient dequeues without looping
Note; with your current implementation the lock-object should only be static if the queue is static (that isn't clear in the question, though) - otherwise all your similar queues may be sharing a lock
Related
I've got an application where there are several threads that provide data, that needs to go through some heavy math. The math part needs a lot of initialization, afterwards it's pretty fast - as such I can't just spawn a thread every time I need to do the calculation, nor should every source thread have its own solver (there can be a LOT of such threads, beyond a certain point the memory requirements are obscene, and the overhead gets in the way or processing power).
I would like to use a following model: The data gathering and using threads would call to a single object, through one thread-safe interface function, like
public OutData DoMath(InData data) {...}
that would take care of the rest. This would involve finding a free worker thread (or waiting and blocking till one is available) passing by some means the data in a thread safe manner to one of the free worker threads, waiting (blocking) for it to do its job and gathering the result and returning it.
The worker thread(s) would then go into some sleep/blocked state, until a new input item would appear on its interface (or a command to clean up and die).
I know how to do this by means of various convoluted locks, queues and waits in a very horrible nasty way. I'm guessing there's a better, more elegant way.
My questions are:
Is this a good architecture for this?
Are there commonly used elegant means of doing this?
The target framework is .NET 4.5 or higher.
Thank you,
David
The math part needs a lot of initialization, afterwards it's pretty fast - as such I can't just spawn a thread every time I need to do the calculation, nor should every source thread have its own solver (there can be a LOT of such threads, beyond a certain point the memory requirements are obscene, and the overhead gets in the way or processing power).
Sounds like a pool of lazy-initialized items. You can use a basic BlockingCollection for this, but I recommend overriding the default queue-like behavior with a stack-like behavior to avoid initializing contexts you may not ever need.
I'll call the expensive-to-initialize type MathContext:
private static readonly BlockingColleciton<Lazy<MathContext>> Pool;
static Constructor()
{
Pool = new BlockingCollection<Lazy<MathContext>>(new ConcurrentStack<Lazy<MathContext>>());
for (int i = 0; i != 100; ++i) // or whatever you want your upper limit to be
Pool.Add(new Lazy<MathContext>());
}
This would involve finding a free worker thread (or waiting and blocking till one is available)
Actually, there's no point in using a worker thread here. Since your interface is synchronous, the calling thread can just do the work itself.
OutData DoMath(InData data)
{
// First, take a context from the pool.
var lazyContext = Pool.Take();
try
{
// Initialize the context if necessary.
var context = lazyContext.Value;
return ... // Do the actual work.
}
finally
{
// Ensure the context is returned to the pool.
Pool.Add(lazyContext);
}
}
I also think you should check out the TPL Dataflow library. It would require a bit of code restructuring, but it sounds like it may be a good fit for your problem domain.
Investigate Task Parallel Library. It has a set of methods for creating and managing threads. And such classes as ReaderWriterLock, ManualResetEvent
and their derivatives may help in synchronizing threads
Don't use locks. This problem sounds nice for a proper nearly lock free approach.
I think what you need to look into is the BlockingCollection. This class is a powerful collection for multiple consumers and producers. If you think about using it with Parallel.ForEach you may want to look into writing your own Partitioner to get some more performance out of it. Parallel contains a couple of very nice methods if you only need a couple of threads for a relatively short time. That sounds like something you need to do. There are also overloads that provide initialization and finalization methods for each spawned thread along with passing thread local variables from one stage of the function to the next. That may really help you.
The general tips apply here of cause too. Try to split up your application in as may small parts as possible. That usually clears things up nicely and the ways how to do things become clearer.
All in all from what you told about the problem at hand I do not think that you need a lot of blocking synchronization. The BlockingCollection is only blocking the consumer threads until new data is ready to be consumed. And the producer if you limit the size...
I can't think of anything beyond that out of the top of my head. This is a very general question and without some specific issues it is hard to help beyond that.
I still hope that helps.
You've pretty much described a thread pool - fortunately, there's quite a few simple APIs you can use for that. The simplest is probably
await Task.Run(() => DoMath(inData));
or just call Task.Run(() => DoMath(inData)).GetAwaiter().GetResult() if you don't mind blocking the requesting thread.
Instead of starting a whole new thread, it will simply borrow a thread from the .NET thread pool for the computation, and then return the result. Since you're doing almost pure CPU work, the thread pool will have only as much threads as you really need (that is, about the same (or double) amount as the number of CPU cores you have).
Using the await based version is a bit trickier - you need to ensure your whole call chain returns Tasks - but it has a major advantage in avoiding the need to keep the calling thread alive while you wait for the results to be done. And even better, if you make sure the original thread is also a thread-pool thread, you don't even need the Task.Run - the threads will be balanced automatically. Since you're only doing synchronous work anyway, this turns your whole problem into simply avoiding any manual new Thread, and using Task.Run(...) instead.
First, create a pool of N such "math service objects" that are heavy. Then, guard usage of that pool with a new SemaphoreSlim(N, N). Accessing those objects is then as easy as:
SemaphoreSlim sem = ...;
//...
await sem.WaitAsync();
var obj = TakeFromPool();
DoWork(obj);
Return(obj);
sem.Release();
You can vary this pattern in many ways. The core of it is the pool plus a semaphore that can be used to wait if the pool is empty at the time.
I was wondering on the Monitor Class.
As far as i know all waiting threads are not FIFO.
The first one that aquires the lock is not allways the first on in the waiting queue.
Is this correct?
Is there some way to ensure the FIFO condition?
Regards
If you are referring to a built-in way, then no. Repeatedly calling TryEnter in a loop is by definition not fair and unfortunately neither is the simple Monitor.Enter. Technically a thread could wait forever without getting the lock.
If you want absolute fairness you will need to implement it yourself using a queue to keep track of arrival order.
Is there some way to ensure the FIFO condition?
In a word: no!
I wrote a short article about this: Is the Ready Queue FIFO?
Look at this question, I think it will very useful for you - Does lock() guarantee acquired in order requested?
especially this quote:
Because monitors use kernel objects internally, they exhibit the same
roughly-FIFO behavior that the OS synchronization mechanisms also
exhibit (described in the previous chapter). Monitors are unfair, so
if another thread tries to acquire the lock before an awakened waiting
thread tries to acquire the lock, the sneaky thread is permitted to
acquire a lock.
I need to implement a module that can have multiple inputs to a dictionary (multiple threads writing to a dictionary) and 1 timed consumer that takes this dictionary, sends it away using some ISender and clears the dictionary for a new bulk of data.
the problem is that i need to design my interlocks that way that the consuming thread takes the quickest snapshot of the bulk while allowing the producing threads to keep writing to a new cleared dictionary.
what is the best consumer producer design you would suggest using interlocks and ConcurrentDictionary?
Best Regards!
Don't let the producer threads put the data in the dictionary directly. Let them put it in some thread-safe queue, such as BlockingCollection. Your consumer thread can then take items from the queue, build the dictionary and send it away, all without blocking the producer threads.
Essentially the same work gets done, but is "spread around" in a way that avoids most of the blocking.
If you are extra-worried about contention on that single queue, you can even have a separate BlockingCollection per producer thread and then use BlockingCollection.TakeFromAny in the consumer.
The problem is, of course, if your consumer threads do anything other than simply writing to the dictionary. If they need, for example, to check if the given key already exists in the dictionary, then this design suddenly becomes much more complicated.
The fastest way I can think of is to use multiple dictionary objects.
When your consumer thread runs, it creates a new ConcurrentDictionary and sets it as the "live" dictionary. This is fast and means the producers can carry on with minimal interruption.
The consumer thread now "owns" the previous dictionary object and can process its contents in its own time.
I am writing a server application which processes request from multiple clients. For the processing of requests I am using the threadpool.
Some of these requests modify a database record, and I want to restrict the access to that specific record to one threadpool thread at a time. For this I am using named semaphores (other processes are also accessing these records).
For each new request that wants to modify a record, the thread should wait in line for its turn.
And this is where the question comes in:
As I don't want the threadpool to fill up with threads waiting for access to a record, I found the RegisterWaitForSingleObject method in the threadpool.
But when I read the documentation (MSDN) under the section Remarks:
New wait threads are created automatically when required. ...
Does this mean that the threadpool will fill up with wait-threads? And how does this affect the performance of the threadpool?
Any other suggestions to boost performance is more than welcome!
Thanks!
Your solution is a viable option. In the absence of more specific details I do not think I can offer other tangible options. However, let me try to illustrate why I think your current solution is, at the very least, based on sound theory.
Lets say you have 64 requests that came in simultaneously. It is reasonable to assume that the thread pool could dispatch each one of those requests to a thread immediately. So you might have 64 threads that immediately begin processing. Now lets assume that the mutex has already been acquired by another thread and it is held for a really long time. That means those 64 threads will be blocked for a long time waiting for the thread that currently owns the mutex to release it. That means those 64 threads are wasted on doing nothing.
On the other hand, if you choose to use RegisterWaitForSingleObject as opposed to using a blocking call to wait for the mutex to be released then you can immediately release those 64 waiting threads (work items) and allow them to be put back into the pool. If I were to implement my own version of RegisterWaitForSingleObject then I would use the WaitHandle.WaitAny method which allows me to specify up to 64 handles (I did not randomly choose 64 for the number of requests afterall) in a single blocking method call. I am not saying it would be easy, but I could replace my 64 waiting threads for only a single thread from the pool. I do not know how Microsoft implemented the RegisterWaitForSingleObject method, but I am guessing they did it in a manner that is at least as efficient as my strategy. To put this another way, you should be able to reduce the number of pending work items in the thread pool by at least a factor of 64 by using RegisterWaitForSingleObject.
So you see, your solution is based on sound theory. I am not saying that your solution is optimal, but I do believe your concern is unwarranted in regards to the specific question asked.
IMHO you should let the database do its own synchronization. All you need to do is to ensure that you're sync'ed within your process.
Interlocked class might be a premature optimization that is too complex to implement. I would recommend using higher-level sync objects, such as ReaderWriterLockSlim. Or better yet, a Monitor.
An approach to this problem that I've used before is to have the first thread that gets one of these work items be responsible for any other ones that occur while it's processing the work item(s), This is done by queueing the work items then dropping into a critical section to process the queue. Only the 'first' thread will drop into the critical section. If a thread can't get the critical section, it'll leave and let the thread already operating in the critical section handle the queued object.
It's really not very complicated - the only thing that might not be obvious is that when leaving the critical section, the processing thread has to do it in a way that doesn't potentially leave a late-arriving workitem on the queue. Basically, the 'processing' critical section lock has to be released while holding the queue lock. If not for this one requirement, a synchronized queue would be sufficient, and the code would really be simple!
Pseudo code:
// `workitem` is an object that contains the database modification request
//
// `queue` is a Queue<T> that can hold these workitem requests
//
// `processing_lock` is an object use to provide a lock
// to indicate a thread is processing the queue
// any number of threads can call this function, but only one
// will end up processing all the workitems.
//
// The other threads will simply drop the workitem in the queue
// and leave
void threadpoolHandleDatabaseUpdateRequest(workitem)
{
// put the workitem on a queue
Monitor.Enter(queue.SyncRoot);
queue.Enqueue(workitem);
Monitor.Exit(queue.SyncRoot);
bool doProcessing;
Monitor.TryEnter(processing_queue, doProcessing);
if (!doProcessing) {
// another thread has the processing lock, it'll
// handle the workitem
return;
}
for (;;) {
Monitor.Enter(queue.SyncRoot);
if (queue.Count() == 0) {
// done processing the queue
// release locks in an order that ensures
// a workitem won't get stranded on the queue
Monitor.Exit(processing_queue);
Monitor.Exit(queue.SyncRoot);
break;
}
workitem = queue.Dequeue();
Monitor.Exit(queue.SyncRoot);
// this will get the database mutex, do the update and release
// the database mutex
doDatabaseModification(workitem);
}
}
ThreadPool creates a wait thread for ~64 waitable objects.
Good comments are here: Thread.sleep vs Monitor.Wait vs RegisteredWaitHandle?
if there are two threads as producer/consumer is it good idea to have following line to prevent deadlocks. I'm aware of live locks but suppose they do a lot of work before calling this Wait() method:
// member variable
object _syncLock = new object();
void Wait()
{
lock (_syncLock)
{
Monitor.Pulse(_syncLock);
Monitor.Wait(_syncLock);
}
}
Here it's impossible both threads be in waiting state.
This seems overly complicated. Just handle your locking correctly in the first place, and avoid the issue. If you only have two threads, and they are trying to acquire the same, single lock (correctly), you shouldn't have deadlocks. A deadlock means there is something else occurring here.
That being said, if you have the option of using the TPL via .NET 4 (or the Rx Extensions on .NET 3.5), you might want to consider using BlockingCollection<T> instead. It's ideally suited to use in a producer/consumer scenario, and works in a lockless manner.
If your intention is to create a paired variant of the producer-consumer pattern then the sequence is Pulse before Wait for the producer and Wait before Pulse for the consumer. You can reference figure 5 in Joe Duffy's article on this. Howerver, keep in mind that since his implementation performs an unconditional Wait in the Enqueue method a ping-pong like effect will occur between the producer and the consumer. The queue, in his implementation, can only ever have one item per producer. So if that is your intention then this your ticket. Otherwise, you can adapt it as-is and apply some condition1 to the Wait in the Enqueue method to make it behave more like a real FIFO buffer.
However, like Reed, I question why BlockingCollection could not be used. This collection should be very efficient since it uses a lock-free strategy for the Add and Take methods. Of course, like I mentioned above, if you really want the paired variant then this collection will not meet your requirements and you will have to roll your own using Joe Duffy's as a starting point.
1Just remember to use a while loop instead of an if check before applying the wait. Monitor.Wait simply waits for a change in the lock state and nothing more so you have to recheck the wait condition.