I need to have the piece of code which allowed to execute only by 1 thread at the same time based on parameter key:
private static readonly ConcurrentDictionary<string, SemaphoreSlim> Semaphores = new();
private async Task<TModel> GetValueWithBlockAsync<TModel>(string valueKey, Func<Task<TModel>> valueAction)
{
var semaphore = Semaphores.GetOrAdd(valueKey, s => new SemaphoreSlim(1, 1));
try
{
await semaphore.WaitAsync();
return await valueAction();
}
finally
{
semaphore.Release(); // Exception here - System.ObjectDisposedException
if (semaphore.CurrentCount > 0 && Semaphores.TryRemove(valueKey, out semaphore))
{
semaphore?.Dispose();
}
}
}
Time to time I got the error:
The semaphore has been disposed. : System.ObjectDisposedException: The semaphore has been disposed.
at System.Threading.SemaphoreSlim.CheckDispose()
at System.Threading.SemaphoreSlim.Release(Int32 releaseCount)
at Project.GetValueWithBlockAsync[TModel](String valueKey, Func`1 valueAction)
All cases that I can imagine here are thread safety. Please help, what case I missed?
You have a thread race here, where another task is trying to acquire the same semaphore, and acquires it when you Release - i.e. another thread is awaiting the semaphore.WaitAsync(). The check against CurrentCount is a race condition, and it could go either way depending on timing. The check for TryRemove is irrelevant, as the competing thread already got the semaphore out - it was, after all, awaiting the WaitAsync().
As discussed in the comments, you have a couple of race conditions here.
Thread 1 holds the lock and Thread 2 is waiting on WaitAsync(). Thread 1 releases the lock, and then checks semaphore.CurrentCount, before Thread 2 is able to acquire it.
Thread 1 holds the lock, releases it, and checks semaphore.CurrentCount which passes. Thread 2 enters GetValueWithBlockAsync, calls Semaphores.GetOrAdd and fetches the semaphore. Thread 1 then calls Semaphores.TryRemove and diposes the semaphore.
You really need locking around the decision to remove an entry from Semaphores -- there's no way around this. You also don't have a way of tracking whether any threads have fetched a semaphore from Semaphores (and are either currently waiting on it, or haven't yet got to that point).
One way is to do something like this: have a lock which is shared between everyone, but which is only needed when fetching/creating a semaphore, and deciding whether to dispose it. We manually keep track of how many threads currently have an interest in a particular semaphore. When a thread has released the semaphore, it then acquires the shared lock to check whether anyone else currently has an interest in that semaphore, and disposes it only if noone else does.
private static readonly object semaphoresLock = new();
private static readonly Dictionary<string, State> semaphores = new();
private async Task<TModel> GetValueWithBlockAsync<TModel>(string valueKey, Func<Task<TModel>> valueAction)
{
State state;
lock (semaphoresLock)
{
if (!semaphores.TryGetValue(valueKey, out state))
{
state = new();
semaphores[valueKey] = state;
}
state.Count++;
}
try
{
await state.Semaphore.WaitAsync();
return await valueAction();
}
finally
{
state.Semaphore.Release();
lock (semaphoresLock)
{
state.Count--;
if (state.Count == 0)
{
semaphores.Remove(valueKey);
state.Semaphore.Dispose();
}
}
}
}
private class State
{
public int Count { get; set; }
public SemaphoreSlim Semaphore { get; } = new(1, 1);
}
The other option, of course, is to let Semaphores grow. Maybe you have a periodic operation to go through and clear out anything which isn't being used, but this will of course need to be protected to ensure that a thread doesn't suddenly become interested in a semaphore which is being cleared up.
Related
I have a Thread (STAThread) in a Windows Service, which performs a big amount of work. When the windows service is restarted I want to stop this thread gracefully.
I know of a couple of ways
A volatile boolean
ManualResetEvent
CancellationToken
As far as I have found out Thread.Abort is a no go...
What is the best practice ?
The work is perfomed in another class than the one where the thread is started, so it is necessary to either introduce a cancellationToken parameter in a constructor or for example have a volatile variable. But I just can't figure out what is smartest.
Update
Just to clarify a little I have wrapped up a very simple example of what I'm talking about. As said earlier, this is being done in a windows service. Right now I'm thinking a volatile boolean that is checked on in the loop or a cancellationToken....
I cannot wait for the loop to finish, as stated below it can take several minutes, making the system administrators of the server believe that something is wrong with the service when they need to restart it.... I can without problems just drop all the work within the loop without problems, however I cannot do this with a Thread.Abort it is "evil" and furthermore a COM interface is called, so a small clean up is needed.
Class Scheduler{
private Thread apartmentThread;
private Worker worker;
void Scheduling(){
worker = new Worker();
apartmentThread = new Thread(Run);
apartmentThread.SetApartmentState(ApartmentState.STA);
apartmentThread.Start();
}
private void Run() {
while (!token.IsCancellationRequested) {
Thread.Sleep(pollInterval * MillisecondsToSeconds);
if (!token.IsCancellationRequested) {
worker.DoWork();
}
}
}
}
Class Worker{
//This will take several minutes....
public void DoWork(){
for(int i = 0; i < 50000; i++){
//Do some work including communication with a COM interface
//Communication with COM interface doesn't take long
}
}
}
UPDATE
Just examined performance, using a cancellationToken where the isCancelled state is "examined" in the code, is much faster than using a waitOne on a ManualResetEventSlim. Some quick figuers, an if on the cancellationToken iterating 100.000.000 times in a for loop costs me approx. 500 ms, where the WaitOne costs approx. 3 seconds. So performance in this scenario it is faster to use the cancellationToken.
You haven't posted enough of your implementation but I would highly recommend a CancellationToken if that is available to you. It's simple enough to use and understand from a maintainability standpoint. You can setup cooperative cancellation as well too if you decide to have more than one worker thread.
If you find yourself in a situation where this thread may block for long periods of time, it's best to setup your architecture so that this doesn't occur. You shouldn't be starting threads that won't play nice when you tell them to stop. If they don't stop when you ask them, the only real way is to tear down the process and let the OS kill them.
Eric Lippert posted a fantastic answer to a somewhat-related question here.
I tend to use a bool flag, a lock object and a Terminate() method, such as:
object locker = new object();
bool do_term = false;
Thread thread = new Thread(ThreadStart(ThreadProc));
thread.Start();
void ThreadProc()
{
while (true) {
lock (locker) {
if (do_term) break;
}
... do work...
}
}
void Terminate()
{
lock (locker) {
do_term = true;
}
}
Asides from Terminate() all the other fields and methods are private to the "worker" class.
Use a WaitHandle, most preferably a ManualResetEvent. Your best bet is to let whatever is in your loop finish. This is the safest way to accomplish your goal.
ManualResetEvent _stopSignal = new ManualResetEvent(false); // Your "stopper"
ManualResetEvent _exitedSignal = new ManualResetEvent(false);
void DoProcessing() {
try {
while (!_stopSignal.WaitOne(0)) {
DoSomething();
}
}
finally {
_exitedSignal.Set();
}
}
void DoSomething() {
//Some work goes here
}
public void Terminate() {
_stopSignal.Set();
_exitedSignal.WaitOne();
}
Then to use it:
Thread thread = new Thread(() => { thing.DoProcessing(); });
thread.Start();
//Some time later...
thing.Terminate();
If you have a particularly long-running process in your "DoSomething" implementation, you may want to call that asynchronously, and provide it with state information. That can get pretty complicated, though -- better to just wait until your process is finished, then exit, if you are able.
There are two situations in which you may find your thread:
Processing.
Blocking.
In the case where your thread is processing something, you must wait for your thread to finish processing in order for it to safely exit. If it's part of a work loop, then you can use a boolean flag to terminate the loop.
In the case where your thread is blocking, then you need to wake your thread and get it processing again. A thread may be blocking on a ManualResetEvent, a database call, a socket call or whatever else you could block on. In order to wake it up, you must call the Thread.Interrupt() method which will raise a ThreadInterruptedException.
It may look something like this:
private object sync = new object():
private bool running = false;
private void Run()
{
running = true;
while(true)
{
try
{
lock(sync)
{
if(!running)
{
break;
}
}
BlockingFunction();
}
catch(ThreadInterruptedException)
{
break;
}
}
}
public void Stop()
{
lock(sync)
{
running = false;
}
}
And here is how you can use it:
MyRunner r = new MyRunner();
Thread t = new Thread(()=>
{
r.Run();
});
t.IsBackground = true;
t.Start();
// To stop the thread
r.Stop();
// Interrupt the thread if it's in a blocking state
t.Interrupt();
// Wait for the thread to exit
t.Join();
Suppose I have a Queue of Tasks, and each Task have a locking object (syncObject) which controls shared resource access, Queue can have multiple Task which share same instances of syncObject. And I have N concurrent threads that should dequeue Tasks and proccess them in queue order, this means acquire lock on syncObject in the order of queue.
Code explanation:
abstract class Task
{
public readonly Object SyncObject = new Object();
}
Queue<Task> taskQueue = new Queue<Task>();
Object queueLock = new Object();
void TakeItemThreadedMethod()
{
Task task;
lock(queueLock) task = taskQueue.Dequeue();
//Between this lines is my problem,
//Other thread can dequeue next task and it may share same syncObject and
//acquire lock on it before this thread, thought this task was first in queue
lock(task.SyncObject)
{
//Do some work here
}
}
How to start proccessing Tasks (acquire Task.SyncObject lock) that share the same SyncObject in the order they were in Queue.
It sounds like potentially your queue shouldn't contain individual tasks - but queues of tasks, where each subqueue is "all the tasks which share a sync-lock".
Your processor would therefore:
Take a subqueue off the main queue
Dequeue the first task off the subqueue and process it
When it's finished put the subqueue back at the end of the main queue (or anywhere, actually - work out how you want the scheduling to work)
This will ensure that only one task per subqueue is ever executed at a time.
You'll probably need a map from lock to subqueue, so that anything creating work can add it to the right subqueue. You'd need to atomically work out when to remove a subqueue from the map (and not put it back on the main queue), assuming you require that functionality at all.
EDIT: As an optimization for the above, you could put the subqueue itself into whatever you're using as the shared sync lock. It could have a reference to either "the single task to next execute" or "a queue of tasks" - only creating the queue lazily. You'd then put the sync lock (which wouldn't actually need to be used as a lock any more) on the queue, and each consumer would just ask it for the next task to execute. If only a single task is available, it's returned (and the "next task" variable set to null). If there are multiple tasks available, the first is dequeued.
When a producer adds a new task, either the "first task" variable is set to the task to execute if it was previously null, or a queue is created if there wasn't a queue but was already a task, or the queue is just added to if one already existed. That solves the inefficiency of unnecessary queue creation.
Again, the tricky part will be working out how to atomically throw away the shared resource lock - because you only want to do so after processing the last item, but equally you don't want to miss a task because you happened to add it at the wrong time. It shouldn't be too bad, but equally you'll need to think about it carefully.
How about this approach:
use a list instead of a queue
have each worker thread loop in order through the queue until it can find an "unlocked" task
Something like (untested):
abstract class Task
{
public readonly Object SyncObject = new Object();
}
List<Task> taskList = new List<Task>();
void TakeItemThreadedMethod()
{
Task task = null;
bool found = false;
try
{
// loop until found an task whose SyncObject is free
while (!found)
{
lock (taskList)
{
for (int i = 0; i < taskList.Count; i++)
{
object syncObj = taskList[i].SyncObject;
if (found = Monitor.TryEnter(syncObj))
{
for (int x = 0; x < taskList.Count; x++)
{
if (Object.ReferenceEquals(
syncObj, taskList[x].SyncObject))
{
task = taskList[x];
taskList.RemoveAt(x);
break;
}
}
break;
}
}
}
}
// process the task...
DoWork(task);
}
finally
{
if (found) Monitor.Exit(task.SyncObject);
}
}
void QueueTask(Task task)
{
lock (taskList)
{
taskList.Add(task);
}
}
I have used QueuedLock class suggested by Matthew Brindley, with slight modification, I have split up Enter function to TakeTicket and Enter which blocks.
Now I can use TakeTicket inside shared QueueLock without blocking whole queue.
Modified code:
abstract class Task
{
public readonly QueuedLock SyncObject = new QueuedLock();
}
Queue<Task> taskQueue = new Queue<Task>();
Object queueLock = new Object();
void TakeItemThreadedMethod()
{
Task task;
int ticket;
lock(queueLock)
{
task = taskQueue.Dequeue();
ticket = task.SyncObject.TakeTicket();
}
task.SyncObject.Enter(ticket);
//Do some work here
task.SyncObject.Exit();
}
I'm not quite sure how to word this, so I'll just paste my code and ask the question:
private void remoteAction_JobStatusUpdated(JobStatus status) {
lock (status) {
status.LastUpdatedTime = DateTime.Now;
doForEachClient(c => c.OnJobStatusUpdated(status));
OnJobStatusUpdated(status);
}
}
private void doForEachClient(Action<IRemoteClient> task) {
lock (clients) {
foreach (KeyValuePair<RemoteClientId, IRemoteClient> entry in clients) {
IRemoteClient clientProxy = entry.Value;
RemoteClientId clientId = entry.Key;
ThreadPool.QueueUserWorkItem(delegate {
try {
task(clientProxy);
#pragma warning disable 168
} catch (CommunicationException ex) {
#pragma warning restore 168
RemoveClient(clientId);
}
});
}
}
}
Assume that any other code which modifies the status object will acquire a lock on it first.
Since the status object is passed all the way through to multiple ThreadPool threads, and the call to ThreadPool.QueueUserWorkItem will complete before the actual tasks complete, am I ensuring that the same status object gets sent to all clients?
Put another way, when does the lock (status) statement "expire" or cause its lock to be released?
Locks don't expire. When a thread tries to pass the lock statement it can only do it if no other thread is executing inside a lock block having a lock on that particular object instance used in the lock statemement.
In your case it seems that you have a main thread executing. It will lock both the status and the clients instances before it spins of new tasks that are executed on seperate threads. If any code in the new threads want to acquire a lock on either status or clients it will have to wait until the main thread has released both locks by leaving both lock blocks. That happens when remoteAction_JobStatusUpdated returns.
You pass the status object to each worker thread and they are all free to do whatever they want to do with that object. The statement lock (status) in no way protects the status instance. However, if any of the threads tries to execute lock (status) they will block until the main thread releases the lock.
Using two separate object instances to lock can lead to deadlock. Assume one thread executes the following code:
lock (status) {
...
lock (clients) {
...
}
}
Another thread executes the following code where the locks are acquired in the reverse sequence:
lock (clients) {
...
lock (status) {
...
}
}
If the first thread manages to get the status first and the second the clients lock first they are deadlocked and both threads will no longer run.
In general I would advice you to encapsulate your shared state in a separate class and make access to it thread safe:
class State {
readonly Object locker = new Object();
public void ModifyState() {
lock (this.locker) {
...
}
}
public String AccessState() {
lock (this.locker) {
...
return ...
}
}
}
You can also mark you methods with the [MethodImpl(MethodImpl.Synchronized)] attribute, but it has its pitfalls as it will surround the method with a lock (this) which in general isn't recommended.
If you want to better understand what is going on behind the scenes of the lock statement you can read the Safe Thread Synchronization article in MSDN Magazine.
The locks certainly don't "expire" on their own, the lock will be valid until the closing brace of the lock(..){} statement.
I work with new Parallel.For that creates multiple threads to perform the same operation.
In case one of the threads fail, it means that I'm working "too fast" and I need to put all the threads to rest for a few seconds.
Is there a way to perform something like Thread.Sleep - only to do the same on all threads at once?
This is a direct answer to the question, except for the Parallel.For bit.
It really is a horrible pattern; you should probably be using a proper synchronization mechanism, and get the worker threads to, without preemption, occasionally check if they need to 'back off.'
In addition, this uses Thread.Suspend and Thread.Resume which are both deprecated, and with good reason (from Thread.Suspend):
"Do not use the Suspend and Resume methods to synchronize the activities of threads. You have no way of knowing what code a thread is executing when you suspend it. If you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked. Deadlocks can occur very easily."
(Untested)
public class Worker
{
private readonly Thread[] _threads;
private readonly object _locker = new object();
private readonly TimeSpan _tooFastSuspensionSpan;
private DateTime _lastSuspensionTime;
public Worker(int numThreads, TimeSpan tooFastSuspensionSpan)
{
_tooFastSuspensionSpan = tooFastSuspensionSpan;
_threads = Enumerable.Repeat(new ThreadStart(DoWork), numThreads)
.Select(ts => new Thread(ts))
.ToArray();
}
public void Run()
{
foreach (var thread in _threads)
{
thread.Start();
}
}
private void DoWork()
{
while (!IsWorkComplete())
{
try
{
// Do work here
}
catch (TooFastException)
{
SuspendAll();
}
}
}
private void SuspendAll()
{
lock (_locker)
{
// We don't want N near-simultaneous failures causing a sleep-duration of N * _tooFastSuspensionSpan
// 1 second is arbitrary. We can't be deterministic about it since we are forcefully suspending threads
var now = DateTime.Now;
if (now.Subtract(_lastSuspensionTime) < _tooFastSuspensionSpan + TimeSpan.FromSeconds(1))
return;
_lastSuspensionTime = now;
var otherThreads = _threads.Where(t => t.ManagedThreadId != Thread.CurrentThread.ManagedThreadId).ToArray();
foreach (var otherThread in otherThreads)
otherThread.Suspend();
Thread.Sleep(_tooFastSuspensionSpan);
foreach (var otherThread in otherThreads)
otherThread.Resume();
}
}
}
You need to create an inventory of your worker threads and then perhaps you can use Thread.Suspend and Resume methods. Mind you that using Suspend can be dangerous (for example, thread may have acquired lock before suspending). And suspend/resume have been marked obsolate due to such issues.
I have scenarios where I need a main thread to wait until every one of a set of possible more than 64 threads have completed their work, and for that I wrote the following helper utility, (to avoid the 64 waithandle limit on WaitHandle.WaitAll())
public static void WaitAll(WaitHandle[] handles)
{
if (handles == null)
throw new ArgumentNullException("handles",
"WaitHandle[] handles was null");
foreach (WaitHandle wh in handles) wh.WaitOne();
}
With this utility method, however, each waithandle is only examined after every preceding one in the array has been signalled... so it is in effect synchronous, and will not work if the waithandles are autoResetEvent wait handles (which clear as soon as a waiting thread has been released)
To fix this issue I am considering changing this code to the following, but would like others to check and see if it looks like it will work, or if anyone sees any issues with it, or can suggest a better way ...
Thanks in advance:
public static void WaitAllParallel(WaitHandle[] handles)
{
if (handles == null)
throw new ArgumentNullException("handles",
"WaitHandle[] handles was null");
int actThreadCount = handles.Length;
object locker = new object();
foreach (WaitHandle wh in handles)
{
WaitHandle qwH = wh;
ThreadPool.QueueUserWorkItem(
delegate
{
try { qwH.WaitOne(); }
finally { lock(locker) --actThreadCount; }
});
}
while (actThreadCount > 0) Thread.Sleep(80);
}
If you know how many threads you have, you can use an interlocked decrement. This is how I usually do it:
{
eventDone = new AutoResetEvent();
totalCount = 128;
for(0...128) {ThreadPool.QueueUserWorkItem(ThreadWorker, ...);}
}
void ThreadWorker(object state)
try
{
... work and more work
}
finally
{
int runningCount = Interlocked.Decrement(ref totalCount);
if (0 == runningCount)
{
// This is the last thread, notify the waiters
eventDone.Set();
}
}
Actually, most times I don't even signal but instead invoke a callback continues the processing from where the waiter would continue. Less blocked threads, more scalability.
I know is different and may not apply to your case (eg. for sure will not work if some of thoe handles are not threads, but I/O or events), but it may worth thinking about this.
I'm not sure what exactly you're trying to do, but would a CountdownEvent (.NET 4.0) conceptually solve your problem?
I'm not a C# or .NET programmer, but you could use a semaphore that is posted when one of your worker threads exits. The monitoring thread would simply wait on the semaphore n times where n is the number of worker threads. Semaphores are traditionally used to count resources in use but they can be used to count jobs completed by waiting on the same semaphore for n times.
When working with lots of simultaneous threads, I prefer to add each thread's ManagedThreadId into a Dictionary when I start the thread, and then have each thread invoke a callback routine that removes the dying thread's id from the Dictionary. The Dictionary's Count property tells you how many threads are active. Use the value side of the key/value pair to hold info that your UI thread can use to report status. Wrap the Dictionary with a lock to keep things safe.
ThreadPool.QueueUserWorkItem(o =>
{
try
{
using (var h = (o as WaitHandle))
{
if (!h.WaitOne(100000))
{
// Alert main thread of the timeout
}
}
}
finally
{
Interlocked.Decrement(ref actThreadCount);
}
}, wh);