I have some code that I borrowed from Steve Marx. The main block is used in an azure worker role thread to take out a lease on an azure blob. This provides a locking mechanism for synchronizing across multiple worker instances when you only want one instance to process a job at a time. However since you might have jobs that will take longer to complete than a blob lease timeout, a new thread is spawned to renew the blob lease every so often.
This renewal thread sleeps and renews on an infinite loop. When the main thread exits (via Dispose in the class' consumer), renewalThread.Abort() is invoked. This causes all kinds of ThreadAbortExceptions to be thrown in the worker role.
I'm wondering, is this a better way to handle this? What I don't like about it is that you can have several renewal threads that remain asleep after the consumer that spawned them has been disposed. Is there anything bad about the code below? If so, is there a better way? Or is Thread.Abort() appropriate here?
public class AutoRenewLease : IDisposable
{
private readonly CloudBlockBlob _blob;
public readonly string LeaseId;
private Thread _renewalThread;
private volatile bool _isRenewing = true;
private bool _disposed;
public bool HasLease { get { return LeaseId != null; } }
public AutoRenewLease(CloudBlockBlob blob)
{
_blob = blob;
// acquire lease
LeaseId = blob.TryAcquireLease(TimeSpan.FromSeconds(60));
if (!HasLease) return;
// keep renewing lease
_renewalThread = new Thread(() =>
{
try
{
while (_isRenewing)
{
Thread.Sleep(TimeSpan.FromSeconds(40.0));
if (_isRenewing)
blob.RenewLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
}
}
catch { }
});
_renewalThread.Start();
}
~AutoRenewLease()
{
Dispose(false);
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (_disposed) return;
if (disposing && _renewalThread != null)
{
//_renewalThread.Abort();
_isRenewing = false;
_blob.ReleaseLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
_renewalThread = null;
}
_disposed = true;
}
}
Update
Let's say you have an azure worker role deployed with 2 or more instances. Let's also say you have a job that both instances have the ability to process for you. During the worker roles Run method you might have something like this:
public override void Run()
{
while (true)
{
foreach (var task in _workforce)
{
var job = task.Key;
var workers = task.Value;
foreach (var worker in workers)
worker.Perform((dynamic)job);
}
Thread.Sleep(1000);
}
}
Every second, the role will check to see if certain jobs are scheduled to run, and if they are, process them. However to avoid having both role instances process the same job, you first take out a lease on a blob. By doing that, the other instance cannot access the blob, so it is effectively blocked until the first instance is finished processing. (Note: taking out a new lease happens within the .Perform method above.)
Now, let's say a job can take anywhere from 1 to 100 seconds to complete. There is a built-in timeout on blob leases, so if you want to keep the other role blocked until the process is finished, you have to periodically renew that lease, to keep it form timing out. That is what the above class encapsulates -- automatically renewing a lease until you dispose of it as a consumer.
My question is mainly about the sleep timeout in the renewalThread. Say the job completed in 2 seconds. The renewalThread will gracefully exit (I think) but not for another 38 seconds. That is where the meat of uncertainty in my question lies. The original code invoked renewalThread.Abort(), which caused it to cease immediately. Is it better to do that, or let it sleep and exit gracefully at a later time? If you are heartbeating the role's Run method at once per second, you could have up to 40 of these renewal threads waiting to exit gracefully. If you have different jobs blocking on different blobs, that number gets multiplied by the number of blobs being leased. However if you do it with Thread.Abort(), you get just as many ThreadAbortExceptions sizzling on the stack.
As I understand it, you have a job that requires a lease on some object. That lease can expire, so you want something to continually renew the lease as long as the job is running.
You don't need a thread in a sleep loop. You need a timer. For example:
public class AutoRenewLease : IDisposable
{
private readonly CloudBlockBlob _blob;
public readonly string LeaseId;
private System.Threading.Timer _renewalTimer;
private volatile bool _isRenewing = true;
private bool _disposed;
public bool HasLease { get { return LeaseId != null; } }
public AutoRenewLease(CloudBlockBlob blob)
{
_blob = blob;
// acquire lease
LeaseId = blob.TryAcquireLease(TimeSpan.FromSeconds(60));
if (!HasLease) return;
_renewalTimer = new System.Threading.Timer(x =>
{
if (_isRenewing)
{
blob.RenewLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
}
}, null, TimeSpan.FromSeconds(40), TimeSpan.FromSeconds(40));
~AutoRenewLease()
{
Dispose(false);
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
if (_disposed) return;
if (disposing && _renewalTimer != null)
{
_isRenewing = false;
_renewalTimer.Dispose();
_blob.ReleaseLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
_renewalTimer = null;
}
_disposed = true;
}
}
There's no need to waste the resources that a thread uses just so that it can sleep most of the time. Using a timer eliminates polling and also eliminates the need for Thread.Abort.
Abort should be avoided whenever possible. There are some places you really need it, but for this scenario I think we can do it better without abort.
Make it simple with ManualResetEvent, This will stop your thread gracefully and immediately without the use of Abort.
private ManualResetEvent jobSignal = new ManualResetEvent(false);
public AutoRenewLease(CloudBlockBlob blob)
{
_blob = blob;
// acquire lease
LeaseId = blob.TryAcquireLease(TimeSpan.FromSeconds(60));
if (!HasLease) return;
// keep renewing lease
_renewalThread = new Thread(() =>
{
try
{
while (_isRenewing)
{
if(jobSignal.WaitOne(TimeSpan.FromSeconds(40.0)))
{
//Disposed so stop working
jobSignal.Dispose();
jobSignal = null;
return;
}
if (_isRenewing)
blob.RenewLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
}
}
catch (Exception ex) {//atleast log it }
});
_renewalThread.Start();
}
protected virtual void Dispose(bool disposing)
{
if (_disposed) return;
if (disposing && _renewalThread != null)
{
jobSignal.Set();//Signal the thread to stop working
_isRenewing = false;
_blob.ReleaseLease(AccessCondition
.GenerateLeaseCondition(LeaseId));
_renewalThread = null;
}
_disposed = true;
}
Hope this helps.
Related
Is this possible to lock method for one thread and force another to go futher rather than waiting until first thread finish? Can this problem be resolved with static thread or some proper pattern with one instance of mendtioned below service.
For presentation purposes, it can be done with static boolen like below.
public class SomeService
{
private readonly IRepository _repo;
public SomeService(IRepository repo)
{
_repo = repo;
}
private Thread threadOne;
public static bool isLocked { get; set; }
public void StartSomeMethod()
{
if(!isLocked)
{
threadOne = new Thread(SomeMethod);
isLocked = true;
}
}
public void SomeMethod()
{
while(true)
{
lots of time
}
...
isLocked = false;
}
}
I want to avoid situation when user clicked, by accident, two times to start and accidentailly second thread starts immediatelly after first finished.
You can use lock :)
object locker = new object();
void MethodToLockForAThread()
{
lock(locker)
{
//put method body here
}
}
Now the result will be that when this method is called by a thread (any thread) it puts something like flag at the beginning of lock: "STOP! You are not allowed to go any further, you must wait!" Like red light on crossroads.
When thread that called this method first, levaes the scope, then at the beginning of the scope this "red light" changes into green.
If you want to not call the method when it is already called by another thread, the only way to do this is by using bool value. For example:
object locker = new object();
bool canAccess = true;
void MethodToLockForAThread()
{
if(!canAccess)
return;
lock(locker)
{
if(!canAccess)
return;
canAccess = false;
//put method body here
canAccess = true;
}
}
Other check of canAccess in lock scope is because of what has been told on comments. No it's really thread safe. This is kind of protection that is advisible in thread safe singleton.
EDIT
After some discussion with mjwills I have to change my mind and turn more into Monitor.TryEnter. You can use it like that:
object locker = new object();
void ThreadMethod()
{
if(Monitor.TryEnter(locker, TimeSpan.FromMiliseconds(1))
{
try
{
//do the thread code
}
finally
{
Monitor.Exit(locker);
}
} else
return; //means that the lock has not been aquired
}
Now, lock could not be aquired because of some exception or because some other thread has already acuired it. In second parameter you can pass the time that a thread will wait to acquire a lock. I gave here short time because you don't want the other thread to do the job, when first is doing it.
So this solution seems the best.
When the other thread could not acquire the lock, it will go further instead of waiting (well it will wait for 1 milisecond).
Since lock is a language-specific wrapper around Monitor class, you need Monitor.TryEnter:
public class SomeService
{
private readonly object lockObject = new object();
public void StartSomeMethod()
{
if (Monitor.TryEnter(lockObject))
{
// start new thread
}
}
public void SomeMethod()
{
try
{
// ...
}
finally
{
Monitor.Exit(lockObject);
}
}
}
You can use a AutoResetEvent instead of your isLocked flag.
AutoResetEvent autoResetEvent = new AutoResetEvent(true);
public void StartSomeMethod()
{
if(autoResetEvent.WaitOne(0))
{
//start thread
}
}
public void SomeMethod()
{
try
{
//Do your work
}
finally
{
autoResetEvent.Set();
}
}
I have multi-threaded application, where different threads may want to perform an operation. I tried to use Mutex to make sure, that thread does not start an operation if it is already running.
System.Threading.Mutex mutex;
bool isRunning = System.Threading.Mutex.TryOpenExisting(name, out mutex);
if (!isRunning)
{
RunMethod();
}
within method I created mutex, and try to release it at the end:
var mutex = new Mutex(true, name);
try{
//do stuff, it takes some time
}
finally
{
//TODO: I want to get rid of Mutex here
}
How do I get rid of mutex? Because even after I called mutex.ReleaseMutex() and mutex.Close(), it still exists and can be found. How can I inform that operation is currently running or finished?
Is there another way to do this?
Same, like CodingGorilla said, using events is easier.
I hope I understand your question well.
This example shows some events techniques:
Waiting for a thread has been started.
The use of waiting on multiple events (WaitHandle.WaitAny())
How to terminate a thread, safe.
Testing an event state without waiting (.WaitOne(0))
Here is an example:
public class MultiThreadedExample : IDisposable
{
private Thread _thread;
private ManualResetEvent _terminatingEvent = new ManualResetEvent(false);
private ManualResetEvent _runningEvent = new ManualResetEvent(false);
private ManualResetEvent _threadStartedEvent = new ManualResetEvent(false);
public MultiThreadedExample()
{
_thread = new Thread(MyThreadMethod);
_thread.Start();
_threadStartedEvent.WaitOne();
}
private void MyThreadMethod()
{
_threadStartedEvent.Set();
var events = new WaitHandle[] { _terminatingEvent, _runningEvent };
while (WaitHandle.WaitAny(events) != 0) // <- WaitAny returns index within the array of the event that was Set.
{
try
{
// do work......
}
finally
{
// reset the event. so it can be triggered again.
_runningEvent.Reset();
}
}
}
public bool TryStartWork()
{
// .Set() will return if the event was set.
return _runningEvent.Set();
}
public bool IsRunning
{
get { return _runningEvent.WaitOne(0); }
}
public void Dispose()
{
// break the whileloop
_terminatingEvent.Set();
// wait for the thread to terminate.
_thread.Join();
}
}
This block of code is being accessed by many threads
// All code is from same class
public void ExecuteCommand(IAsciiCommand command, IAsciiCommandSynchronousResponder responder)
{
lock (commander)
{
if (commander.IsConnected)
{
commander.ExecuteCommand(command, responder);
}
}
}
public void Disconnect()
{
var tmp = commander.IsConnected;
commander.Disconnect();
if (commander.IsConnected != tmp && !commander.IsConnected)
{
OnPropertyChanged("IsConnected");
}
}
And eventually i get this:
How is this possible, that thread accessed into if statement, whose condition returns false? How can i fix it?
This is happening because the check and the call lack atomicity. Here is a sequence of events that could lead to an exception:
Two threads, A and B, are reaching the condition at the same time
Thread A checks the condition, which returns true, so it enters the if block
At the same time, thread scheduler decides that thread A has exhausted its time slot, and suspends it
Thread B calls Disconnect
Thread scheduler resumes thread A, which is inside the if condition. However, the command is no longer connected
This causes the exception
You can fix it by locking commander inside Disconnect().
public void Disconnect()
{
bool doEvent;
lock(commander) {
var tmp = commander.IsConnected;
commander.Disconnect();
doEvent = (commander.IsConnected != tmp && !commander.IsConnected)
}
// Run OnPropertyChanged outside the locked context
if (doEvent)
{
OnPropertyChanged("IsConnected");
}
}
You need to lock on a static object. Right now you're creating separate locks based on the object your are working with (commander). Try this:
public class WhatEverClassHasTheExecuteCommandMethod
{
private static object _lock = new object();
public void ExecuteCommand(IAsciiCommand command, IAsciiCommandSynchronousResponder responder)
{
lock (_lock)
if (commander.IsConnected)
commander.ExecuteCommand(command, responder);
}
}
If you are not locking while disconnecting, it's entirely possible to get a race condition. The basic solution is to add a lock inside the Disconnect method:
public void Disconnect()
{
lock (commander)
{
var tmp = commander.IsConnected;
commander.Disconnect();
if (commander.IsConnected != tmp && !commander.IsConnected)
OnPropertyChanged("IsConnected");
}
}
I have a job object made up of a collection of work items. Each job has it's own WatcherClass associated with it that checks the database every so often to see if it needs to cancel execution. It could be cancelled at any iteration in the workflow. IF it is cancelled, any threads running from the foreach block will propagate the cancellation and exit gracefully.
Is there any problem in my watcher code that could create a deadlock? I am trying to only allow one thread to process on the timer callback by using Timer.Change(Timeout.Infinite, Timeout.Infinite), but does the fact that I am changing WatcherClass.Job inside the lock statement defeat the lock's purpose (since I wrapped the same get/set for _Job in the same lock object)? The code appears to be working fine, but I know that is no indication of anything really.
The code in the main thread looks similiar to this:
using (WatcherClass watcher = new WatcherClass())
{
watcher.CancelTokenSource = new CancellationTokenSource();
watcher.Start();
foreach (SomeJob job in worksflow.Jobs)
{
watcher.Job = job;
//Do some stuff async
//Do some more stuff async
}
}
public class WatcherClass : IDisposable
{
private System.Threading.Timer _WatcherTimer;
private readonly object locker = new object();
private bool _Disposed = false;
private SomeJob _Job;
public SomeJob Job
{
get
{
lock (locker)
{
return _Job;
}
}
set
{
lock (locker)
{
_Job= value;
}
}
}
public System.Threading.Task.CancellationTokenSource
CancelToken { get; set; }
public WatcherClass()
{
_WatcherTimer = new Timer(new TimerCallback(DoCheck), null,
Timeout.Infinite, Timeout.Infinite);
}
public void Start()
{
_WatcherTimer.Change(30000, Timeout.Infinite);
}
public void DoCheck(object state)
{
lock (locker)
{
if (_Disposed || this.CancelToken.IsCancellationRequested)
return;
_WatcherTimer.Change(Timeout.Infinite, Timeout.Infinite);
//Check database to see if task is cancelled
if (cancelled)
{
this.CancelToken.Cancel();
_Job.CancelResult = CancelResult.CanceledByUser;
_Job.SomeOtherProperty = true;
}
else
{
//Safe to continue
_WatcherTimer.Change(30000, Timeout.Infinite);
}
}
}
public void Dispose(bool disposing)
{
lock (locker)
{
if (disposing)
{
if (_WatcherTimer != null)
_WatcherTimer.Dispose();
}
_Disposed = true;
}
}
}
The lock you aquire around the Task property and in the DoCheck function only protects access to the internal _task field of the WatcherClass. In DoCheck, you are also modifying properties of the _task object itself. The lock does not prevent anyone else from also modifying the task object's fields at the same time from other threads.
If in your application the task object is only manipulated by DoCheck, then you're probably ok. If the task object may be manipulated by code other than DoCheck, then you may have a problem.
Also keep in mind that every additional lock you create is an additional opportunity for deadlock. Multiple locks can be deadlock-free if they are always acquired in a specific order. If the code flow allows for lock A to be acquired before lock B in some situations, or lock B before lock A in other situations, then you have a serious deadlock risk. (Thread 1 locks A, tries to lock B while thread 2 locks B and tries to lock A => deadlock)
In your WatcherClass case, if you are going to have multiple watcherclass instances each with their own locks, be careful not to make external calls (or fire events) that could end up trying to acquire locks in other watcherclass instances. That's an AB / BA deadlock waiting to happen.
I was playing with a project of mine today and found an interesting little snippet, given the following pattern, you can safely cleanup a thread, even if it's forced to close early. My project is a network server where it spawns a new thread for each client. I've found this useful for early termination from the remote side, but also from the local side (I can just call .Abort() from inside my processing code).
Are there any problems you can see with this, or any suggestions you'd make to anyone looking at a similar approach?
Test case follows:
using System;
using System.Threading;
class Program
{
static Thread t1 = new Thread(thread1);
static Thread t2 = new Thread(thread2);
public static void Main(string[] args)
{
t1.Start();
t2.Start();
t1.Join();
}
public static void thread1() {
try {
// Do our work here, for this test just look busy.
while(true) {
Thread.Sleep(100);
}
} finally {
Console.WriteLine("We're exiting thread1 cleanly.\n");
// Do any cleanup that might be needed here.
}
}
public static void thread2() {
Thread.Sleep(500);
t1.Abort();
}
}
For reference, without the try/finally block, the thread just dies as one would expect.
Aborting another thread at all is just a bad idea unless the whole application is coming down. It's too easy to leave your program in an unknown state. Aborting your own thread is occasionally useful - ASP.NET throws a ThreadAbortException if you want to prematurely end the response, for example - but it's not a terribly nice design.
Safe clean-up of a thread should be mutual - there should be some shared flag requesting that the thread shuts down. The thread should check that flag periodically and quit appropriately.
Whether or not this will "safely" cleanup a thread cannot be discerned from a general code sample unfortunately. It's highly dependent upon the actual code that is executed within the thread. There are multiple issues you must consider. Each represents a potential bug in the code.
If the thread is currently in native code, it will not immediately respect the Thread.Abort call. It will do all of the work it wants to do in native code and will not throw until the code returns back to managed. Until this happens thread2 will hang.
Any native resources that are not freed in a finally block will be leaked in this scenario. All native resources should be freed in a finally block but not all code does this and it's an issue to consider.
Any locks that are not freed in a finally block will remain in a lock'd state and can lead to future dead locks.
There are other issues which are slipping my mind at the moment. But hopefully this will give you some guidance with your application.
It is generally not a good idea to abort threads. What you can do is poll for a stopRequested flag which can be set from other threads. Below is a sample WorkerThread class for your reference. For more information on how to use it, please refer to http://devpinoy.org/blogs/jakelite/archive/2008/12/20/threading-patterns-the-worker-thread-pattern.aspx
public abstract class WorkerThreadBase : IDisposable
{
private Thread _workerThread;
protected internal ManualResetEvent _stopping;
protected internal ManualResetEvent _stopped;
private bool _disposed;
private bool _disposing;
private string _name;
protected WorkerThreadBase()
: this(null, ThreadPriority.Normal)
{
}
protected WorkerThreadBase(string name)
: this(name, ThreadPriority.Normal)
{
}
protected WorkerThreadBase(string name,
ThreadPriority priority)
: this(name, priority, false)
{
}
protected WorkerThreadBase(string name,
ThreadPriority priority,
bool isBackground)
{
_disposing = false;
_disposed = false;
_stopping = new ManualResetEvent(false);
_stopped = new ManualResetEvent(false);
_name = name == null ? GetType().Name : name; ;
_workerThread = new Thread(threadProc);
_workerThread.Name = _name;
_workerThread.Priority = priority;
_workerThread.IsBackground = isBackground;
}
protected bool StopRequested
{
get { return _stopping.WaitOne(1, true); }
}
protected bool Disposing
{
get { return _disposing; }
}
protected bool Disposed
{
get { return _disposed; }
}
public string Name
{
get { return _name; }
}
public void Start()
{
ThrowIfDisposedOrDisposing();
_workerThread.Start();
}
public void Stop()
{
ThrowIfDisposedOrDisposing();
_stopping.Set();
_stopped.WaitOne();
}
public void WaitForExit()
{
ThrowIfDisposedOrDisposing();
_stopped.WaitOne();
}
#region IDisposable Members
public void Dispose()
{
dispose(true);
}
#endregion
public static void WaitAll(params WorkerThreadBase[] threads)
{
WaitHandle.WaitAll(
Array.ConvertAll<WorkerThreadBase, WaitHandle>(
threads,
delegate(WorkerThreadBase workerThread)
{ return workerThread._stopped; }));
}
public static void WaitAny(params WorkerThreadBase[] threads)
{
WaitHandle.WaitAny(
Array.ConvertAll<WorkerThreadBase, WaitHandle>(
threads,
delegate(WorkerThreadBase workerThread)
{ return workerThread._stopped; }));
}
protected virtual void Dispose(bool disposing)
{
//stop the thread;
Stop();
//make sure the thread joins the main thread
_workerThread.Join(1000);
//dispose of the waithandles
DisposeWaitHandle(_stopping);
DisposeWaitHandle(_stopped);
}
protected void ThrowIfDisposedOrDisposing()
{
if (_disposing)
{
throw new InvalidOperationException(
Properties.Resources.ERROR_OBJECT_DISPOSING);
}
if (_disposed)
{
throw new ObjectDisposedException(
GetType().Name,
Properties.Resources.ERROR_OBJECT_DISPOSED);
}
}
protected void DisposeWaitHandle(WaitHandle waitHandle)
{
if (waitHandle != null)
{
waitHandle.Close();
waitHandle = null;
}
}
protected abstract void Work();
private void dispose(bool disposing)
{
//do nothing if disposed more than once
if (_disposed)
{
return;
}
if (disposing)
{
_disposing = disposing;
Dispose(disposing);
_disposing = false;
//mark as disposed
_disposed = true;
}
}
private void threadProc()
{
Work();
_stopped.Set();
}
}