We Keep Coding

Stopping a service that reads MSMQ

I'm a Java programmer who has been asked to make some changes to C# applications. I've been working with C# for a week now, and I've finally hit a point where looking at the documentation isn't helping and I can't find solutions when I google.
In this case I have a Windows Service that processes messages that arrive in a MSMQ. When a message is received the currently listening thread picks it up and goes off to do an operation that takes a couple of seconds.
public void Start()
{
this.listen = true;
for (int i = 0; i < Constants.ThreadMaxCount; i++)
{
ThreadPool.QueueUserWorkItem(new WaitCallback(this.StartListening), i);
}
...
private void StartListening(Object threadContext)
{
int threadId = (int)threadContext;
threads[threadId] = Thread.CurrentThread;
PostRequest postReq;
while(this.listen)
{
System.Threading.Monitor.Enter(locker);
try
{
postReq = GettingAMessage();
}
finally
{
System.Threading.Monitor.Exit(locker);
}
}
...
}
GettingAMessage() has the following lines that listen for a message:
Task<Message> ts = Task.Factory.FromAsync<Message>
(queue.BeginReceive(), queue.EndReceive);
ts.Wait();
The problem is, when the Stop() method is called and there are no messages going into the MSMQ the threads all sit there waiting for a message. I have tried using timeouts, but that method doesn't seem elegant to me(and having switched over to the Task Factory, I'm not sure how to implement them currently). My solution to this was to add a reference of each thread to an array, so that I could cancel them. The following is called by each worker thread after being created.
threads[threadId] = Thread.CurrentThread;
and then supposed to be aborted by
public void Stop()
{
try
{
this.listen = false;
foreach(Thread a in threads) {
a.Abort();
}
}
catch
{...}
}
Any advice on why this isn't shutting the threads down? (Or even better, can anyone tell me where I should look for how to cancel the ts.Wait() properly?)

Use the ManualResetEvent class to achieve a proper & graceful stopping of your running threads.
In addition, don't use the ThreadPool for long running threads, use your own created threads, otherwise, with lots of long-running tasks, you could end up with thread-pool starvation, possibly even leading to deadlock:
public class MsmqListener
{
privatec ManualResetEvent _stopRequested = new ManualResetEvent(false);
private List<Thread> _listenerThreads;
private object _locker = new _locker();
//-----------------------------------------------------------------------------------------------------
public MsmqListener
{
CreateListenerThreads();
}
//-----------------------------------------------------------------------------------------------------
public void Start()
{
StartListenerThreads();
}
//-----------------------------------------------------------------------------------------------------
public void Stop()
{
try
{
_stopRequested.Set();
foreach(Thread thread in _listenerThreads)
{
thread.Join(); // Wait for all threads to complete gracefully
}
}
catch( Exception ex)
{...}
}
//-----------------------------------------------------------------------------------------------------
private void StartListening()
{
while( !_stopRequested.WaitOne(0) ) // Blocks the current thread for 0 ms until the current WaitHandle receives a signal
{
lock( _locker )
{
postReq = GettingAMessage();
}
...
}
//-----------------------------------------------------------------------------------------------------
private void CreateListenerThreads()
{
_listenerThreads = new List<Thread>();
for (int i = 0; i < Constants.ThreadMaxCount; i++)
{
listenerThread = new Thread(StartListening);
listenerThreads.Add(listenerThread);
}
}
//-----------------------------------------------------------------------------------------------------
private void StartListenerThreads()
{
foreach(var thread in _listenerThreads)
{
thread.Start();
}
}
}
UPDATE:
I changed the use of AutoResetEvent with ManualResetEvent in order to support the stopping of multiple waiting threads (Using ManualResetEvent, once you signaled, all waiting threads will be notified and be free to proceed theirs job - stop pooling for messages, in your case).
Using volatile bool does not provide all the guaranties. It may still read stale data. Better to use underlying OS synchronisation mechanism as it provides much stronger guaranties. Source: stackoverflow.com/a/11953661/952310

Do I need to use locking in the following c# code?

In the following code I am using two threads to share sane resource in this example it's a queue so do I need to use lock while en-queueing or dequeuing if yes then why because program seems to work fine.
class Program
{
static Queue<string> sQ = new Queue<string>();
static void Main(string[] args)
{
Thread prodThread = new Thread(ProduceData);
Thread consumeThread = new Thread(ConsumeData);
prodThread.Start();
consumeThread.Start();
Console.ReadLine();
}
private static void ProduceData()
{
for (int i = 0; i < 100; i++)
{
sQ.Enqueue(i.ToString());
}
}
private static void ConsumeData()
{
while (true)
{
if (sQ.Count > 0)
{
string s = sQ.Dequeue();
Console.WriteLine("DEQUEUE::::" + s);
}
}
}
}

Yes you do, System.Collections.Generic.Queue<T> is not thread safe for being written to and read from at the same time. You either need to lock on the same object before enquing or dequing or if you are using .NET 4/4.5 use the System.Collections.Concurrent.ConcurrentQueue<T> class instead and use the TryDequeue method.
The reason your current implementation has not caused you a problem so far, is due to the Thread.Sleep(500) call (not something you should be using in production code) which means that the prodThread doesn't write to the queue while the consumeThread reads from it since the read operation takes less than 500ms. If you remove the Thread.Sleep odds are it will throw an exception at some point.

Why does the process lose threads?

Here is some code that perpetually generate GUIDs. I've written it to learn about threading. In it you'll notice that I've got a lock around where I generate GUIDs and enqueue them even though the ConcurrentQueue is thread safe. It's because my actual code will need to use NHibernate and so I must make sure that only one thread gets to fill the queue.
While I monitor this code in Task Manager, I notice the process drops the number of threads from 18 (on my machine) to 14 but no less. Is this because my code isn't good?
Also can someone refactor this if they see fit? I love shorter code.
class Program
{
ConcurrentNewsBreaker Breaker;
static void Main(string[] args)
{
new Program().Execute();
Console.Read();
}
public void Execute()
{
Breaker = new ConcurrentNewsBreaker();
QueueSome();
}
public void QueueSome()
{
ThreadPool.QueueUserWorkItem(DoExecute);
}
public void DoExecute(Object State)
{
String Id = Breaker.Pop();
Console.WriteLine(String.Format("- {0} {1}", Thread.CurrentThread.ManagedThreadId, Breaker.Pop()));
if (Breaker.Any())
QueueSome();
else
Console.WriteLine(String.Format("- {0} XXXX ", Thread.CurrentThread.ManagedThreadId));
}
}
public class ConcurrentNewsBreaker
{
static readonly Object LockObject = new Object();
ConcurrentQueue<String> Store = new ConcurrentQueue<String>();
public String Pop()
{
String Result = null;
if (Any())
Store.TryDequeue(out Result);
return Result;
}
public Boolean Any()
{
if (!Store.Any())
{
Task FillTask = new Task(FillupTheQueue, Store);
FillTask.Start();
FillTask.Wait();
}
return Store.Any();
}
private void FillupTheQueue(Object StoreObject)
{
ConcurrentQueue<String> Store = StoreObject as ConcurrentQueue<String>;
lock(LockObject)
{
for(Int32 i = 0; i < 100; i++)
Store.Enqueue(Guid.NewGuid().ToString());
}
}
}

You are using .NET's ThreadPool so .NET/Windows manages the number of threads based on the amount of work waiting to be processed.

While I monitor this code in Task
Manager, I notice the process drops
the number of threads from 18 (on my
machine) to 14 but no less. Is this
because my code isn't good?
This does not indicate a problem. 14 is still high, unless you've got a 16-core cpu.
The threadpool will try to adjust and do the work with as few threads as possible.
You should start to worry when the number of threads goes up significantly.

New inside a lock

I noticed the following code from our foreign programmers:
private Client[] clients = new Client[0];
public CreateClients(int count)
{
lock (clients)
{
clients = new Client[count];
for(int i=0; i<count; i++)
{
Client[i] = new Client();//Stripped
}
}
}
It's not exactly proper code but I was wondering what exactly this will do. Will this lock on a new object each time this method is called?

To answer your question of "I was wondering what exactly this will do" consider what happens if two threads try to do this.
Thread 1: locks on the clients reference, which is `new Client[0]`
Thread 1 has entered the critical block
Thread 1: makes a array and assigns it to the clients reference
Thread 2: locks on the clients reference, which is the array just made in thread 1
Thread 2 has entered the critical block
You know have two threads in the critical block at the same time. That's bad.

This lock really does nothing. It locks an instance of an object which is immediately changed such that other threads entering this method will lock on a different object. The result is 2 threads executing in the middle of the lock which is probably not what was intended.
A much better approach here is to use a different, non-changing object to lock on
private readonly object clientsLock = new object();
private Client[] clients = new Client[0];
public CreateClients(int count) {
lock (clientsLock) {
clients = new string[count];
...
}
}

This code is wrong - it will lock on a new instance every time it's called.
It should look like that:
private static readonly object clientsLock = new object();
private static string[] Clients = null;
public CreateClients(int count)
{
if(clients == null)
{
lock (clientsLock)
{
if(clients == null)
{
clients = new string[count];
for(int i=0; i<count; i++)
{
client[i] = new Client();//Stripped
}
}
}
}
}
There's no point in locking every time the method is called - that's why the surrounding if clause.

Use :
private object = new Object();
lock(object){
//your code
}

I think you're correct to doubt this code!
This code will lock on the previous instance each time - this might be the desired effect, but I doubt it. It won't stop multiple threads from creating multiple arrays.

Make all threads sleep

I work with new Parallel.For that creates multiple threads to perform the same operation.
In case one of the threads fail, it means that I'm working "too fast" and I need to put all the threads to rest for a few seconds.
Is there a way to perform something like Thread.Sleep - only to do the same on all threads at once?

This is a direct answer to the question, except for the Parallel.For bit.
It really is a horrible pattern; you should probably be using a proper synchronization mechanism, and get the worker threads to, without preemption, occasionally check if they need to 'back off.'
In addition, this uses Thread.Suspend and Thread.Resume which are both deprecated, and with good reason (from Thread.Suspend):
"Do not use the Suspend and Resume methods to synchronize the activities of threads. You have no way of knowing what code a thread is executing when you suspend it. If you suspend a thread while it holds locks during a security permission evaluation, other threads in the AppDomain might be blocked. If you suspend a thread while it is executing a class constructor, other threads in the AppDomain that attempt to use that class are blocked. Deadlocks can occur very easily."
(Untested)
public class Worker
{
private readonly Thread[] _threads;
private readonly object _locker = new object();
private readonly TimeSpan _tooFastSuspensionSpan;
private DateTime _lastSuspensionTime;
public Worker(int numThreads, TimeSpan tooFastSuspensionSpan)
{
_tooFastSuspensionSpan = tooFastSuspensionSpan;
_threads = Enumerable.Repeat(new ThreadStart(DoWork), numThreads)
.Select(ts => new Thread(ts))
.ToArray();
}
public void Run()
{
foreach (var thread in _threads)
{
thread.Start();
}
}
private void DoWork()
{
while (!IsWorkComplete())
{
try
{
// Do work here
}
catch (TooFastException)
{
SuspendAll();
}
}
}
private void SuspendAll()
{
lock (_locker)
{
// We don't want N near-simultaneous failures causing a sleep-duration of N * _tooFastSuspensionSpan
// 1 second is arbitrary. We can't be deterministic about it since we are forcefully suspending threads
var now = DateTime.Now;
if (now.Subtract(_lastSuspensionTime) < _tooFastSuspensionSpan + TimeSpan.FromSeconds(1))
return;
_lastSuspensionTime = now;
var otherThreads = _threads.Where(t => t.ManagedThreadId != Thread.CurrentThread.ManagedThreadId).ToArray();
foreach (var otherThread in otherThreads)
otherThread.Suspend();
Thread.Sleep(_tooFastSuspensionSpan);
foreach (var otherThread in otherThreads)
otherThread.Resume();
}
}
}

You need to create an inventory of your worker threads and then perhaps you can use Thread.Suspend and Resume methods. Mind you that using Suspend can be dangerous (for example, thread may have acquired lock before suspending). And suspend/resume have been marked obsolate due to such issues.