We Keep Coding

Multithreading BlockingCollection Alternatives to GetConsumingEnumerable() Producer-Consumer

I have a situation where I have multiple producers and multiple consumers. The producers enters a job into a queue. I chose the BlockingCollection and it works great since I need the consumers to wait for a job to be found. However, if I use the GetConsumingEnumerable() feature the order of the items in the collection change... this is not what I need.
It even says in MSDN http://msdn.microsoft.com/en-us/library/dd287186.aspx
that it does not preserve the order of the items.
Does anyone know an alternative for this situation?
I see that the Take method is available but does it also provide a 'wait' condition for the consumer threads?
It says http://msdn.microsoft.com/en-us/library/dd287085.aspx
'A call to Take may block until an item is available to be removed.' Is it better to use TryTake? I really need the thread to wait and keep checking for a job.

Take blocks the thread till something comes available.
TryTake as the name implies tries to do so but returns a bool if it fails or succeeds.
Allowing for more flex using it:
while(goingOn){
if( q.TryTake(out var){
Process(var)
}
else{
DoSomething_Usefull_OrNotUseFull_OrEvenSleep();
}
}
instead of
while(goingOn){
if( var x = q.Take(){
//w'll wait till this ever will happen and then we:
Process(var)
}
}
My votes are for TryTake :-)
EXAMPLE:
public class ProducerConsumer<T> {
public struct Message {
public T Data;
}
private readonly ThreadRunner _producer;
private readonly ThreadRunner _consumer;
public ProducerConsumer(Func<T> produce, Action<T> consume) {
var q = new BlockingCollection<Message>();
_producer = new Producer(produce,q);
_consumer = new Consumer(consume,q);
}
public void Start() {
_producer.Run();
_consumer.Run();
}
public void Stop() {
_producer.Stop();
_consumer.Stop();
}
private class Producer : ThreadRunner {
public Producer(Func<T> produce, BlockingCollection<Message> q) : base(q) {
_produce = produce;
}
private readonly Func<T> _produce;
public override void Worker() {
try {
while (KeepRunning) {
var item = _produce();
MessageQ.TryAdd(new Message{Data = item});
}
}
catch (ThreadInterruptedException) {
WasInterrupted = true;
}
}
}
public abstract class ThreadRunner {
protected readonly BlockingCollection<Message> MessageQ;
protected ThreadRunner(BlockingCollection<Message> q) {
MessageQ = q;
}
protected Thread Runner;
protected bool KeepRunning = true;
public bool WasInterrupted;
public abstract void Worker();
public void Run() {
Runner = new Thread(Worker);
Runner.Start();
}
public void Stop() {
KeepRunning = false;
Runner.Interrupt();
Runner.Join();
}
}
class Consumer : ThreadRunner {
private readonly Action<T> _consume;
public Consumer(Action<T> consume,BlockingCollection<Message> q) : base(q) {
_consume = consume;
}
public override void Worker() {
try {
while (KeepRunning) {
Message message;
if (MessageQ.TryTake(out message, TimeSpan.FromMilliseconds(100))) {
_consume(message.Data);
}
else {
//There's nothing in the Q so I have some spare time...
//Excellent moment to update my statisics or update some history to logfiles
//for now we sleep:
Thread.Sleep(TimeSpan.FromMilliseconds(100));
}
}
}
catch (ThreadInterruptedException) {
WasInterrupted = true;
}
}
}
}
}
USAGE:
[Fact]
public void ConsumerShouldConsume() {
var produced = 0;
var consumed = 0;
Func<int> produce = () => {
Thread.Sleep(TimeSpan.FromMilliseconds(100));
produced++;
return new Random(2).Next(1000);
};
Action<int> consume = c => { consumed++; };
var t = new ProducerConsumer<int>(produce, consume);
t.Start();
Thread.Sleep(TimeSpan.FromSeconds(5));
t.Stop();
Assert.InRange(produced,40,60);
Assert.InRange(consumed, 40, 60);
}

Better way to handle read-only access to state with another thread?

This is a design question, not a bug fix problem.
The situation is this. I have a lot of collections and objects contained in one class. Their contents are only changed by a single message handler thread. There is one other thread which is doing rendering. Each frame it iterates through some of these collections and draws to the screen based on the value of these objects. It does not alter the objects in any way, it is just reading their values.
Now when the rendering is being done, if any of the collections are altered, my foreach loops in the rendering method fail. How should I make this thread safe? Edit: So I have to lock the collections outside each foreach loop I run on them. This works, but it seems like a lot of repetitive code to solve this problem.
As a short, contrived example:
class State
{
public object LockObjects;
public List<object> Objects;
// Called by message handler thread
void HandleMessage()
{
lock (LockObjects)
{
Objects.Add(new object());
}
}
}
class Renderer
{
State m_state;
// Called by rendering thread
void Render()
{
lock (m_state.LockObjects)
{
foreach (var obj in m_state.Objects)
{
DrawObject(obj);
}
}
}
}
This is all well and good, but I'd rather not put locks on all my state collections if there's a better way. Is this "the right" way to do it or is there a better way?

The better way is to use begin/end methods and separated lists for your both threads and synchronization using auto events for example. It will be lock-free to your message handler thread and enables you to have a lot of render/message handler threads:
class State : IDisposable
{
private List<object> _objects;
private ReaderWriterLockSlim _locker;
private object _cacheLocker;
private List<object> _objectsCache;
private Thread _synchronizeThread;
private AutoResetEvent _synchronizationEvent;
private bool _abortThreadToken;
public State()
{
_objects = new List<object>();
_objectsCache = new List<object>();
_cacheLocker = new object();
_locker = new ReaderWriterLockSlim();
_synchronizationEvent = new AutoResetEvent(false);
_abortThreadToken = false;
_synchronizeThread = new Thread(Synchronize);
_synchronizeThread.Start();
}
private void Synchronize()
{
while (!_abortThreadToken)
{
_synchronizationEvent.WaitOne();
int objectsCacheCount;
lock (_cacheLocker)
{
objectsCacheCount = _objectsCache.Count;
}
if (objectsCacheCount > 0)
{
_locker.EnterWriteLock();
lock (_cacheLocker)
{
_objects.AddRange(_objectsCache);
_objectsCache.Clear();
}
_locker.ExitWriteLock();
}
}
}
public IEnumerator<object> GetEnumerator()
{
_locker.EnterReadLock();
foreach (var o in _objects)
{
yield return o;
}
_locker.ExitReadLock();
}
// Called by message handler thread
public void HandleMessage()
{
lock (_cacheLocker)
{
_objectsCache.Add(new object());
}
_synchronizationEvent.Set();
}
public void Dispose()
{
_abortThreadToken = true;
_synchronizationEvent.Set();
}
}
Or (the simpler way) you can use ReaderWriteerLockSlim (Or just locks if you sure you have only one reader) like in the following code:
class State
{
List<object> m_objects = new List<object>();
ReaderWriterLockSlim locker = new ReaderWriterLockSlim();
public IEnumerator<object> GetEnumerator()
{
locker.EnterReadLock();
foreach (var o in Objects)
{
yield return o;
}
locker.ExitReadLock();
}
private List<object> Objects
{
get { return m_objects; }
set { m_objects = value; }
}
// Called by message handler thread
public void HandleMessage()
{
locker.EnterWriteLock();
Objects.Add(new object());
locker.ExitWriteLock();
}
}

Humm... have you tried with a ReaderWriterLockSlim ? Enclose each conllection with one of this, and ensure you start a read or write operation each time you access it.

Possible to turn callback calls into IEnumerable

I'm writing a wrapper around a 3rd party library, and it has a method to scan the data it manages. The method takes a callback method that it calls for each item in the data that it finds.
e.g. The method is essentially: void Scan(Action<object> callback);
I want to wrap it and expose a method like IEnumerable<object> Scan();
Is this possible without resorting to a separate thread to do the actual scan and a buffer?

You can do this quite simply with Reactive:
class Program
{
static void Main(string[] args)
{
foreach (var x in CallBackToEnumerable<int>(Scan))
Console.WriteLine(x);
}
static IEnumerable<T> CallBackToEnumerable<T>(Action<Action<T>> functionReceivingCallback)
{
return Observable.Create<T>(o =>
{
// Schedule this onto another thread, otherwise it will block:
Scheduler.Later.Schedule(() =>
{
functionReceivingCallback(o.OnNext);
o.OnCompleted();
});
return () => { };
}).ToEnumerable();
}
public static void Scan(Action<int> act)
{
for (int i = 0; i < 100; i++)
{
// Delay to prove this is working asynchronously.
Thread.Sleep(100);
act(i);
}
}
}
Remember that this doesn't take care of things like cancellation, since the callback method doesn't really allow it. A proper solution would require work on the part of the external library.

You should investigate the Rx project — this allows an event source to be consumed as an IEnumerable.
I'm not sure if it allows vanilla callbacks to be presented as such (it's aimed at .NET events) but it would be worth a look as it should be possible to present a regular callback as an IObservable.

Here is a blocking enumerator (the Scan method needs to run in a separate thread)
public class MyEnumerator : IEnumerator<object>
{
private readonly Queue<object> _queue = new Queue<object>();
private ManualResetEvent _event = new ManualResetEvent(false);
public void Callback(object value)
{
lock (_queue)
{
_queue.Enqueue(value);
_event.Set();
}
}
public void Dispose()
{
}
public bool MoveNext()
{
_event.WaitOne();
lock (_queue)
{
Current = _queue.Dequeue();
if (_queue.Count == 0)
_event.Reset();
}
return true;
}
public void Reset()
{
_queue.Clear();
}
public object Current { get; private set; }
object IEnumerator.Current
{
get { return Current; }
}
}
static void Main(string[] args)
{
var enumerator = new MyEnumerator();
Scan(enumerator.Callback);
while (enumerator.MoveNext())
{
Console.WriteLine(enumerator.Current);
}
}
You could wrap it in a simple IEnumerable<Object>, but I would not recommend it. IEnumerable lists implies that you can run multiple enumerators on the same list, which you can't in this case.

How about this one:
IEnumerable<Object> Scan()
{
List<Object> objList = new List<Object>();
Action<Object> action = (obj) => { objList.Add(obj); };
Scan(action);
return objList;
}

Take a look at the yield keyword -- which will allow you to have a method that looks like an IEnumerable but which actually does processing for each return value.

Synchronizing thread communication?

Just for the heck of it I'm trying to emulate how JRuby generators work using threads in C#.
Also, I'm fully aware that C# has built in support for yield return, I'm just toying around a bit.
I guess it's some sort of poor mans coroutines by keeping multiple callstacks alive using threads. (even though none of the callstacks should execute at the same time)
The idea is like this:
The consumer thread requests a value
The worker thread provides a value and yields back to the consumer thread
Repeat untill worker thread is done
So, what would be the correct way of doing the following?
//example
class Program
{
static void Main(string[] args)
{
ThreadedEnumerator<string> enumerator = new ThreadedEnumerator<string>();
enumerator.Init(() =>
{
for (int i = 1; i < 100; i++)
{
enumerator.Yield(i.ToString());
}
});
foreach (var item in enumerator)
{
Console.WriteLine(item);
};
Console.ReadLine();
}
}
//naive threaded enumerator
public class ThreadedEnumerator<T> : IEnumerator<T>, IEnumerable<T>
{
private Thread enumeratorThread;
private T current;
private bool hasMore = true;
private bool isStarted = false;
AutoResetEvent enumeratorEvent = new AutoResetEvent(false);
AutoResetEvent consumerEvent = new AutoResetEvent(false);
public void Yield(T item)
{
//wait for consumer to request a value
consumerEvent.WaitOne();
//assign the value
current = item;
//signal that we have yielded the requested
enumeratorEvent.Set();
}
public void Init(Action userAction)
{
Action WrappedAction = () =>
{
userAction();
consumerEvent.WaitOne();
enumeratorEvent.Set();
hasMore = false;
};
ThreadStart ts = new ThreadStart(WrappedAction);
enumeratorThread = new Thread(ts);
enumeratorThread.IsBackground = true;
isStarted = false;
}
public T Current
{
get { return current; }
}
public void Dispose()
{
enumeratorThread.Abort();
}
object System.Collections.IEnumerator.Current
{
get { return Current; }
}
public bool MoveNext()
{
if (!isStarted)
{
isStarted = true;
enumeratorThread.Start();
}
//signal that we are ready to receive a value
consumerEvent.Set();
//wait for the enumerator to yield
enumeratorEvent.WaitOne();
return hasMore;
}
public void Reset()
{
throw new NotImplementedException();
}
public IEnumerator<T> GetEnumerator()
{
return this;
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return this;
}
}
Ideas?

There are many ways to implement the producer/consumer pattern in C#.
The best way, I guess, is using TPL (Task, BlockingCollection). See an example here.

ReaderWriterLockSlim Extension Method Performance

I've been playing with collections and threading and came across the nifty extension methods people have created to ease the use of ReaderWriterLockSlim by allowing the IDisposable pattern.
However, I believe I have come to realize that something in the implementation is a performance killer. I realize that extension methods are not supposed to really impact performance, so I am left assuming that something in the implementation is the cause... the amount of Disposable structs created/collected?
Here's some test code:
using System;
using System.Collections.Generic;
using System.Threading;
using System.Diagnostics;
namespace LockPlay {
static class RWLSExtension {
struct Disposable : IDisposable {
readonly Action _action;
public Disposable(Action action) {
_action = action;
}
public void Dispose() {
_action();
}
} // end struct
public static IDisposable ReadLock(this ReaderWriterLockSlim rwls) {
rwls.EnterReadLock();
return new Disposable(rwls.ExitReadLock);
}
public static IDisposable UpgradableReadLock(this ReaderWriterLockSlim rwls) {
rwls.EnterUpgradeableReadLock();
return new Disposable(rwls.ExitUpgradeableReadLock);
}
public static IDisposable WriteLock(this ReaderWriterLockSlim rwls) {
rwls.EnterWriteLock();
return new Disposable(rwls.ExitWriteLock);
}
} // end class
class Program {
class MonitorList<T> : List<T>, IList<T> {
object _syncLock = new object();
public MonitorList(IEnumerable<T> collection) : base(collection) { }
T IList<T>.this[int index] {
get {
lock(_syncLock)
return base[index];
}
set {
lock(_syncLock)
base[index] = value;
}
}
} // end class
class RWLSList<T> : List<T>, IList<T> {
ReaderWriterLockSlim _rwls = new ReaderWriterLockSlim();
public RWLSList(IEnumerable<T> collection) : base(collection) { }
T IList<T>.this[int index] {
get {
try {
_rwls.EnterReadLock();
return base[index];
} finally {
_rwls.ExitReadLock();
}
}
set {
try {
_rwls.EnterWriteLock();
base[index] = value;
} finally {
_rwls.ExitWriteLock();
}
}
}
} // end class
class RWLSExtList<T> : List<T>, IList<T> {
ReaderWriterLockSlim _rwls = new ReaderWriterLockSlim();
public RWLSExtList(IEnumerable<T> collection) : base(collection) { }
T IList<T>.this[int index] {
get {
using(_rwls.ReadLock())
return base[index];
}
set {
using(_rwls.WriteLock())
base[index] = value;
}
}
} // end class
static void Main(string[] args) {
const int ITERATIONS = 100;
const int WORK = 10000;
const int WRITE_THREADS = 4;
const int READ_THREADS = WRITE_THREADS * 3;
// create data - first List is for comparison only... not thread safe
int[] copy = new int[WORK];
IList<int>[] l = { new List<int>(copy), new MonitorList<int>(copy), new RWLSList<int>(copy), new RWLSExtList<int>(copy) };
// test each list
Thread[] writeThreads = new Thread[WRITE_THREADS];
Thread[] readThreads = new Thread[READ_THREADS];
foreach(var list in l) {
Stopwatch sw = Stopwatch.StartNew();
for(int k=0; k < ITERATIONS; k++) {
for(int i = 0; i < writeThreads.Length; i++) {
writeThreads[i] = new Thread(p => {
IList<int> il = p as IList<int>;
int c = il.Count;
for(int j = 0; j < c; j++) {
il[j] = j;
}
});
writeThreads[i].Start(list);
}
for(int i = 0; i < readThreads.Length; i++) {
readThreads[i] = new Thread(p => {
IList<int> il = p as IList<int>;
int c = il.Count;
for(int j = 0; j < c; j++) {
int temp = il[j];
}
});
readThreads[i].Start(list);
}
for(int i = 0; i < readThreads.Length; i++)
readThreads[i].Join();
for(int i = 0; i < writeThreads.Length; i++)
writeThreads[i].Join();
};
sw.Stop();
Console.WriteLine("time: {0} class: {1}", sw.Elapsed, list.GetType());
}
Console.WriteLine("DONE");
Console.ReadLine();
}
} // end class
} // end namespace
Here's a typical result:
time: 00:00:03.0965242 class: System.Collections.Generic.List`1[System.Int32]
time: 00:00:11.9194573 class: LockPlay.Program+MonitorList`1[System.Int32]
time: 00:00:08.9510258 class: LockPlay.Program+RWLSList`1[System.Int32]
time: 00:00:16.9888435 class: LockPlay.Program+RWLSExtList`1[System.Int32]
DONE
As you can see, using the extensions actually makes the performance WORSE than just using lock (monitor).

Looks like its the price of instantiating millions of structs and the extra bit of invocations.
I would go as far as to say that the ReaderWriterLockSlim is being misused in this sample, a lock is good enough in this case and the performance edge you get with the ReaderWriterLockSlim is negligible compared to the price of explaining these concepts to junior devs.
You get a huge advantage with reader writer style locks when it takes a non-negligable amount of time to perform reads and writes. The boost will be biggest when you have a predominantly read based system.
Try inserting a Thread.Sleep(1) while the locks are acquired to see how huge a difference it makes.
See this benchmark:
Time for Test.SynchronizedList`1[System.Int32] Time Elapsed 12310 ms
Time for Test.ReaderWriterLockedList`1[System.Int32] Time Elapsed 547 ms
Time for Test.ManualReaderWriterLockedList`1[System.Int32] Time Elapsed 566 ms
In my benchmarking I do not really notice much of a difference between the two styles, I would feel comfortable using it provided it had some finalizer protection in case people forget to dispose ....
using System.Threading;
using System.Diagnostics;
using System.Collections.Generic;
using System;
using System.Linq;
namespace Test {
static class RWLSExtension {
struct Disposable : IDisposable {
readonly Action _action;
public Disposable(Action action) {
_action = action;
}
public void Dispose() {
_action();
}
}
public static IDisposable ReadLock(this ReaderWriterLockSlim rwls) {
rwls.EnterReadLock();
return new Disposable(rwls.ExitReadLock);
}
public static IDisposable UpgradableReadLock(this ReaderWriterLockSlim rwls) {
rwls.EnterUpgradeableReadLock();
return new Disposable(rwls.ExitUpgradeableReadLock);
}
public static IDisposable WriteLock(this ReaderWriterLockSlim rwls) {
rwls.EnterWriteLock();
return new Disposable(rwls.ExitWriteLock);
}
}
class SlowList<T> {
List<T> baseList = new List<T>();
public void AddRange(IEnumerable<T> items) {
baseList.AddRange(items);
}
public virtual T this[int index] {
get {
Thread.Sleep(1);
return baseList[index];
}
set {
baseList[index] = value;
Thread.Sleep(1);
}
}
}
class SynchronizedList<T> : SlowList<T> {
object sync = new object();
public override T this[int index] {
get {
lock (sync) {
return base[index];
}
}
set {
lock (sync) {
base[index] = value;
}
}
}
}
class ManualReaderWriterLockedList<T> : SlowList<T> {
ReaderWriterLockSlim slimLock = new ReaderWriterLockSlim();
public override T this[int index] {
get {
T item;
try {
slimLock.EnterReadLock();
item = base[index];
} finally {
slimLock.ExitReadLock();
}
return item;
}
set {
try {
slimLock.EnterWriteLock();
base[index] = value;
} finally {
slimLock.ExitWriteLock();
}
}
}
}
class ReaderWriterLockedList<T> : SlowList<T> {
ReaderWriterLockSlim slimLock = new ReaderWriterLockSlim();
public override T this[int index] {
get {
using (slimLock.ReadLock()) {
return base[index];
}
}
set {
using (slimLock.WriteLock()) {
base[index] = value;
}
}
}
}
class Program {
private static void Repeat(int times, int asyncThreads, Action action) {
if (asyncThreads > 0) {
var threads = new List<Thread>();
for (int i = 0; i < asyncThreads; i++) {
int iterations = times / asyncThreads;
if (i == 0) {
iterations += times % asyncThreads;
}
Thread thread = new Thread(new ThreadStart(() => Repeat(iterations, 0, action)));
thread.Start();
threads.Add(thread);
}
foreach (var thread in threads) {
thread.Join();
}
} else {
for (int i = 0; i < times; i++) {
action();
}
}
}
static void TimeAction(string description, Action func) {
var watch = new Stopwatch();
watch.Start();
func();
watch.Stop();
Console.Write(description);
Console.WriteLine(" Time Elapsed {0} ms", watch.ElapsedMilliseconds);
}
static void Main(string[] args) {
int threadCount = 40;
int iterations = 200;
int readToWriteRatio = 60;
var baseList = Enumerable.Range(0, 10000).ToList();
List<SlowList<int>> lists = new List<SlowList<int>>() {
new SynchronizedList<int>() ,
new ReaderWriterLockedList<int>(),
new ManualReaderWriterLockedList<int>()
};
foreach (var list in lists) {
list.AddRange(baseList);
}
foreach (var list in lists) {
TimeAction("Time for " + list.GetType().ToString(), () =>
{
Repeat(iterations, threadCount, () =>
{
list[100] = 99;
for (int i = 0; i < readToWriteRatio; i++) {
int ignore = list[i];
}
});
});
}
Console.WriteLine("DONE");
Console.ReadLine();
}
}
}

The code appears to use a struct to avoid object creation overhead, but doesn't take the other necessary steps to keep this lightweight. I believe it boxes the return value from ReadLock, and if so negates the entire advantage of the struct. This should fix all the issues and perform just as well as not going through the IDisposable interface.
Edit: Benchmarks demanded. These results are normalized so the manual method (call Enter/ExitReadLock and Enter/ExitWriteLock inline with the protected code) have a time value of 1.00. The original method is slow because it allocates objects on the heap that the manual method does not. I fixed this problem, and in release mode even the extension method call overhead goes away leaving it identically as fast as the manual method.
Debug Build:
Manual: 1.00
Original Extensions: 1.62
My Extensions: 1.24
Release Build:
Manual: 1.00
Original Extensions: 1.51
My Extensions: 1.00
My code:
internal static class RWLSExtension
{
public static ReadLockHelper ReadLock(this ReaderWriterLockSlim readerWriterLock)
{
return new ReadLockHelper(readerWriterLock);
}
public static UpgradeableReadLockHelper UpgradableReadLock(this ReaderWriterLockSlim readerWriterLock)
{
return new UpgradeableReadLockHelper(readerWriterLock);
}
public static WriteLockHelper WriteLock(this ReaderWriterLockSlim readerWriterLock)
{
return new WriteLockHelper(readerWriterLock);
}
public struct ReadLockHelper : IDisposable
{
private readonly ReaderWriterLockSlim readerWriterLock;
public ReadLockHelper(ReaderWriterLockSlim readerWriterLock)
{
readerWriterLock.EnterReadLock();
this.readerWriterLock = readerWriterLock;
}
public void Dispose()
{
this.readerWriterLock.ExitReadLock();
}
}
public struct UpgradeableReadLockHelper : IDisposable
{
private readonly ReaderWriterLockSlim readerWriterLock;
public UpgradeableReadLockHelper(ReaderWriterLockSlim readerWriterLock)
{
readerWriterLock.EnterUpgradeableReadLock();
this.readerWriterLock = readerWriterLock;
}
public void Dispose()
{
this.readerWriterLock.ExitUpgradeableReadLock();
}
}
public struct WriteLockHelper : IDisposable
{
private readonly ReaderWriterLockSlim readerWriterLock;
public WriteLockHelper(ReaderWriterLockSlim readerWriterLock)
{
readerWriterLock.EnterWriteLock();
this.readerWriterLock = readerWriterLock;
}
public void Dispose()
{
this.readerWriterLock.ExitWriteLock();
}
}
}

My guess (you would need to profile to verify) is that the performance drop isn't from creating the Disposable instances (they should be fairly cheap, being structs). Instead I expect it's from creating the Action delegates. You could try changing the implementation of your Disposable struct to store the instance of ReaderWriterLockSlim instead of creating an Action delegate.
Edit: #280Z28's post confirms that it's the heap allocation of Action delegates that's causing the slowdown.