We Keep Coding

Need help tweaking c# multi-threaded routine

I've created a windows service which runs a multi-threaded routine on a machine with 24 cores, 48 virtual, using Parallel.ForEach. This service, which has been running great in a production environment, bulk copies data into an SQL Server database. Currently it does this very well, around 6000 inserts per second, but I believe it can be tweaked. Below is part of the code I am using; there's an example of current functionality and proposed changes for tweaking. As can be seen from the code, currently a lock is taken for every call to Add, which I believe makes the Parallel.ForEach somewhat non-parallel. So I'm looking for a "fix"; and hoping my new method, also defined in the code, would do the trick.
public class MainLoop
{
public void DoWork()
{
var options = new ParallelOptions
{
MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
};
var workQueueManager = new ObjWorkQueueManager(queueSize: 1000);
// ignore the fact that this while loop would be a never ending loop,
// there's other logic not shown here that exits the loop!
while (true)
{
ICollection<object> work = GetWork();
Parallel.ForEach(work, options, (item) =>
{
workQueueManager.AddOLD(item);
});
}
}
private ICollection<object> GetWork()
{
// return list of work from some arbitrary source
throw new NotImplementedException();
}
}
public class ObjWorkQueueManager
{
private readonly int _queueSize;
private ObjDataReader _queueDataHandler;
private readonly object _sync;
public ObjWorkQueueManager(int queueSize)
{
_queueSize = queueSize;
_queueDataHandler = new ObjDataReader(queueSize);
_sync = new object();
}
// current Add method works great, but blocks with EVERY call
public void AddOLD(object value)
{
lock (_sync)
{
if (_queueDataHandler.Add(value) == _queueSize)
{
// create a new thread to handle copying the queued data to repository
Thread t = new Thread(SaveQueuedData);
t.Start(_queueDataHandler);
// start a new queue
_queueDataHandler = new ObjDataReader(_queueSize);
}
}
}
// hoping for a new Add method to work better by blocking only
// every nth call where n = _queueSize
public void AddNEW(object value)
{
int queued;
if ((queued = _queueDataHandler.Add(value)) >= _queueSize)
{
lock (_sync)
{
if (queued == _queueSize)
{
Thread t = new Thread(SaveQueuedData);
t.Start(_queueDataHandler);
}
}
}
else if (queued == 0)
{
lock (_sync)
{
_queueDataHandler = new ObjDataReader(_queueSize);
AddNEW(value);
}
}
}
// this method will Bulk Copy data into an SQL DB
private void SaveQueuedData(object o)
{
// do something with o as ObjDataReader
}
}
// implements IDataReader, Read method of IDataReader dequeues from _innerQueue
public class ObjDataReader
{
private readonly int _capacity;
private Queue<object> _innerQueue;
public ObjDataReader(int capacity)
{
_capacity = capacity;
_innerQueue = new Queue<object>(capacity);
}
public int Add(object value)
{
if (_innerQueue.Count < _capacity)
{
_innerQueue.Enqueue(value);
return _innerQueue.Count;
}
return 0;
}
}

Collection was modified; enumeration operation may not execute even though the collection was modified exclusively in lock statements

I have the following base code. The ActionMonitor can be used by anyone, in whatever setting, regardless of single-thread or multi-thread.
using System;
public class ActionMonitor
{
public ActionMonitor()
{
}
private object _lockObj = new object();
public void OnActionEnded()
{
lock (_lockObj)
{
IsInAction = false;
foreach (var trigger in _triggers)
trigger();
_triggers.Clear();
}
}
public void OnActionStarted()
{
IsInAction = true;
}
private ISet<Action> _triggers = new HashSet<Action>();
public void ExecuteAfterAction(Action action)
{
lock (_lockObj)
{
if (IsInAction)
_triggers.Add(action);
else
action();
}
}
public bool IsInAction
{
get;private set;
}
}
On exactly one occasion, when I examined a crash on client's machine, an exception was thrown at:
System.Core: System.InvalidOperationException Collection was modified;enumeration operation may not execute. at
System.Collections.Generic.HashSet`1.Enumerator.MoveNext() at
WPFApplication.ActionMonitor.OnActionEnded()
My reaction when seeing this stack trace: this is unbelievable! This must be a .Net bug!.
Because although ActionMonitor can be used in multithreading setting, but the crash above shouldn't occur-- all the _triggers ( the collection) modification happens inside a lock statement. This guarantees that one cannot iterate over the collection and modifying it at the same time.
And, if _triggers happened to contain an Action that involves ActionMonitor, then the we might get a deadlock, but it would never crash.
I have seen this crash exactly once, so I can't reproduce the problem at all. But base on my understanding of multithreading and lock statement, this exception can never have occurred.
Do I miss something here? Or is it known that .Net can behave it a very quirky way, when it involves System.Action?

You didn't shield your code against the following call:
private static ActionMonitor _actionMonitor;
static void Main(string[] args)
{
_actionMonitor = new ActionMonitor();
_actionMonitor.OnActionStarted();
_actionMonitor.ExecuteAfterAction(Foo1);
_actionMonitor.ExecuteAfterAction(Foo2);
_actionMonitor.OnActionEnded();
Console.ReadLine();
}
private static void Foo1()
{
_actionMonitor.OnActionStarted();
//Notice that if you would call _actionMonitor.OnActionEnded(); here instead of _actionMonitor.OnActionStarted(); - you would get a StackOverflow Exception
_actionMonitor.ExecuteAfterAction(Foo3);
}
private static void Foo2()
{
}
private static void Foo3()
{
}
FYI - that's the scenario Damien_The_Unbeliever is talking about in the comments.
To fix that issue the only 2 things that come in mind are
Don't call it like this, it's your class and your code is calling it so make sure you stick to your own rules
Get a copy of the _trigger list and enumarate this
About point 1, you could track if OnActionEnded is running and throw an exception if OnActionStarted is called while running:
private bool _isRunning = false;
public void OnActionEnded()
{
lock (_lockObj)
{
try
{
_isRunning = true;
IsInAction = false;
foreach (var trigger in _triggers)
trigger();
_triggers.Clear();
}
finally
{
_isRunning = false;
}
}
}
public void OnActionStarted()
{
lock (_lockObj)
{
if (_isRunning)
throw new NotSupportedException();
IsInAction = true;
}
}
About point 2, how about this
public class ActionMonitor
{
public ActionMonitor()
{
}
private object _lockObj = new object();
public void OnActionEnded()
{
lock (_lockObj)
{
IsInAction = false;
var tmpTriggers = _triggers;
_triggers = new HashSet<Action>();
foreach (var trigger in tmpTriggers)
trigger();
//have to decide what to do if _triggers isn't empty here, we could use a while loop till its empty
//so for example
while (true)
{
var tmpTriggers = _triggers;
_triggers = new HashSet<Action>();
if (tmpTriggers.Count == 0)
break;
foreach (var trigger in tmpTriggers)
trigger();
}
}
}
public void OnActionStarted()
{
lock (_lockObj) //fix the error #EricLippert talked about in comments
IsInAction = true;
}
private ISet<Action> _triggers = new HashSet<Action>();
public void ExecuteAfterAction(Action action)
{
lock (_lockObj)
{
if (IsInAction)
_triggers.Add(action);
else
action();
}
}
public bool IsInAction
{
get;private set;
}
}

This guarantees that one cannot iterate over the collection and modifying it at the same time.
No. You have a reentrancy problem.
Consider what happens if inside the call to trigger (same thread, so lock is already held), you modify the collection:
csharp
foreach (var trigger in _triggers)
trigger(); // _triggers modified in here
In fact if you look at your full callstack, you will be able to find the frame that is enumerating the collection. (by the time the exception happens, the code that modified the collection has been popped off the stack)

I feel like I'm re-inventing the wheel. Dispatch work to a specific thread. I want to process events from multiple threads sequentially on one thread

I know that similar things exist in WPF and forms applications with the Control.Invoke method, I also know of the existence of BackgroundWorker, ThreadPool etc.
However, I don't want to depend on Forms/WPF, and I want to make sure work is executed sequentially and on one thread.
Edit: Rationale: I want to drive a state machine from one thread. The events come from other threads tough. There is no UI.
So far I couldn't really figure out how to do this with existing framework classes but I might have misunderstood the documentation.
Edit: I forgot to mention I'm bound to .NET Framework 3.5
What I wrote so far:
public class Dispatcher
{
string Name;
Thread WorkerThread;
Queue<Action> WorkQueue;
List<Exception> Exceptions;
ManualResetEvent Gate;
volatile bool KeepRunning;
readonly object WorkLocker;
public override string ToString()
{
return String.Format("{0}({1})", this.GetType().Name, Name);
}
public Dispatcher(string name)
{
Name = name;
WorkLocker = new Object();
Gate = new ManualResetEvent(false);
WorkQueue = new Queue<Action>();
Exceptions = new List<Exception>();
}
public void Start()
{
if (WorkerThread == null)
{
WorkerThread = new Thread(doDispatch)
{
IsBackground = true,
Name = this.Name
};
WorkerThread.Start();
}
}
public void Stop()
{
if (WorkerThread != null && WorkerThread.IsAlive)
{
Dispatch(() => { KeepRunning = false; });
WorkerThread.Join();
}
WorkerThread = null;
}
public void Reset()
{
Stop();
lock (WorkLocker)
{
WorkQueue = new Queue<Action>();
Exceptions = new List<Exception>();
}
}
public void Dispatch(Action a)
{
lock (WorkLocker)
{
WorkQueue.Enqueue(a);
}
Gate.Set();
}
public List<Exception> CollectExceptions()
{
List<Exception> result = new List<Exception>();
lock(WorkLocker)
{
foreach(Exception e in Exceptions)
{
result.Add(e);
}
Exceptions.Clear();
}
return result;
}
private void doDispatch()
{
KeepRunning = true;
while (KeepRunning)
{
Gate.WaitOne();
lock (WorkLocker)
{
while (WorkQueue.Count > 0)
{
try
{
WorkQueue.Dequeue()?.Invoke();
}
catch (Exception e)
{
Exceptions.Add(e);
}
}
}
}
}
}
Is there a way to do something like this in a simpler way? Another nice feature would be being able to dispatch calls that have multiple arguments.

Since you are bound to 3.5 you can't use BlockingCollection or the DataFlow library...you'll have to roll your own implementation.
The sample code you provided is a good start, but you should apply the Single Responsibility Principle to make it cleaner and easier to refactor when(if?) you upgrade the .NET Framework.
I would do it like this:
Create a thread safe wrapper class around Queue that somewhat
mimics BlockingCollection, this answer provides a nice example
Structure your code around a consumer/producer flow and inject the wrapper

What is the correct way to dispose elements held inside a ThreadLocal<IDisposable>?

When you use a ThreadLocal<T> and T implements IDisposable, how are you supposed to dispose of the members being held inside of the ThreadLocal?
According to ILSpy, the Dispose() and Dispose(bool) methods of ThreadLocal are
public void Dispose()
{
this.Dispose(true);
GC.SuppressFinalize(this);
}
protected virtual void Dispose(bool disposing)
{
int currentInstanceIndex = this.m_currentInstanceIndex;
if (currentInstanceIndex > -1 && Interlocked.CompareExchange(ref this.m_currentInstanceIndex, -1, currentInstanceIndex) == currentInstanceIndex)
{
ThreadLocal<T>.s_availableIndices.Push(currentInstanceIndex);
}
this.m_holder = null;
}
It does not appear that ThreadLocal attempts to call Dispose on its child members. I can't tell how to reference each thread it internally has allocated so I can take care of it.
I ran a test with the following code, the class is never disposed
static class Sandbox
{
static void Main()
{
ThreadLocal<TestClass> test = new ThreadLocal<TestClass>();
test.Value = new TestClass();
test.Dispose();
Console.Read();
}
}
class TestClass : IDisposable
{
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
protected void Dispose(bool Disposing)
{
Console.Write("I was disposed!");
}
}

I had a look at the code in ThreadLocal<T> to see what the current Dispose is doing and it appears to be a lot of voodoo. Obviously disposing of thread-related stuff.
But it doesn't dispose of the values if T itself is disposable.
Now, I have a solution - a ThreadLocalDisposables<T> class, but before I give the full definition it's worth thinking about what should happen if you wrote this code:
var tl = new ThreadLocalDisposables<IExpensiveDisposableResource>();
tl.Value = myEdr1;
tl.Value = myEdr2;
tl.Dispose();
Should both myEdr1 & myEdr2 both be disposed? Or just myEdr2? Or should myEdr1 be disposed when myEdr2 was assigned?
It's not clear to me what the semantics should be.
It is clear to me, however, that if I wrote this code:
var tl = new ThreadLocalDisposables<IExpensiveDisposableResource>(
() => new ExpensiveDisposableResource());
tl.Value.DoSomething();
tl.Dispose();
Then I would expect that the resource created by the factory for each thread should be disposed of.
So I'm not going to allow the direct assignment of the disposable value for ThreadLocalDisposables and only allow the factory constructor.
Here's ThreadLocalDisposables:
public class ThreadLocalDisposables<T> : IDisposable
where T : IDisposable
{
private ThreadLocal<T> _threadLocal = null;
private ConcurrentBag<T> _values = new ConcurrentBag<T>();
public ThreadLocalDisposables(Func<T> valueFactory)
{
_threadLocal = new ThreadLocal<T>(() =>
{
var value = valueFactory();
_values.Add(value);
return value;
});
}
public void Dispose()
{
_threadLocal.Dispose();
Array.ForEach(_values.ToArray(), t => t.Dispose());
}
public override string ToString()
{
return _threadLocal.ToString();
}
public bool IsValueCreated
{
get { return _threadLocal.IsValueCreated; }
}
public T Value
{
get { return _threadLocal.Value; }
}
}
Does this help?

In .NET 4.5, the Values property was added to ThreadLocal<> to deal with the problem of manually managing the lifetime of ThreadLocal objects. It returns a list of all current instances bound to that ThreadLocal variable.
An example using a Parallel.For loop accessing a ThreadLocal database connection pool was presented in this MSDN article. The relevant code snippet is below.
var threadDbConn = new ThreadLocal<MyDbConnection>(() => MyDbConnection.Open(), true);
try
{
Parallel.For(0, 10000, i =>
{
var inputData = threadDbConn.Value.GetData(i);
...
});
}
finally
{
foreach(var dbConn in threadDbConn.Values)
{
dbConn.Close();
}
}

Normally when you don't explicitly dispose of a class that holds an unmanaged resource, the garbage collector will eventually run and dispose of it. For this to happen, the class has to have a finalizer that disposes of its resource. Your sample class doesn't have a finalizer.
Now, to dispose of a class that's held inside a ThreadLocal<T> where T is IDisposable you also have to do it yourself. ThreadLocal<T> is just a wrapper, it won't attempt to guess what's the correct behavior for its wrapped reference when it is itself disposed. The class could, e.g., survive its thread local storage.

This is related to ThreadLocal<> and memory leak
My guess is because there is no IDisposable constraint on T, it is assumed that the user of ThreadLocal<T> will dispose of the local object, when appropriate.

How is the ThreadLocal.Dispose method itself getting called? I would expect that it would most likely be within something like a "using" block. I would suggest that one wrap the "using" block for the ThreadLocal with a "using" block for the resource that's going to be stored there.

MSDN reference states that the ThreadLocal values should be disposed by the thread using them once its done. However in some instances such as event threading using a thread pool A thread may use the value and go off to do something else and then come back to the value N number of times.
Specific example is where I want an Entity Framework DBContext to persist across the lifespan of a series of service bus worker threads.
I've written up the following class which I use in these instances:
Either DisposeThreadCompletedValues can be called manually every so often by another thread or the internal monitor thread can be activated
Hopefully this helps?
using System.Threading;
public class DisposableThreadLocal<T> : IDisposable
where T : IDisposable
{
public DisposableThreadLocal(Func<T> _ValueFactory)
{
Initialize(_ValueFactory, false, 1);
}
public DisposableThreadLocal(Func<T> _ValueFactory, bool CreateLocalWatcherThread, int _CheckEverySeconds)
{
Initialize(_ValueFactory, CreateLocalWatcherThread, _CheckEverySeconds);
}
private void Initialize(Func<T> _ValueFactory, bool CreateLocalWatcherThread, int _CheckEverySeconds)
{
m_ValueFactory = _ValueFactory;
m_CheckEverySeconds = _CheckEverySeconds * 1000;
if (CreateLocalWatcherThread)
{
System.Threading.ThreadStart WatcherThreadStart;
WatcherThreadStart = new ThreadStart(InternalMonitor);
WatcherThread = new Thread(WatcherThreadStart);
WatcherThread.Start();
}
}
private object SyncRoot = new object();
private Func<T> m_ValueFactory;
public Func<T> ValueFactory
{
get
{
return m_ValueFactory;
}
}
private Dictionary<Thread, T> m_InternalDict = new Dictionary<Thread, T>();
private Dictionary<Thread, T> InternalDict
{
get
{
return m_InternalDict;
}
}
public T Value
{
get
{
T Result;
lock(SyncRoot)
{
if (!InternalDict.TryGetValue(Thread.CurrentThread,out Result))
{
Result = ValueFactory.Invoke();
InternalDict.Add(Thread.CurrentThread, Result);
}
}
return Result;
}
set
{
lock (SyncRoot)
{
if (InternalDict.ContainsKey(Thread.CurrentThread))
{
InternalDict[Thread.CurrentThread] = value;
}
else
{
InternalDict.Add(Thread.CurrentThread, value);
}
}
}
}
public bool IsValueCreated
{
get
{
lock (SyncRoot)
{
return InternalDict.ContainsKey(Thread.CurrentThread);
}
}
}
public void DisposeThreadCompletedValues()
{
lock (SyncRoot)
{
List<Thread> CompletedThreads;
CompletedThreads = new List<Thread>();
foreach (Thread ThreadInstance in InternalDict.Keys)
{
if (!ThreadInstance.IsAlive)
{
CompletedThreads.Add(ThreadInstance);
}
}
foreach (Thread ThreadInstance in CompletedThreads)
{
InternalDict[ThreadInstance].Dispose();
InternalDict.Remove(ThreadInstance);
}
}
}
private int m_CheckEverySeconds;
private int CheckEverySeconds
{
get
{
return m_CheckEverySeconds;
}
}
private Thread WatcherThread;
private void InternalMonitor()
{
while (!IsDisposed)
{
System.Threading.Thread.Sleep(CheckEverySeconds);
DisposeThreadCompletedValues();
}
}
private bool IsDisposed = false;
public void Dispose()
{
if (!IsDisposed)
{
IsDisposed = true;
DoDispose();
}
}
private void DoDispose()
{
if (WatcherThread != null)
{
WatcherThread.Abort();
}
//InternalDict.Values.ToList().ForEach(Value => Value.Dispose());
foreach (T Value in InternalDict.Values)
{
Value.Dispose();
}
InternalDict.Clear();
m_InternalDict = null;
m_ValueFactory = null;
GC.SuppressFinalize(this);
}
}

Possible to turn callback calls into IEnumerable

I'm writing a wrapper around a 3rd party library, and it has a method to scan the data it manages. The method takes a callback method that it calls for each item in the data that it finds.
e.g. The method is essentially: void Scan(Action<object> callback);
I want to wrap it and expose a method like IEnumerable<object> Scan();
Is this possible without resorting to a separate thread to do the actual scan and a buffer?

You can do this quite simply with Reactive:
class Program
{
static void Main(string[] args)
{
foreach (var x in CallBackToEnumerable<int>(Scan))
Console.WriteLine(x);
}
static IEnumerable<T> CallBackToEnumerable<T>(Action<Action<T>> functionReceivingCallback)
{
return Observable.Create<T>(o =>
{
// Schedule this onto another thread, otherwise it will block:
Scheduler.Later.Schedule(() =>
{
functionReceivingCallback(o.OnNext);
o.OnCompleted();
});
return () => { };
}).ToEnumerable();
}
public static void Scan(Action<int> act)
{
for (int i = 0; i < 100; i++)
{
// Delay to prove this is working asynchronously.
Thread.Sleep(100);
act(i);
}
}
}
Remember that this doesn't take care of things like cancellation, since the callback method doesn't really allow it. A proper solution would require work on the part of the external library.

You should investigate the Rx project — this allows an event source to be consumed as an IEnumerable.
I'm not sure if it allows vanilla callbacks to be presented as such (it's aimed at .NET events) but it would be worth a look as it should be possible to present a regular callback as an IObservable.

Here is a blocking enumerator (the Scan method needs to run in a separate thread)
public class MyEnumerator : IEnumerator<object>
{
private readonly Queue<object> _queue = new Queue<object>();
private ManualResetEvent _event = new ManualResetEvent(false);
public void Callback(object value)
{
lock (_queue)
{
_queue.Enqueue(value);
_event.Set();
}
}
public void Dispose()
{
}
public bool MoveNext()
{
_event.WaitOne();
lock (_queue)
{
Current = _queue.Dequeue();
if (_queue.Count == 0)
_event.Reset();
}
return true;
}
public void Reset()
{
_queue.Clear();
}
public object Current { get; private set; }
object IEnumerator.Current
{
get { return Current; }
}
}
static void Main(string[] args)
{
var enumerator = new MyEnumerator();
Scan(enumerator.Callback);
while (enumerator.MoveNext())
{
Console.WriteLine(enumerator.Current);
}
}
You could wrap it in a simple IEnumerable<Object>, but I would not recommend it. IEnumerable lists implies that you can run multiple enumerators on the same list, which you can't in this case.

How about this one:
IEnumerable<Object> Scan()
{
List<Object> objList = new List<Object>();
Action<Object> action = (obj) => { objList.Add(obj); };
Scan(action);
return objList;
}

Take a look at the yield keyword -- which will allow you to have a method that looks like an IEnumerable but which actually does processing for each return value.