We Keep Coding

Need help tweaking c# multi-threaded routine

I've created a windows service which runs a multi-threaded routine on a machine with 24 cores, 48 virtual, using Parallel.ForEach. This service, which has been running great in a production environment, bulk copies data into an SQL Server database. Currently it does this very well, around 6000 inserts per second, but I believe it can be tweaked. Below is part of the code I am using; there's an example of current functionality and proposed changes for tweaking. As can be seen from the code, currently a lock is taken for every call to Add, which I believe makes the Parallel.ForEach somewhat non-parallel. So I'm looking for a "fix"; and hoping my new method, also defined in the code, would do the trick.
public class MainLoop
{
public void DoWork()
{
var options = new ParallelOptions
{
MaxDegreeOfParallelism = System.Environment.ProcessorCount * 2
};
var workQueueManager = new ObjWorkQueueManager(queueSize: 1000);
// ignore the fact that this while loop would be a never ending loop,
// there's other logic not shown here that exits the loop!
while (true)
{
ICollection<object> work = GetWork();
Parallel.ForEach(work, options, (item) =>
{
workQueueManager.AddOLD(item);
});
}
}
private ICollection<object> GetWork()
{
// return list of work from some arbitrary source
throw new NotImplementedException();
}
}
public class ObjWorkQueueManager
{
private readonly int _queueSize;
private ObjDataReader _queueDataHandler;
private readonly object _sync;
public ObjWorkQueueManager(int queueSize)
{
_queueSize = queueSize;
_queueDataHandler = new ObjDataReader(queueSize);
_sync = new object();
}
// current Add method works great, but blocks with EVERY call
public void AddOLD(object value)
{
lock (_sync)
{
if (_queueDataHandler.Add(value) == _queueSize)
{
// create a new thread to handle copying the queued data to repository
Thread t = new Thread(SaveQueuedData);
t.Start(_queueDataHandler);
// start a new queue
_queueDataHandler = new ObjDataReader(_queueSize);
}
}
}
// hoping for a new Add method to work better by blocking only
// every nth call where n = _queueSize
public void AddNEW(object value)
{
int queued;
if ((queued = _queueDataHandler.Add(value)) >= _queueSize)
{
lock (_sync)
{
if (queued == _queueSize)
{
Thread t = new Thread(SaveQueuedData);
t.Start(_queueDataHandler);
}
}
}
else if (queued == 0)
{
lock (_sync)
{
_queueDataHandler = new ObjDataReader(_queueSize);
AddNEW(value);
}
}
}
// this method will Bulk Copy data into an SQL DB
private void SaveQueuedData(object o)
{
// do something with o as ObjDataReader
}
}
// implements IDataReader, Read method of IDataReader dequeues from _innerQueue
public class ObjDataReader
{
private readonly int _capacity;
private Queue<object> _innerQueue;
public ObjDataReader(int capacity)
{
_capacity = capacity;
_innerQueue = new Queue<object>(capacity);
}
public int Add(object value)
{
if (_innerQueue.Count < _capacity)
{
_innerQueue.Enqueue(value);
return _innerQueue.Count;
}
return 0;
}
}

Performance of ConcurrentBag, many reads, rare modifications

I'm trying to build a model where there will me multiple reads of an entire collection and rare additions and modifications to it.
I thought I might use the ConcurrentBag in .NET as I've read the documentation and it's supposed to be good for concurrent reads and writes.
The code would look like this:
public class Cache
{
ConcurrentBag<string> cache = new ConcurrentBag<string>();
// this method gets called frequently
public IEnumerable<string> GetAllEntries()
{
return cache.ToList();
}
// this method gets rarely called
public void Add(string newEntry)
{
// add to concurrentBag
}
public void Remove(string entryToRemove)
{
// remove from concurrent bag
}
}
However, I've decompiled the ConcurrentBag class and on theGetEnumerator there's always a lock taken, which means any call to GetAllEntries will lock the entire collection and it will not perform.
I'm thinking to get around this and code it in this manner instead, using a list.
public class Cache
{
private object guard = new object();
IList<string> cache = new List<string>();
// this method gets called frequently
public IEnumerable<string> GetAllEntries()
{
var currentCache = cache;
return currentCache;
}
// this method gets rarely called
public void Add(string newEntry)
{
lock (guard)
{
cache.Add(newEntry);
}
}
public void Remove(string entryToRemove)
{
lock (guard)
{
cache.Remove(entryToRemove);
}
}
}
Since the Add and Remove are rarely called I don't care too much about locking the access to the list there. On Get I might get a stale version of the list, but again I don't care, it will be fine for the next request.
Is the second implementation a good way to go?
EDIT
I've run a quick performance test and the results are the following:
Setup: populated the in memory collection with 10000 strings.
Action: GetAllEntries concurrently 50000 times.
Result:
00:00:35.2393871 to finish operation using ConcurrentBag (first implementation)
00:00:00.0036959 to finish operation using normal list (second implementation)
Code below:
class Program
{
static void Main(string[] args)
{
// warmup caches and stopwatch
var cacheWitBag = new CacheWithBag();
var cacheWitList = new CacheWithList();
cacheWitBag.Add("abc");
cacheWitBag.GetAllEntries();
cacheWitList.Add("abc");
cacheWitList.GetAllEntries();
var sw = new Stopwatch();
// warmup stowtach as well
sw.Start();
// initialize caches (rare writes so no real reason to measure here
for (int i =0; i < 50000; i++)
{
cacheWitBag.Add(new Guid().ToString());
cacheWitList.Add(new Guid().ToString());
}
sw.Stop();
// measure
var program = new Program();
sw.Start();
program.Run(cacheWitBag).Wait();
sw.Stop();
Console.WriteLine(sw.Elapsed);
sw.Restart();
program.Run2(cacheWitList).Wait();
sw.Stop();
Console.WriteLine(sw.Elapsed);
}
public async Task Run(CacheWithBag cache1)
{
List<Task> tasks = new List<Task>();
for (int i = 0; i < 10000; i++)
{
tasks.Add(Task.Run(() => cache1.GetAllEntries()));
}
await Task.WhenAll(tasks);
}
public async Task Run2(CacheWithList cache)
{
List<Task> tasks = new List<Task>();
for (int i = 0; i < 10000; i++)
{
tasks.Add(Task.Run(() => cache.GetAllEntries()));
}
await Task.WhenAll(tasks);
}
public class CacheWithBag
{
ConcurrentBag<string> cache = new ConcurrentBag<string>();
// this method gets called frequently
public IEnumerable<string> GetAllEntries()
{
return cache.ToList();
}
// this method gets rarely called
public void Add(string newEntry)
{
cache.Add(newEntry);
}
}
public class CacheWithList
{
private object guard = new object();
IList<string> cache = new List<string>();
// this method gets called frequently
public IEnumerable<string> GetAllEntries()
{
var currentCache = cache;
return currentCache;
}
// this method gets rarely called
public void Add(string newEntry)
{
lock (guard)
{
cache.Add(newEntry);
}
}
public void Remove(string entryToRemove)
{
lock (guard)
{
cache.Remove(entryToRemove);
}
}
}
}
}

To improve on InBetween's solution:
class Cache
{
ImmutableHashSet<string> cache = ImmutableHashSet.Create<string>();
public IEnumerable<string> GetAllEntries()
{
return cache;
}
public void Add(string newEntry)
{
ImmutableInterlocked.Update(ref cache, (set,item) => set.Add(item), newEntry);
}
public void Remove(string entryToRemove)
{
ImmutableInterlocked.Update(ref cache, (set,item) => set.Remove(item), newEntry);
}
}
This performs only atomic operations (no locking) and uses the .NET Immutable types.

In your current scenario, where Add and Remove are rarely called, I'd consider the following approach:
public class Cache
{
private object guard = new object();
var cache = new SomeImmutableCollection<string>();
// this method gets called frequently
public IEnumerable<string> GetAllEntries()
{
return cache;
}
// this method gets rarely called
public void Add(string newEntry)
{
lock (guard)
{
cache = cache.Add(newEntry);
}
}
public void Remove(string entryToRemove)
{
lock (guard)
{
cache = cache.Remove(entryToRemove);
}
}
}
The fundamental change here is that cache now is an immutable collection, which means it can't change....ever. So concurrency problems with the collection itself simply disappear, something that can't change is inherently thread safe.
Also, depending on how rare calls to Add and Remove are you can even consider removing the lock in both of them because all its doing now is avoiding a race between Add and Remove and a potential loss of a cache update. If that scenario is very very improbable you could get away with it. That said, I very much doubt the few nanoseconds an uncontended lock takes is a relevant factor here to actually consider this ;)
SomeImmutableCollection can be any of the collections found in System.Collections.Immutable that better suit your needs.

Instead of a 'lock' on a guard object to protect a simple container you should consider the 'ReaderWriterLockSlim' which is optimized and very performant for the read/write scenario : multiple readers are allowed at same time but only one writer is allowed and blocks other readers/writers. It is very useful in your scenario where you read a lot but write only few.
Please note you can be a reader and then, for some reason, decide to become a writer (upgrade the slim lock) in your "reading" code.

Writing errors to a text file in a Web Service

I have a web service that has a default ErrorLog method for adding Logs to a Db Table. If there's an exception is caught on the ErrorLog Add or the Stored Procedure returns that it failed to add. I'd like to write the error to a textfile on the server, (in theory this should never happen).
Now before actually implementing this, I realize there's a good chance that multiple people could get an error, all of them fail, and all of them try to write to the same text file.
How can I implement a queue on the service, so that the messages get added to the queue and another service / job loops through this queue and adds the errors to the file?
I have tried looking for examples, most of them are very basic. The only thing I really want to know is how I should keep track of the queue? Do I simply create a static class?
Would the below work?
public class ErrorLogging
{
public ErrorLogging(Error error)
{
if (ErrorLoggingQueue._GlobalQueue == null)
{
ErrorLoggingQueue._GlobalQueue = new Queue<Error>();
}
ErrorLoggingQueue._GlobalQueue.Enqueue(error);
}
}
public static class ErrorLoggingQueue
{
public static Queue<Error> _GlobalQueue;
}
// Assume that this class/method gets called every x minutes or seconds from a job or something.
public class JobClass
{
public void WriteErrors()
{
if (ErrorLoggingQueue._GlobalQueue != null)
{
while (ErrorLoggingQueue._GlobalQueue.Count != 0)
{
Error error = (Error)ErrorLoggingQueue._GlobalQueue.Dequeue();
// Do stuff here
}
}
}
}

Yes, static variable with a Queue will work and would be shared between requests. Just add locking for enqueue and dequeue to make those operation atomic. Something along these lines:
class YourWebservice
{
static Queue<Error> _GlobalQueue = new Queue<Error>();
static readonly object queueLock = new object();
static Thread errorLogger;
public void SomeWebserviceMethod()
{
//Some code...
//.
//.
//Here we want to log an error
EnqueueError(new Error());
}
private void EnqueueError(Error err )
{
lock(queueLock)
{
_GlobalQueue?.Enqueue(err);
if ( errorLogger==null || !(errorLogger?.IsAlive ?? false) )
{
errorLogger = new Thread(new ThreadStart(WriteErrors));
errorLogger?.Start();
}
}
}
private static Error DequeueError()
{
try
{
lock (queueLock)
{
return _GlobalQueue?.Dequeue();
}
}
catch(Exception)
{
//if we got here it means queue is empty.
}
return null;
}
private static void WriteErrors()
{
Error error = DequeueError();
while (error!=null)
{
//Log error here
//...
//..
error = DequeueError();
}
}
}

Locking a thread by user

I need a way to lock c# threads by user
I have my data object and I create new instance for every user.
Every user has several threads that use this object and in I.O. operations I want to lock this object instance for this user only.
Using simple Lock {} is locking all the object instances, there for blocking other user.
I need some simple solution.
Edit
I build new instance of MyDataObj per user;
Then run job that updating some data in MyDataObj every minute;
Using lockObj as lock, lock the data to all the users (Although it's not static Variables)
I need only to lock the data to the current user
this is the code sample
public sealed class MyDataObj
{
private static readonly Dictionary<object, MyDataObj> _instances = new Dictionary<object, MyDataObj>();
public object lockObj = new object();
public bool jobRunning = false;
private string data = string.Empty;
//// --------- constractor -------------------
private MyDataObj(int key)
{
LoadMyDataObj(key);
}
public static MyDataObj GetInstance(int key)
{
lock (_instances)
{
MyDataObj instance;
if (_instances.TryGetValue(key, out instance))
{
instance = _instances[key];
return instance;
}
instance = new MyDataObj(key);
return instance;
}
}
private void LoadMyDataObj(int key)
{
// get the data from db
}
public void UpdateMyData(string newData)
{
lock (lockObj)
{
this.data = newData;
}
}
public string ReadMyData()
{
lock (lockObj)
{
return this.data;
}
}
public class ActionObject
{
MyDataObj myDataObj;
int UserID;
//// --------- constractor -------------------
public ActionObject(int userid)
{
this.UserID = userid;
myDataObj = MyDataObj.GetInstance(userid);
if (!myDataObj.jobRunning)
{
jobs jbs = new jobs(myDataObj);
System.Threading.Thread RunJob = new System.Threading.Thread(new System.Threading.ThreadStart(jbs.dominutesAssignment));
RunJob.Start();
myDataObj.jobRunning = true;
}
}
public ActionObject()
{
myDataObj = MyDataObj.GetInstance(this.UserID);
myDataObj.UpdateMyData("some data");
}
}
public class jobs
{
MyDataObj myDataObj = null;
public jobs(MyDataObj grp)
{
this.myDataObj = grp;
}
public void dominutesAssignment()
{
while (true)
{
myDataObj.ReadMyData();
System.Threading.Thread.Sleep(1000);
}
}
}
}

I need a way to lock c# threads by user. I have my data object and I create new instance for every user
Create one lock per user. Or if the user exists longer than the threads: Use the user object as the lock.
lock (userOrTheUserObject)
{
//Do some op
}
Every user has several threads that use this object and in I.O. operations
That sounds more like you should use asynchronous IO instead of creating several threads (which will be less effecient)
I want to lock this object instance for this user only. Using simple Lock {} is locking all the object instances, there for blocking other user.
If the object is shared between all users you HAVE to lock it using lock. The lock won't be very effective otherwise. The other object is to redesign the object to now be shared.
I need some simple solution.
There are no simple threading solutions.

You can use Monitor. In this sample anyone but user 1 can execute DoIt method concurrently. While user 1 executing DoIt no one can enter it. A weak point is if user 1 tries to execute DoIt when user 2 already executing it, user 2 continues its execution. Also you have to handle exceptions properly otherwise there may be dead locks.
private static readonly object lockObj = new Object();
public void Do(int userId)
{
Monitor.Enter(lockObj);
if (userId != 1)
Monitor.Exit(lockObj);
try
{
DoIt();
}
finally
{
if (userId == 1)
Monitor.Exit(lockObj);
}
}
public void DoIt()
{
// Do It
}

Better way to handle read-only access to state with another thread?

This is a design question, not a bug fix problem.
The situation is this. I have a lot of collections and objects contained in one class. Their contents are only changed by a single message handler thread. There is one other thread which is doing rendering. Each frame it iterates through some of these collections and draws to the screen based on the value of these objects. It does not alter the objects in any way, it is just reading their values.
Now when the rendering is being done, if any of the collections are altered, my foreach loops in the rendering method fail. How should I make this thread safe? Edit: So I have to lock the collections outside each foreach loop I run on them. This works, but it seems like a lot of repetitive code to solve this problem.
As a short, contrived example:
class State
{
public object LockObjects;
public List<object> Objects;
// Called by message handler thread
void HandleMessage()
{
lock (LockObjects)
{
Objects.Add(new object());
}
}
}
class Renderer
{
State m_state;
// Called by rendering thread
void Render()
{
lock (m_state.LockObjects)
{
foreach (var obj in m_state.Objects)
{
DrawObject(obj);
}
}
}
}
This is all well and good, but I'd rather not put locks on all my state collections if there's a better way. Is this "the right" way to do it or is there a better way?

The better way is to use begin/end methods and separated lists for your both threads and synchronization using auto events for example. It will be lock-free to your message handler thread and enables you to have a lot of render/message handler threads:
class State : IDisposable
{
private List<object> _objects;
private ReaderWriterLockSlim _locker;
private object _cacheLocker;
private List<object> _objectsCache;
private Thread _synchronizeThread;
private AutoResetEvent _synchronizationEvent;
private bool _abortThreadToken;
public State()
{
_objects = new List<object>();
_objectsCache = new List<object>();
_cacheLocker = new object();
_locker = new ReaderWriterLockSlim();
_synchronizationEvent = new AutoResetEvent(false);
_abortThreadToken = false;
_synchronizeThread = new Thread(Synchronize);
_synchronizeThread.Start();
}
private void Synchronize()
{
while (!_abortThreadToken)
{
_synchronizationEvent.WaitOne();
int objectsCacheCount;
lock (_cacheLocker)
{
objectsCacheCount = _objectsCache.Count;
}
if (objectsCacheCount > 0)
{
_locker.EnterWriteLock();
lock (_cacheLocker)
{
_objects.AddRange(_objectsCache);
_objectsCache.Clear();
}
_locker.ExitWriteLock();
}
}
}
public IEnumerator<object> GetEnumerator()
{
_locker.EnterReadLock();
foreach (var o in _objects)
{
yield return o;
}
_locker.ExitReadLock();
}
// Called by message handler thread
public void HandleMessage()
{
lock (_cacheLocker)
{
_objectsCache.Add(new object());
}
_synchronizationEvent.Set();
}
public void Dispose()
{
_abortThreadToken = true;
_synchronizationEvent.Set();
}
}
Or (the simpler way) you can use ReaderWriteerLockSlim (Or just locks if you sure you have only one reader) like in the following code:
class State
{
List<object> m_objects = new List<object>();
ReaderWriterLockSlim locker = new ReaderWriterLockSlim();
public IEnumerator<object> GetEnumerator()
{
locker.EnterReadLock();
foreach (var o in Objects)
{
yield return o;
}
locker.ExitReadLock();
}
private List<object> Objects
{
get { return m_objects; }
set { m_objects = value; }
}
// Called by message handler thread
public void HandleMessage()
{
locker.EnterWriteLock();
Objects.Add(new object());
locker.ExitWriteLock();
}
}

Humm... have you tried with a ReaderWriterLockSlim ? Enclose each conllection with one of this, and ensure you start a read or write operation each time you access it.