We Keep Coding

c# multiple threads waiting for a ManualResetEvent

I'm messing around with multithreading and making some sort of task engine. The idea is that the engine can have a configurable amount of threads waiting and when a new task arrives the first free thread picks it up and executes it.
The problem is that something 2 threads pickup the same task somehow. I looked it through and I think that this code should work but obviously it doesn't. If I add the 10ms sleep where it is now commented out it works, but I'm not sure I understand why. It looks like the .Reset() function returns before it actually resets the event?
Can somebody explain? Is there a better way to let only a single thread continue when there are multiple waiting?
Thanks
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace TaskTest
{
public class Engine
{
private ManualResetEvent taskEvent;
private ConcurrentQueue<Task> tasks;
private bool running;
private List<Thread> threads;
private int threadAmount;
private int threadsBusy = 0;
public Engine(int amountOfThreads)
{
taskEvent = new ManualResetEvent(false);
tasks = new ConcurrentQueue<Task>();
threads = new List<Thread>();
threadAmount = amountOfThreads;
}
public void Start()
{
running = true;
for (var i = 0; i < threadAmount; i++)
{
var thread = new Thread(Process);
thread.Name = "Thread " + i;
threads.Add(thread);
thread.Start();
}
}
public void Stop()
{
running = false;
taskEvent.Set();
threads.ForEach(t => t.Join());
}
private void Process()
{
while (running)
{
lock (taskEvent)
{
// Lock it so only a single thread is waiting on the event at the same time
taskEvent.WaitOne();
taskEvent.Reset();
//Thread.Sleep(10);
}
if (!running)
{
taskEvent.Set();
return;
}
threadsBusy += 1;
if (threadsBusy > 1)
Console.WriteLine("Failed");
Task task;
if (tasks.TryDequeue(out task))
task.Execute();
threadsBusy -= 1;
}
}
public void Enqueue(Task t)
{
tasks.Enqueue(t);
taskEvent.Set();
}
}
}
EDIT
Rest of the code:
namespace TaskTest
{
public class Start
{
public static void Main(params string[] args)
{
var engine = new Engine(4);
engine.Start();
while (true)
{
Console.Read();
engine.Enqueue(new Task());
}
}
}
}
namespace TaskTest
{
public class Task
{
public void Execute()
{
Console.WriteLine(Thread.CurrentThread.Name);
}
}
}

When using Console.Read() on a key press, two characters are read from the input. You should use Console.ReadLine() instead.
Note that your code can be simplified a lot by using a BlockingCollection to handle the synchronization:
public class Engine
{
private BlockingCollection<Task> tasks;
private List<Thread> threads;
private int threadAmount;
public Engine(int amountOfThreads)
{
tasks = new BlockingCollection<Task>();
threads = new List<Thread>();
threadAmount = amountOfThreads;
}
public void Start()
{
for (var i = 0; i < threadAmount; i++)
{
var thread = new Thread(Process);
thread.Name = "Thread " + i;
threads.Add(thread);
thread.Start();
}
}
public void Stop()
{
tasks.CompleteAdding();
threads.ForEach(t => t.Join());
}
private void Process()
{
foreach (var task in tasks.GetConsumingEnumerable())
{
task.Execute();
}
}
public void Enqueue(Task t)
{
tasks.Add(t);
}
}

C# multiple reusable worker threads

I am trying to build some kind of multi-thread game engine with reusable worker threads. That means the workers are running an endless loop and, after every iteration, call a method that suspends the thread with an AutoResetEvent until the next iteration is started from outside. Thet works fine.
But then I want to wait for all of the worker threads to get into the waiting state. But the last thread is always ignored in the first step.
This is my code:
class WorkerThread
{
private readonly Game game;
private readonly Task task;
private readonly AutoResetEvent waitEvent;
private readonly AutoResetEvent finishEvent;
public string Message { get; set; }
public bool Active { get; private set; }
public WorkerThread(Game game)
{
this.game = game;
waitEvent = new AutoResetEvent(false);
finishEvent = new AutoResetEvent(false);
task = new Task(startTask);
}
public void Start()
{
task.Start();
}
public void PerformFrame()
{
finishEvent.Reset();
waitEvent.Set();
}
public void WaitForFrameToFinish()
{
finishEvent.WaitOne();
}
private void startTask()
{
WaitFrame();
run();
}
long sum;
private void run()
{
while (true)
{
sum = 0;
for (int i = 0; i < 1000000; i++)
{
sum += i;
}
Console.WriteLine(Message);
WaitFrame();
}
}
private void WaitFrame()
{
Active = false;
finishEvent.Set();
waitEvent.WaitOne();
Active = true;
}
}
class Game
{
private readonly List<WorkerThread> workers = new List<WorkerThread>();
public Game()
{
workers.Add(new WorkerThread(this) { Message = "Worker 1" });
workers.Add(new WorkerThread(this) { Message = "Worker 2" });
workers.Add(new WorkerThread(this) { Message = "Worker 3" });
workers.Add(new WorkerThread(this) { Message = "Worker 4" });
workers.Add(new WorkerThread(this) { Message = "Worker 5" });
workers.Add(new WorkerThread(this) { Message = "Worker 6" });
}
public void Start()
{
foreach (WorkerThread worker in workers)
{
worker.Start();
}
}
public void OneFrame()
{
//start every worker
foreach (WorkerThread worker in workers)
{
worker.PerformFrame();
}
//wait for the workers to finish
foreach (WorkerThread worker in workers)
{
worker.WaitForFrameToFinish();
}
//check if there are workers still running
foreach (WorkerThread worker in workers)
{
if (worker.Active)
{
Console.WriteLine(worker.Message + " still active");
}
}
Console.WriteLine("Frame finished.\n");
}
}
Has anyone an idea why the Game doen't wait for the last WorkerThread do finish its task?
EDIT: Using a ManualResetEventSlim instead of an AutoResetEvent solves the problem. But I still don't understand why the above doesn't work.
EDIT 2: I found the answer and posted it below.

I just found the solution. startTask uses the WaitFrame method that sets the finishEvent. That way WaitForFrameToFinish will not wait for the worker on the first iteration because finishEvent is already set.
If you do it like this, it works fine:
private void startTask()
{
waitEvent.WaitOne();
waitEvent.Reset();
finishEvent.Reset();
run();
}

Joining a thread started with StartNew()

When using the StartNew() method to kick off a process on a new thread, I need to figure out how to make another call into this object in that same thread (I assume this would be some sort of Join operation?).
The following example is dumbed down to illustrate the meat of what I am trying to do. I am well aware it is severely lacking in basic concurrency considerations. But I didn't want to cloud the code with all of that logic, so please forgive me on that.
The following console app shows what I am trying to accomplish. Assume on the StartNew() call a new thread with ID 9976 is created and the method invoked there. I would like the subsequent call to ProcessImmediate() in the file system watcher change event handler to be made on thread 9976 as well. As it stands, the call would share the same thread that is used for the file system watcher change event.
Can this be done, and if so, how?
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var runner = new Runner();
runner.Run();
Console.ReadKey();
}
}
public class Runner
{
private Activity _activity = null;
private FileSystemWatcher _fileSystemWatcher;
public void Run()
{
_activity = new Activity();
// start activity on a new thread
Task.Factory.StartNew(() => _activity.Go());
_fileSystemWatcher = new FileSystemWatcher();
_fileSystemWatcher.Filter = "*.watcher";
_fileSystemWatcher.Path = "c:\temp";
_fileSystemWatcher.Changed += FileSystemWatcher_Changed;
_fileSystemWatcher.EnableRaisingEvents = true;
}
private void FileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
// WANT TO CALL THIS FOR ACTIVITY RUNNING ON PREVIOUSLY CALLED THREAD
_activity.ProcessImmediate();
}
}
public class Activity
{
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
System.Threading.Thread.Sleep(2000);
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate()
{
// for purposes of this example, assume that Go is magically in its sleep state when ProcessImmediate is called
DoSomethingInteresting();
}
public bool Stop { get; set; }
}
}
* UPDATE *
Thanks for the excellent responses. I took Mike's suggestion and implemented it for my console app. Below is the full working code which also includes the use of a cancellation token. I post this in case someone else might find it useful.
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var runner = new Runner();
runner.Run();
Console.ReadKey();
runner.Stop();
Console.ReadKey();
}
}
public class Runner
{
private Activity _activity = null;
private FileSystemWatcher _fileSystemWatcher;
private CancellationTokenSource _cts = new CancellationTokenSource();
public void Stop() { _cts.Cancel(); }
public void Run()
{
_activity = new Activity();
// start activity on a new thread
var task = new Task(() => _activity.Go(_cts.Token), _cts.Token, TaskCreationOptions.LongRunning);
task.Start();
_fileSystemWatcher = new FileSystemWatcher();
_fileSystemWatcher.Filter = "*.watcher";
_fileSystemWatcher.Path = "C:\\Temp\\FileSystemWatcherPath";
_fileSystemWatcher.Changed += FileSystemWatcher_Changed;
_fileSystemWatcher.EnableRaisingEvents = true;
}
private void FileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
// WANT TO CALL THIS FOR ACTIVITY RUNNING ON PREVIOUSLY CALLED THREAD
_activity.ProcessImmediate();
}
}
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
public void Go(CancellationToken ct)
{
Thread.CurrentThread.Name = "Go";
while (!ct.IsCancellationRequested)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(5000);
}
Console.WriteLine("Exiting");
}
protected virtual void DoSomethingInteresting()
{
Console.WriteLine(string.Format("Doing Something Interesting on thread {0}", Thread.CurrentThread.ManagedThreadId));
}
public void ProcessImmediate()
{
// for purposes of this example, assume that Go is magically in its sleep state when ProcessImmediate is called
_processing.Set();
}
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}
}

First, you should use TaskCreationOptions.LongRunning if you are creating a task that will not complete quickly. Second, use an AutoResetEvent to signal the waiting thread to wake up. Note that below ProcessImmediate will return before DoSomethingInteresting has completed running on the other thread. Example:
using System.Threading;
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(2000);
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate()
{
_processing.Set();
}
public bool Stop { get; set; }
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}

User mike has given a better solution, which will be appropriate when you like to call the same method immediately. If you want to call a different methods immediately I'll expand mike's answer to achieve that.
using System.Threading;
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
private ConcurrentQueue<Action> actionsToProcess = new ConcurrentQueue<Action>();
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(2000);
while(!actionsToProcess.IsEmpty)
{
Action action;
if(actionsToProcess.TryDeque(out action))
action();
}
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate(Action action)
{
actionsToProcess.Enqueue(action);
_processing.Set();
}
public bool Stop { get; set; }
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}

To execute different methods on the same thread you can use a message loop that dispatches incoming requests. A simple option would be to use the event loop scheduler of the Reactive Extensions and to "recursively" schedule your Go() function - if in the mean time a different operation is scheduled it would be processed before the next Go() operation.
Here is a sample:
class Loop
: IDisposable
{
IScheduler scheduler = new EventLoopScheduler();
MultipleAssignmentDisposable stopper = new MultipleAssignmentDisposable();
public Loop()
{
Next();
}
void Next()
{
if (!stopper.IsDisposed)
stopper.Disposable = scheduler.Schedule(Handler);
}
void Handler()
{
Thread.Sleep(1000);
Console.WriteLine("Handler: {0}", Thread.CurrentThread.ManagedThreadId);
Next();
}
public void Notify()
{
scheduler.Schedule(() =>
{
Console.WriteLine("Notify: {0}", Thread.CurrentThread.ManagedThreadId);
});
}
public void Dispose()
{
stopper.Dispose();
}
}
static void Main(string[] args)
{
using (var l = new Loop())
{
Console.WriteLine("Press 'q' to quit.");
while (Console.ReadKey().Key != ConsoleKey.Q)
l.Notify();
}
}

Frequent lag spikes in multi-threaded code

I'm attempting to make my simple C# graphics library multi-threaded. However, after the introduction of this code:
/* foreach (IAffector affector in affectorLookup.Values)
affector.Update(timestep); */
taskManager.Value = timestep; taskManager.Start();
foreach (IAffector affector in affectorLookup.Values)
taskManager.AddToQueue(affector.Update);
taskManager.StopWhenDone();
taskManager.Wait();
the simulation starts experiencing sharp lag-spikes, which seem to originate in TaskHandler.Run (I can't tell for sure, because adding the previous code makes my code profiler ignore anything outside TaskHandler.Run).
The task manager:
public class TaskManager
{
public delegate void MethodDel(float timestep);
private Queue<MethodDel> queue;
private List<TaskHandler> handlers;
private float value;
public float Value
{
get
{
return value;
}
set
{
this.value = value;
}
}
public TaskManager()
{
this.queue = new Queue<MethodDel>();
this.handlers = new List<TaskHandler>(System.Environment.ProcessorCount);
for (int t = 0; t < this.handlers.Capacity; ++t)
this.handlers.Add(new TaskHandler(this));
this.value = 0;
}
public void Start()
{
foreach (var handler in handlers)
handler.Start();
}
public void Stop()
{
lock (queue)
queue.Clear();
foreach (var handler in handlers)
handler.StopWhenDone();
}
public void StopWhenDone()
{
foreach (var handler in handlers)
handler.StopWhenDone();
}
public void AddToQueue(MethodDel method)
{
lock (queue)
queue.Enqueue(method);
}
public bool GetFromQueue(out MethodDel method)
{
lock (queue)
{
if (queue.Count == 0) { method = null; return false; }
method = queue.Dequeue();
return true;
}
}
public int GetQueueCount()
{
return queue.Count;
}
internal void Wait()
{
// Have to wait for them one at a time because the main thread is STA.
WaitHandle[] waitHandles = new WaitHandle[1];
// for (int t = 0; t < handlers.Count; ++t)
// waitHandles[t] = handlers[t].WaitHandle;
// WaitHandle.WaitAll(waitHandles);
for (int t = 0; t < handlers.Count; ++t)
{ waitHandles[0] = handlers[t].WaitHandle; WaitHandle.WaitAll(waitHandles); }
}
}
And the task handler:
public class TaskHandler
{
private TaskManager manager;
private Thread thread;
private bool stopWhenDone;
private ManualResetEvent waitHandle;
public ManualResetEvent WaitHandle
{
get
{
return waitHandle;
}
}
public TaskHandler(TaskManager manager)
{
this.manager = manager;
}
public void Start()
{
waitHandle = new ManualResetEvent(false);
stopWhenDone = false;
thread = new Thread(Run);
thread.IsBackground = true;
thread.SetApartmentState(ApartmentState.MTA);
thread.Start();
}
public void StopWhenDone()
{
this.stopWhenDone = true;
}
// Possible source of slowdown
private void Run()
{
TaskManager.MethodDel curMethod;
while (!stopWhenDone || manager.GetQueueCount() > 0)
{
if (manager.GetFromQueue(out curMethod))
{
curMethod(manager.Value);
}
}
waitHandle.Set();
}
}

Starting a thread is a heavy operation. Not sure if it's as heavy as you are experiencing, but that could be it. Also, having all your processing run parallel can be putting a big strain on your system with possibly little benefit...

I'm going to venture that the spikes have something to do with waitHandle.Set();
I like the overall design, but I have not used WaitHandle before, so I am unsure how this interacts with your design.

How to share data between different threads In C# using AOP?

How to share data between different threads In C# without using the static variables?
Can we create a such machanism using attribute?
Will Aspect oriented programming help in such cases?
To acheive this all the different threads should work on single object?

You can't beat the simplicity of a locked message queue. I say don't waste your time with anything more complex.
Read up on the lock statement.
lock
EDIT
Here is an example of the Microsoft Queue object wrapped so all actions against it are thread safe.
public class Queue<T>
{
/// <summary>Used as a lock target to ensure thread safety.</summary>
private readonly Locker _Locker = new Locker();
private readonly System.Collections.Generic.Queue<T> _Queue = new System.Collections.Generic.Queue<T>();
/// <summary></summary>
public void Enqueue(T item)
{
lock (_Locker)
{
_Queue.Enqueue(item);
}
}
/// <summary>Enqueues a collection of items into this queue.</summary>
public virtual void EnqueueRange(IEnumerable<T> items)
{
lock (_Locker)
{
if (items == null)
{
return;
}
foreach (T item in items)
{
_Queue.Enqueue(item);
}
}
}
/// <summary></summary>
public T Dequeue()
{
lock (_Locker)
{
return _Queue.Dequeue();
}
}
/// <summary></summary>
public void Clear()
{
lock (_Locker)
{
_Queue.Clear();
}
}
/// <summary></summary>
public Int32 Count
{
get
{
lock (_Locker)
{
return _Queue.Count;
}
}
}
/// <summary></summary>
public Boolean TryDequeue(out T item)
{
lock (_Locker)
{
if (_Queue.Count > 0)
{
item = _Queue.Dequeue();
return true;
}
else
{
item = default(T);
return false;
}
}
}
}
EDIT 2
I hope this example helps.
Remember this is bare bones.
Using these basic ideas you can safely harness the power of threads.
public class WorkState
{
private readonly Object _Lock = new Object();
private Int32 _State;
public Int32 GetState()
{
lock (_Lock)
{
return _State;
}
}
public void UpdateState()
{
lock (_Lock)
{
_State++;
}
}
}
public class Worker
{
private readonly WorkState _State;
private readonly Thread _Thread;
private volatile Boolean _KeepWorking;
public Worker(WorkState state)
{
_State = state;
_Thread = new Thread(DoWork);
_KeepWorking = true;
}
public void DoWork()
{
while (_KeepWorking)
{
_State.UpdateState();
}
}
public void StartWorking()
{
_Thread.Start();
}
public void StopWorking()
{
_KeepWorking = false;
}
}
private void Execute()
{
WorkState state = new WorkState();
Worker worker = new Worker(state);
worker.StartWorking();
while (true)
{
if (state.GetState() > 100)
{
worker.StopWorking();
break;
}
}
}

You can pass an object as argument to the Thread.Start and use it as a shared data storage between the current thread and the initiating thread.
You can also just directly access (with the appropriate locking of course) your data members, if you started the thread using the instance form of the ThreadStart delegate.
You can't use attributes to create shared data between threads. You can use the attribute instances attached to your class as a data storage, but I fail to see how that is better than using static or instance data members.

Look at the following example code:
public class MyWorker
{
public SharedData state;
public void DoWork(SharedData someData)
{
this.state = someData;
while (true) ;
}
}
public class SharedData {
X myX;
public getX() { etc
public setX(anX) { etc
}
public class Program
{
public static void Main()
{
SharedData data = new SharedDate()
MyWorker work1 = new MyWorker(data);
MyWorker work2 = new MyWorker(data);
Thread thread = new Thread(new ThreadStart(work1.DoWork));
thread.Start();
Thread thread2 = new Thread(new ThreadStart(work2.DoWork));
thread2.Start();
}
}
In this case, the thread class MyWorker has a variable state. We initialise it with the same object. Now you can see that the two workers access the same SharedData object. Changes made by one worker are visible to the other.
You have quite a few remaining issues. How does worker 2 know when changes have been made by worker 1 and vice-versa? How do you prevent conflicting changes? Maybe read: this tutorial.

When you start a thread you are executing a method of some chosen class. All attributes of that class are visible.
Worker myWorker = new Worker( /* arguments */ );
Thread myThread = new Thread(new ThreadStart(myWorker.doWork));
myThread.Start();
Your thread is now in the doWork() method and can see any atrributes of myWorker, which may themselves be other objects. Now you just need to be careful to deal with the cases of having several threads all hitting those attributes at the same time.