Detect Duplicate Items in DataFlow - c#

I've been building out a service that processes files using a Queue<string> object to manage the items.
public partial class BasicQueueService : ServiceBase
{
private readonly EventWaitHandle completeHandle =
new EventWaitHandle(false, EventResetMode.ManualReset, "ThreadCompleters");
public BasicQueueService()
{
QueueManager = new Queue<string>();
}
public bool Stopping { get; set; }
private Queue<string> QueueManager { get; }
protected override void OnStart(string[] args)
{
Stopping = false;
ProcessFiles();
}
protected override void OnStop()
{
Stopping = true;
}
private void ProcessFiles()
{
while (!Stopping)
{
var count = QueueManager.Count;
for (var i = 0; i < count; i++)
{
//Check the Stopping Variable again.
if (Stopping) break;
var fileName = QueueManager.Dequeue();
if (string.IsNullOrWhiteSpace(fileName) || !File.Exists(fileName))
continue;
Console.WriteLine($"Processing {fileName}");
Task.Run(() =>
{
DoWork(fileName);
})
.ContinueWith(ThreadComplete);
}
if (Stopping) continue;
Console.WriteLine("Waiting for thread to finish, or 1 minute.");
completeHandle.WaitOne(new TimeSpan(0, 0, 15));
completeHandle.Reset();
}
}
partial void DoWork(string fileName);
private void ThreadComplete(Task task)
{
completeHandle.Set();
}
public void AddToQueue(string file)
{
//Called by FileWatcher/Manual classes, not included for brevity.
lock (QueueManager)
{
if (QueueManager.Contains(file)) return;
QueueManager.Enqueue(file);
}
}
}
Whilst researching how to limit the number of threads on this (I've tried a manual class with an incrementing int, but there's an issue where it doesn't decrement properly in my code), I came across TPL DataFlow, which seems like its a better fit for what I'm trying to achieve - specifically, it allows me to let the framework handle threading/queueing, etc.
This is now my service:
public partial class BasicDataFlowService : ServiceBase
{
private readonly ActionBlock<string> workerBlock;
public BasicDataFlowService()
{
workerBlock = new ActionBlock<string>(file => DoWork(file), new ExecutionDataflowBlockOptions()
{
MaxDegreeOfParallelism = 32
});
}
public bool Stopping { get; set; }
protected override void OnStart(string[] args)
{
Stopping = false;
}
protected override void OnStop()
{
Stopping = true;
}
partial void DoWork(string fileName);
private void AddToDataFlow(string file)
{
workerBlock.Post(file);
}
}
This works well. However, I want to ensure that a file is only ever added to the TPL DataFlow once. With the Queue, I can check that using .Contains(). Is there a mechanism that I can use for TPL DataFlow?

Your solution with Queue works only if file goes into your service twice in a small period of time. If it came again in, say, few hours, queue will not contain it, as you Dequeue it from there.
If this solution is expected, then you may use a MemoryCache to store file paths being already handled, like this:
using System.Runtime.Caching;
private static object _lock = new object();
private void AddToDataFlow(string file)
{
lock (_lock)
{
if (MemoryCache.Default.Contains(file))
{
return;
}
// no matter what to put into the cache
MemoryCache.Default[file] = true;
// we can now exit the lock
}
workerBlock.Post(file);
}
However, if your application must run for a long time (which service is intended to do), you'll eventually run out of memory. In that case you probably need to store your file paths in database or something, so even after restarting the service your code will restore the state.

You can check it inside of DoWork.
You have to save in Hash already works items and check current filename doesn't exist in hash.

Related

Does function1 and function2 use my serial port class in thread safe way?

I have a wrapper class around serial port which looks something like this:
static class HASPCLass
{
private static SerialPort m_port;
private static bool m_initialized;
private static int m_baudRate;
static readonly object _syncObject = new object();
public DoInitialization(int baudRate /*also could be other params*/)
{
lock(_syncObject)
{
if (!m_initialized)
{
Initialize(baudRate);
}
}
}
private Initialize(int baudrate /*also could have other params*/)
{
m_port.open(..);
m_baudRate = baudRate;
m_initialized = true;
}
private Uninitialize()
{
m_port.close();
m_initialized = false;
}
public void Read(byte[] buff)
{
lock(_syncObject)
{
//Other custom read stuff
m_port.Read(buff);
}
}
public void Write(byte [] buff)
{
lock(_syncObject)
{
//Other write related code
m_port.Write(buff);
}
}
public void Close()
{
lock(_syncObject)
{
if (m_initialized)
{
Uninitialize();
}
}
}
}
I tried making this class thread safe. Someone initializes it - read and writes maybe used from other threads - and in the end calls Close.
Now Imagine I have two additional static methods from other class which do something like this:
public static void function1()
{
HASPClass.Read(...);
// Some other code
HASPClass.Write(...);
}
public static void function2()
{
HASPClass.Read(...);
// Some other code
HASPClass.Write(...);
}
For overall thread safety I also enclosed these functions in locks:
public static void function1()
{
lock(otherlock1)
{
HASPClass.Read(...);
// Some other code
HASPClass.Write(...);
}
}
public static void function2()
{
lock(otherlock1)
{
HASPClass.Read(...);
// Some other code
HASPClass.Write(...);
}
}
Because order in which read and writes are called might be relavant for the HASP.
My question is: is now my final approach (of using function1 and function2) correct/thread safe?
Since you kind of use a singleton you are fine without additional locks as long as the functions do not use resources that have to be locked in // Some other code.
The class itself is thread safe because it locks all uses of the variables with the same lock. This is as tight as it gets. But make sure to not introduce dead locks in the code that lies behind the comments.
In general you should make sure no one closes your object before all threads are done with it.
Besides this code example is more or less inconsistent. You don't declare it static and write no return types and all.
Edit: From the higher persepctive of the need to give commands in a special order I correct the statement and say yes you need to lock it.
But beware of dead locks.
A more explicit way how this can go wrong (though I don't see it happening in your example code):
There are 2 threads that can hold the lock. Your device will always send you 1 except if you transmit 2 to it then it will send you 2.
Thread 1 is trying to first read a 1 and after that a 2 from the device without releasing the lock.
Now suppose somehow the actions taken after receiving 1 start Thread 2 which wants to transmit 2 to the device. But it can not because Thread 1 is still waiting but it will wait forever because Thread 2 can not transmit.
The most often case for this is GUI events used with invoke (which leads to an other thread executing code).
Imagine I have two additional static methods from other class ... To ensure thread safety do I have to put additional locks ... ?
No.
A lock does not care about the calling method or the stack trace - it only concerns the current thread. Since you already put locks in the critical sections, there is no point in putting higher level locks in your case.
You don't want a thread-safe class, you want a message queue.
By the comments I see your concern is if read/writes are mixed, you write from one thread and other issues a read before the writer thread reads the response.
In that scenario the best you can do is to create a queue of operations, when a write must read then you add a Read and Write operation in only one call, in this way the sequence will be warranted to follow the correct order, and in this way you only need to lock the queue.
Something like this:
Queue:
public class SerialQueue
{
SerialPort sp;
ManualResetEvent processQueue = new ManualResetEvent(false);
Queue<QueueCommand> queue = new Queue<QueueCommand>();
public event EventHandler<ReadEventArgs> ReadSuccess;
public event EventHandler<IdEventArgs> WriteSuccess;
public SerialQueue()
{
ThreadPool.QueueUserWorkItem(ProcessQueueThread);
sp = new SerialPort(); //Initialize it according to your needs.
sp.Open();
}
void ProcessQueueThread(object state)
{
while (true)
{
processQueue.WaitOne();
QueueCommand cmd;
while(true)
{
lock (queue)
{
if (queue.Count > 0)
cmd = queue.Dequeue();
else
{
processQueue.Reset();
break;
}
}
if (cmd.Operation == SerialOperation.Write || cmd.Operation == SerialOperation.WriteRead)
{
sp.Write(cmd.BytesToWrite, 0, cmd.BytesToWrite.Length);
if (WriteSuccess != null)
WriteSuccess(this, new IdEventArgs { Id = cmd.Id });
}
if(cmd.Operation == SerialOperation.Read || cmd.Operation == SerialOperation.WriteRead)
{
byte[] buffer = new byte[cmd.BytesToRead];
sp.Read(buffer, 0, buffer.Length);
if (ReadSuccess != null)
ReadSuccess(this, new ReadEventArgs { Id = cmd.Id, Data = buffer });
}
}
}
}
public void EnqueueCommand(QueueCommand Command)
{
lock(queue)
{
queue.Enqueue(Command);
processQueue.Set();
}
}
}
QueueCommand:
public class QueueCommand
{
public QueueCommand()
{
Id = Guid.NewGuid();
}
public Guid Id { get; set; }
public SerialOperation Operation { get; set; }
public int BytesToRead { get; set; }
public byte[] BytesToWrite { get; set; }
}
Enums:
public enum SerialOperation
{
Read,
Write,
WriteRead
}
Event arguments:
public class IdEventArgs : EventArgs
{
public Guid Id { get; set; }
}
public class ReadEventArgs : IdEventArgs
{
public byte[] Data{ get; set; }
}
To use the queue you instantiate it and hook to the WriteSucces and ReadSucces.
SerialQueue queue = new SerialQueue();
queue.ReadSuccess += (o, args) => { /*Do whatever you need to do with the read data*/ };
queue.WriteSuccess += (o, args) => { /*Do whatever you need to do after the write */ };
Note that each QueueCommand has a property named Id which is a unique Guid, it allows you to track when the commands are executed.
Now, when you want to perform a read you do:
QueueCommand cmd = new QueueCommand { Operation = SerialOperation.Read, BytesToRead = 1024 };
queue.Enqueue(cmd);
In this moment the queue will add the command and set the reset event, when the reset event is set the thread processing the commands will continue it's execution (if wasn't already executing) and process all the possible commands in the queue.
For a write you will do:
QueueCommand cmd = new QueueCommand { Operation = SerialOperation.Write, BytesToWrite = new byte[]{ 1, 10, 40 } };
And for a write followed by a read you will do:
QueueCommand cmd = new QueueCommand { Operation = SerialOperation.WriteRead, BytesToWrite = new byte[]{ 1, 10, 40 }, BytesToRead = 230 };
I have been working with serial ports for years in multi threaded environments and this is the only way to ensure sequentiallity between sent commands and received responses, else you will mix responses from different commands.
Remember this is just a base implementation, you need to add error handling and customize it to your needs.
The thread safety of a method has nothing to deal with serial port operations (see this interesting discussion What Makes a Method Thread-safe? What are the rules?).
At the end, I think that your lock(_syncObject) in your first class is not necessary (but I don't know the rest of your code!), if you call the methods in the way you did, because the Read() and Write() calls are enclosed in a sync-lock to the same object (I'm supposing that your lock object is declared like private static readonly object otherlock1 = new object();).
In my opinion, if you only call function1 and function2 in the rest of your code, your approach is definitely thread-safe (supposed that your // Some other code don't spawn another thread that can make some thread-unsafe operations on the same variables on which function1 and function2 are working...).
Talking about the serial port protocol, what does it happen if your // Some other code fails for some reason? For example a computation error between your HASPClass.Read(...) and HASPClass.Write(...). This might not affect the thread-safety it-self, but damage the sequence of the read-write operations (but only you can know the details on that).
First of all, using singletons in such manner is a bad practice. You should consider using something like this.
public sealed class SerialPortExt
{
private readonly SerialPort _serialPort;
private readonly object _serialPortLock = new object();
public SerialPortExt(SerialPort serialPort)
{
_serialPort = serialPort;
}
public void DoSomething()
{
}
public IDisposable Lock()
{
return new DisposableLock(_serialPortLock);
}
}
Where DisposableLock looks like this.
public sealed class DisposableLock : IDisposable
{
private readonly object _lock;
public DisposableLock(object #lock)
{
_lock = #lock;
Monitor.Enter(_lock);
}
#region Implementation of IDisposable
public void Dispose()
{
Monitor.Exit(_lock);
}
#endregion
}
Then you can work with your instance in the following way.
class Program
{
static void Main()
{
var serialPortExt = new SerialPortExt(new SerialPort());
var tasks =
new[]
{
Task.Run(() => DoSomething(serialPortExt)),
Task.Run(() => DoSomething(serialPortExt))
};
Task.WaitAll(tasks);
}
public static void DoSomething(SerialPortExt serialPortExt)
{
using (serialPortExt.Lock())
{
serialPortExt.DoSomething();
Thread.Sleep(TimeSpan.FromSeconds(5));
}
}
}
Since I cannot try out your code and it wouldn't compile I would just advice that you make your wrapper into a singleton and perform the locking from there.
Here is an example of your sample code converted to a singleton class based on MSDN Implementing Singleton in C#:
public class HASPCLass
{
private static SerialPort m_port;
private static bool m_initialized;
private static int m_baudRate;
static readonly object _syncObject = new object();
private static HASPCLass _instance;
public static HASPCLass Instance
{
get
{
if(_instance == null)
{
lock(_syncObject)
{
if (_instance == null)
{
_instance = new HASPCLass();
}
}
}
return _instance;
}
}
public void DoInitialization(int baudRate /*also could be other params*/)
{
if (!m_initialized)
{
Initialize(baudRate);
}
}
private void Initialize(int baudrate /*also could have other params*/)
{
m_port.Open();
m_baudRate = baudrate;
m_initialized = true;
}
private void Uninitialize()
{
m_port.Close();
m_initialized = false;
}
public void Read(byte[] buff)
{
m_port.Read(buff, 0, buff.Length);
}
public void Write(byte[] buff)
{
m_port.Write(buff, 0, buff.Length);
}
public void Close()
{
if (m_initialized)
{
Uninitialize();
}
}
}
Notice that locking is only applied on the instance of HASPCLass.
if(_instance == null)
This check is added because when multiple threads try to access the singleton instance it will be null. In this case that is the time where it should wait and check if it is currently locked. These modifications has already made your HASPCLass thread safe! Now consider adding more functions such as for setting the port name and other properties as needed.
Generally, in this case of situation, you have to use a Mutex().
A mutex permits mutual exclusion to shared resources.

Using CacheItemRemovedCallback to simulate a Windows Service acts different on first run

I simulated a Windows Service with the CacheItemRemovedCallback, based on this source:
Simulate a Windows Service using ASP.NET to run scheduled jobs
Everything is working great, except from the first run after the application recycles or restart.
Global.asax.cs
protected void Application_Start(object sender, EventArgs e)
{
RegisterCacheEntry();
}
private bool RegisterCacheEntry()
{
if (null != HttpContext.Current.Cache[DummyCacheItemKey]) return false;
int interval = Properties.Settings.Default.SchedulerInterval;
HttpContext.Current.Cache.Add(
DummyCacheItemKey, "StartingScheduler", null,
Cache.NoAbsoluteExpiration, TimeSpan.FromHours(interval),
CacheItemPriority.NotRemovable,
new CacheItemRemovedCallback(CacheItemRemovedCallback)
);
return true;
}
public void CacheItemRemovedCallback(string key, object value, CacheItemRemovedReason reason)
{
Scheduler.HitPage(Properties.Settings.Default.Dummy);
string dirScheduler = Properties.Settings.Default.Scheduler;
Scheduler.Manage(dirScheduler);
}
protected void Application_BeginRequest(object sender, EventArgs e)
{
if (HttpContext.Current.Request.Url.ToString() == Properties.Settings.Default.Dummy)
{
RegisterCacheEntry();
}
}
public static class Scheduler
public static string _DirScheduler { get; set; }
public static void Manage(string dirScheduler)
{
_DirScheduler = dirScheduler;
string[] schedulesCfg = Directory.GetFiles(dirScheduler, "*.xml");
foreach (string scheduleCfg in schedulesCfg)
{
//scheduleData = Class to store the schedule data
ThreadPool.QueueUserWorkItem(o => FireAway(scheduleData));
}
}
private static void FireAway(Schedule schedule)
{
string month = DateTime.Now.Month.ToString();
if (month.Length == 1)
{
month = month.Insert(0, "0");
}
// Wait Handle to Manage Access to Log File
EventWaitHandle waitHandle = new EventWaitHandle(true, EventResetMode.AutoReset, "LogFile_Shared");
// Lock the log file
waitHandle.WaitOne();
WriteLog("path_to_log_file", schedule, "started");
// Release the lock on log file
waitHandle.Set();
// Ok until here !
string message = ScheduleScript.Launch(schedule);
// Code not executed from here (only after application end or recycle)
if (schedule.Repeat == "0")
{
waitHandle.WaitOne();
UpdateScheduler(schedule);
waitHandle.Set();
}
// Lock the log file
waitHandle.WaitOne();
WriteLog("path_to_log_file", schedule, message);
// Release the lock on log file
waitHandle.Set();
}
private static void UpdateScheduler(Schedule schedule)
{
// Update a xml file containing scheduler data
}
public static class ScheduleScript
public static string Launch(Schedule schedule)
{
string message = "finished";
string messageScheduler = string.Empty;
using (WebClientCustom web = new WebClientCustom(10800000)) // 3 hours timeout
{
try
{
string res = web.DownloadString(schedule.Script);
}
catch (Exception e)
{
message = e.Message;
}
}
return message;
}
In the static class Scheduler, everything is executed until ScheduleScript.Launch(schedule). Then, the code is not executed (the log is not updated with Status finished and the Scheduler xml is not updated either with the UpdateScheduler method).
I tried everything I could without success. This situation happened only when a schedule starts after the application is recycled.
This behavior happen for example if I have a schedule expected to launch at 10am and has not yet been executed and I publish a new version of the application or manually execute an application pool recycle in the same hour (10am) of the schedule.

executing a block of code repeatedly without a timeout. windows service

I have a simple windows service written, here is its skeleton:
internal class ServiceModel {
private Thread workerThread;
private AutoResetEvent finishedEvent;
private Int32 timeout = 60000*15;
public void Start() {
this.workerThread = new Thread(this.Process);
this.finishedEvent = new AutoResetEvent(false);
this.workerThread.Start();
}
public void Stop() {
this.finishedEvent.Set();
this.workerThread.Join(30000);
}
public void Process() {
while(!this.finishedEvent.WaitOne(timeout)) {
// run things here
}
}
}
the first thing
The first thing that I can't understand is that service waits one timeout before running. Would rewriting the new AutoResetEvent(false); to new AutoResetEvent(true); cause a service to start without waiting?
the second thing
Due to some internal reasons (requesting data from external server/service, exception handling) sometimes it is not enough to wait that fixed 15..30-minutes timeout.
How do I rewrite it to work without a fixed timeout?
Do I need to remove that AutoResetEvent instance at all and run Process body inside an infinite loop?
public void Process() {
while(true) {
// run things here
}
}
edit. try-catch/lock
In Process method there is a global try-catch block:
public void Process() {
do {
try {
// processing goes here
}
catch(Exception ex) {
Logger.Log.Warn(ex); // or Log.Fatal(ex)...
}
}
while(true);
}
if I use a synchronization object where do I put the lock statement so that I'm able to call break when isStopped is true?
You don't have to deal with low-level thread and synchronization primitives API. Consider using Task Parallel Library (TPL). It's easy to implement OnStop using TPL cancellation framework:
using System.ServiceProcess;
using System.Threading;
using System.Threading.Tasks;
namespace WindowsService1
{
public partial class Service1 : ServiceBase
{
CancellationTokenSource _mainCts;
Task _mainTask;
public Service1()
{
InitializeComponent();
}
async Task MainTaskAsync(CancellationToken token)
{
while (true)
{
token.ThrowIfCancellationRequested();
// ...
await DoPollingAsync(token);
// ...
}
}
protected override void OnStart(string[] args)
{
_mainCts = new CancellationTokenSource();
_mainTask = MainTaskAsync(_mainCts.Token);
}
protected override void OnStop()
{
_mainCts.Cancel();
try
{
_mainTask.Wait();
}
catch
{
if (!_mainTask.IsCanceled)
throw;
}
}
}
}
Inside MainTaskAsync you can use Task.Run for any CPU-bound work items.
using Threads you can achieve your requirement using the following code:
internal class ServiceModel {
private Thread workerThread;
private object syncLock = new object();
private bool stop = false;
public void Start() {
this.workerThread = new Thread(this.Process);
this.workerThread.Start();
}
public void Stop() {
lock(syncLock) stop = true;
this.workerThread.Join(30000);
}
public void Process() {
while(true){
//your stuff here.
lock(syncLock)
{
if(stop)
break;
}
Thread.Sleep(30000);
}
}
}

Joining a thread started with StartNew()

When using the StartNew() method to kick off a process on a new thread, I need to figure out how to make another call into this object in that same thread (I assume this would be some sort of Join operation?).
The following example is dumbed down to illustrate the meat of what I am trying to do. I am well aware it is severely lacking in basic concurrency considerations. But I didn't want to cloud the code with all of that logic, so please forgive me on that.
The following console app shows what I am trying to accomplish. Assume on the StartNew() call a new thread with ID 9976 is created and the method invoked there. I would like the subsequent call to ProcessImmediate() in the file system watcher change event handler to be made on thread 9976 as well. As it stands, the call would share the same thread that is used for the file system watcher change event.
Can this be done, and if so, how?
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var runner = new Runner();
runner.Run();
Console.ReadKey();
}
}
public class Runner
{
private Activity _activity = null;
private FileSystemWatcher _fileSystemWatcher;
public void Run()
{
_activity = new Activity();
// start activity on a new thread
Task.Factory.StartNew(() => _activity.Go());
_fileSystemWatcher = new FileSystemWatcher();
_fileSystemWatcher.Filter = "*.watcher";
_fileSystemWatcher.Path = "c:\temp";
_fileSystemWatcher.Changed += FileSystemWatcher_Changed;
_fileSystemWatcher.EnableRaisingEvents = true;
}
private void FileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
// WANT TO CALL THIS FOR ACTIVITY RUNNING ON PREVIOUSLY CALLED THREAD
_activity.ProcessImmediate();
}
}
public class Activity
{
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
System.Threading.Thread.Sleep(2000);
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate()
{
// for purposes of this example, assume that Go is magically in its sleep state when ProcessImmediate is called
DoSomethingInteresting();
}
public bool Stop { get; set; }
}
}
* UPDATE *
Thanks for the excellent responses. I took Mike's suggestion and implemented it for my console app. Below is the full working code which also includes the use of a cancellation token. I post this in case someone else might find it useful.
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
var runner = new Runner();
runner.Run();
Console.ReadKey();
runner.Stop();
Console.ReadKey();
}
}
public class Runner
{
private Activity _activity = null;
private FileSystemWatcher _fileSystemWatcher;
private CancellationTokenSource _cts = new CancellationTokenSource();
public void Stop() { _cts.Cancel(); }
public void Run()
{
_activity = new Activity();
// start activity on a new thread
var task = new Task(() => _activity.Go(_cts.Token), _cts.Token, TaskCreationOptions.LongRunning);
task.Start();
_fileSystemWatcher = new FileSystemWatcher();
_fileSystemWatcher.Filter = "*.watcher";
_fileSystemWatcher.Path = "C:\\Temp\\FileSystemWatcherPath";
_fileSystemWatcher.Changed += FileSystemWatcher_Changed;
_fileSystemWatcher.EnableRaisingEvents = true;
}
private void FileSystemWatcher_Changed(object sender, FileSystemEventArgs e)
{
// WANT TO CALL THIS FOR ACTIVITY RUNNING ON PREVIOUSLY CALLED THREAD
_activity.ProcessImmediate();
}
}
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
public void Go(CancellationToken ct)
{
Thread.CurrentThread.Name = "Go";
while (!ct.IsCancellationRequested)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(5000);
}
Console.WriteLine("Exiting");
}
protected virtual void DoSomethingInteresting()
{
Console.WriteLine(string.Format("Doing Something Interesting on thread {0}", Thread.CurrentThread.ManagedThreadId));
}
public void ProcessImmediate()
{
// for purposes of this example, assume that Go is magically in its sleep state when ProcessImmediate is called
_processing.Set();
}
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}
}
First, you should use TaskCreationOptions.LongRunning if you are creating a task that will not complete quickly. Second, use an AutoResetEvent to signal the waiting thread to wake up. Note that below ProcessImmediate will return before DoSomethingInteresting has completed running on the other thread. Example:
using System.Threading;
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(2000);
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate()
{
_processing.Set();
}
public bool Stop { get; set; }
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}
User mike has given a better solution, which will be appropriate when you like to call the same method immediately. If you want to call a different methods immediately I'll expand mike's answer to achieve that.
using System.Threading;
public class Activity : IDisposable
{
private AutoResetEvent _processing = new AutoResetEvent(false);
private ConcurrentQueue<Action> actionsToProcess = new ConcurrentQueue<Action>();
public void Go()
{
while (!Stop)
{
// for purposes of this example, magically assume that ProcessImmediate has not been called when this is called
DoSomethingInteresting();
_processing.WaitOne(2000);
while(!actionsToProcess.IsEmpty)
{
Action action;
if(actionsToProcess.TryDeque(out action))
action();
}
}
}
protected virtual void DoSomethingInteresting() { }
public void ProcessImmediate(Action action)
{
actionsToProcess.Enqueue(action);
_processing.Set();
}
public bool Stop { get; set; }
public void Dispose()
{
if (_processing != null)
{
_processing.Dispose();
_processing = null;
}
}
}
To execute different methods on the same thread you can use a message loop that dispatches incoming requests. A simple option would be to use the event loop scheduler of the Reactive Extensions and to "recursively" schedule your Go() function - if in the mean time a different operation is scheduled it would be processed before the next Go() operation.
Here is a sample:
class Loop
: IDisposable
{
IScheduler scheduler = new EventLoopScheduler();
MultipleAssignmentDisposable stopper = new MultipleAssignmentDisposable();
public Loop()
{
Next();
}
void Next()
{
if (!stopper.IsDisposed)
stopper.Disposable = scheduler.Schedule(Handler);
}
void Handler()
{
Thread.Sleep(1000);
Console.WriteLine("Handler: {0}", Thread.CurrentThread.ManagedThreadId);
Next();
}
public void Notify()
{
scheduler.Schedule(() =>
{
Console.WriteLine("Notify: {0}", Thread.CurrentThread.ManagedThreadId);
});
}
public void Dispose()
{
stopper.Dispose();
}
}
static void Main(string[] args)
{
using (var l = new Loop())
{
Console.WriteLine("Press 'q' to quit.");
while (Console.ReadKey().Key != ConsoleKey.Q)
l.Notify();
}
}

How to keep object member variables private to a thread in a multi threaded environment

I have a multi threaded program that opens a few threads to query an external CRM and save the results in an in-memory IDictionary in order to speed up the system.
I'm a little confused about multi threading and critical sections. I want my class QueryThreadProcess to have a thread which runs the query and to manage starting and stopping the query. It has an object of type query and saves the results in a list.
The class QueryManager will kill all the processes or start all processes, basically collection wide methods.
I have a feeling that the private members for QueryThreadProcess are shared between all threads. How would I be able to make them private to each thread, but also kill each thread separately from an external class?
I don't want to lock because I want all the threads to run parallel.
Here is my manager class:
public class QueryManager
{
private IDictionary<int, QueryThreadProcess> _queries;
public QueryManager()
{
_queries = new Dictionary<int, QueryThreadProcess>();
}
public void Start()
{
CreateQueryThreadsFromDb();
StartAllThreads();
}
private void StartAllThreads()
{
if (_queries != null && _queries.Count > 0)
{
StopThreadsAndWaitForKill();
}
foreach (var query in _queries)
query.Value.Start();
}
private void CreateQueryThreadsFromDb()
{
var queries = new QueryProvider().GetAllQueries();
if (_queries != null && _queries.Count > 0)
{
StopThreadsAndWaitForKill();
_queries.Clear();
}
foreach (var query in queries)
_queries.Add(query.Id, new QueryThreadProcess(query));
}
private void StopThreadsAndWaitForKill()
{
KillAllThreads();
while (!AreAllThreadsKilled()) { }
}
private void KillAllThreads()
{
foreach (var query in _queries)
query.Value.Kill();
}
private bool AreAllThreadsKilled()
{
return _queries.All(query => query.Value.IsKilled);
}
public IList<User> GetQueryResultById(int id)
{
return _queries[id].Result;
}
}
and here is my class for QueryProcesses which holds the threads that do the actual query:
using System.Collections.Generic;
using System.Threading;
using Intra.BLL.MessageProviders;
using Intra.BO;
using Intra.BO.Messages;
namespace Intra.BLL.QueryProcess
{
internal class QueryThreadProcess
{
private readonly Thread _thread;
private readonly Query _query;
private bool _isStoppingQuery = false;
private bool _isKilled = true;
private IList<User> _result;
private readonly object _objSync = new object();
public QueryThreadProcess(Query query)
{
_query = query;
_thread = new Thread(RetrieveQueries);
}
public void Start()
{
_isStoppingQuery = true;
while (!_isKilled) { }
_isStoppingQuery = false;
_thread.Start();
}
private void RetrieveQueries()
{
const string BROKERNAME = "bla";
_isKilled = false;
while (!_isStoppingQuery)
{
Broker broker = new BrokerProvider().GetBrokerByName(BROKERNAME);
var users = new QueryProvider().GetUserObjectsByQuery(_query, ParaTokenGenerator.GetBrokerAuthToken(broker));
_result = users;
}
_isKilled = true;
}
public bool IsKilled
{
get { return _isKilled; }
}
public IList<User> Result
{
get
{
lock (_objSync)
return _result;
}
}
public void Kill()
{
_isStoppingQuery = true;
}
}
}
It doesn't really answer your question, but it looks like a more modern approach using the Task Parallel Library of .NET 4 could save you some headache. Controlling Threads by yourself isn't necessary. It looks like you could refactor your classes to a few lines of code and get rid of the described problems.
.NET 4 has ThreadLocal<T> which may be of interest to you
The _thread and _query fields probably don’t matter as they are declared readonly and are not changed after each thread is run. These are not shared between worker threads as they are private to the class and you create a separate instance of the class for each thread.
_isStoppingQuery and _isKilled are accessed by both the worker thread and the controlling thread. As such these should be declared volatile to ensure they are not cashed in a processor register and don’t suffer from execution reordering.
There is a potential issue with _result. The lock in Result/get is not enough to protect the contents of _result. It is only protecting the reference to the list not the list itself. However as your worker thread is only overwriting the reference each cycle it may not be an issue. I would probably do away with the lock on _objSync and declare _result volatile too.

Categories