We Keep Coding

Modbus read and write value conflict

My colleague and I working on project, where some physical devices are connected to Modbus interface(e.g. Lamp), and an desktop app where we sending requests to modbus using NModbus package, and also from time to time(e.g. after every 1 second) reading data from Modbus. We have a classic read/write conflict. Reading has no issue, but sometimes when we writing new value to modbus, the physical lamp going crazy and changing his state every 1 second.
Reading data from modbus is in different task, so do writing new values to modbus.
What we`ve tried:
lock critical section(only writing), and ignoring reading data when new value comes. After that we have a problem with queue and very slow working
CancellationToken - had no effect, or I writing it bad
Currently we have a class that collects property date of last state change(registered in IoC) and when new writing value comes, then we updating this property, and blocking from reading data from Modbus. Unfortunately this code sometimes working, sometimes not.
Please help. We are going crazy with this issue and we don`t know how to fix it.
Edit: I posting current code.
This is task, where we executing GetCurrentState handler
public void Start()
{
var cancellationToken = _cancellationTokenSource.Token;
Task.Run(async () =>
{
while (!cancellationToken.IsCancellationRequested)
{
await Task.Delay(ReadStateDelayInMiliseconds).ConfigureAwait(false);
if ((DateTime.Now - _lastCommandExecutionTimeContainer.LastCommandExecutionTime).TotalMilliseconds < _userCommandThresholdInMiliseconds)
continue;
try
{
await _service.GetCurrentState().ConfigureAwait(false);
}
catch (Exception ex)
{
Logger.Error($"Error while checking current state. Message '{ex.Message}', Stack trace: '{ex.StackTrace}'.");
}
}
}, cancellationToken);
}
This is GetCurrentState handler, where we reading data from Modbus
protected override IgnisLightState Handle(GetState request)
{
if ((DateTime.Now - _lastCommandExecutionTimeContainer.LastCommandExecutionTime).TotalMilliseconds <= _configuration.UserCommandThresholdInMiliseconds)
{
Logger.Debug($"Ignore before read state from lights, time after last execution: '{(DateTime.Now - _lastCommandExecutionTimeContainer.LastCommandExecutionTime).TotalMilliseconds}'.");
return _state;
}
var currentLightState = _state.DeepClone();
foreach (var head in currentLightState.Heads)
{
try
{
ushort startAddress = 0x0000;
ushort numberOfPointsToRead = 0x0006;
var values = _modbusMaster.ReadHoldingRegisters((byte)head.UniqueId, startAddress, numberOfPointsToRead);
var isOn = values[IgnisRegistry.On.Value];
var isEndo = values[IgnisRegistry.Endo.Value];
var isCentrum = values[IgnisRegistry.Centrum.Value];
var tempValue = values[IgnisRegistry.Temp.Value];
var illuminanceValue = values[IgnisRegistry.Vol.Value];
head.ColorValue = Convert.ToInt32(tempValue);
head.IlluminanceValue = Convert.ToInt32(illuminanceValue);
head.IsCentrumOn = Convert.ToBoolean(isCentrum);
head.IsEndoOn = Convert.ToBoolean(isEndo);
head.IsTurnedOn = Convert.ToBoolean(isOn);
if (currentLightState.CameraState != null &&
_configuration.CameraHeadId.HasValue &&
_configuration.CameraHeadId.Value == head.UniqueId)
{
var camMode = values[IgnisRegistry.Cam.Value];
currentLightState.CameraState.IsTurnedOn = Convert.ToBoolean(isOn);
currentLightState.CameraState.CurrentMode = (IgnisCameraMode)Convert.ToInt32(camMode);
}
}
catch (Exception ex)
{
Logger.ErrorFixed(ex, $"Error while getting data from headId {head.UniqueId}.");
}
}
if (_state.Equals(currentLightState)
|| (DateTime.Now - _lastCommandExecutionTimeContainer.LastCommandExecutionTime).TotalMilliseconds < _configuration.UserCommandThresholdInMiliseconds)
{
Logger.Debug($"Ignore after read state from lights, time after last execution: '{(DateTime.Now - _lastCommandExecutionTimeContainer.LastCommandExecutionTime).TotalMilliseconds}'.");
return _state;
}
foreach (var currentHeadState in currentLightState.Heads)
{
_lightHeadStateUpdater.UpdateState(currentHeadState);
}
Logger.Debug($"Broadcast new state to clients '{JsonConvert.SerializeObject(currentLightState)}'.");
_hubContext.Clients.All.StateChanged(currentLightState);
return currentLightState;
}
This is an Turn on lamp handler where we writing new value to modbus:
protected override Response Handle(TurnOn request)
{
_lastCommandExecutionTimeContainer.SetLastCommandExecutionTime(DateTime.Now);
if (request.HeadId <= 0
|| !_state.Heads.Any(x=>x.UniqueId == request.HeadId))
{
return ResponseFactory.FromError(Error.NotExist);
}
var headState = _state.Heads.Single(x=>x.UniqueId == request.HeadId);
if (headState.IsTurnedOn)
return ResponseFactory.FromError(Error.AlreadyAsRequested);
_modbusMaster.WriteSingleRegister((byte)request.HeadId, IgnisRegistry.On.Value, 0x0001);
headState.IsTurnedOn = true;
if (_state.CameraState != null &&
_ignisLightConfiguration.CameraHeadId.HasValue &&
_ignisLightConfiguration.CameraHeadId.Value == request.HeadId)
{
_state.CameraState.IsTurnedOn = true;
}
Logger.Trace($"Turn head lamp {request.HeadId} on.");
_hubContext.Clients.All.HeadStateChanged(headState);
return ResponseFactory.Success;
}

Cassandra CSharp driver batch statement failing

I am trying to use a batch statement instead of a single binded insert statement.
Even though this is a very small change this fails and I am not looking for a good way for the error handling and to find out which part is the issue. One issue is definetly that the Java API has a getStatements method which is missing in the C# driver.
The pseudo code looks like this:
private BatchStatement batchStatement = new BatchStatement();
private const int blockingFactor = 5;
private int i = 0;
private object locker = new object();
public CassandraBufferHandler()
{
Cluster = Cluster.Builder().AddContactPoints("localhost").Build();
Session = Cluster.Connect("my_keyspace");
InsertStatement = Session.Prepare("Insert into ticks (instrumentcode, timestamp, type, exchange, price, volume) values(?,?,?,?,?,?) if not exists;");
}
public void OnEvent(TickCassandra tickCassandra, long sequence, bool endOfBatch)
{
try
{
lock (locker)
batchStatement.Add(
InsertStatement.Bind(tickCassandra.Instrumentcode,
tickCassandra.Timestamp,
tickCassandra.Type,
tickCassandra.Exchange,
tickCassandra.Price,
tickCassandra.Volume));
if (i++ % blockingFactor == 0)
{
BatchStatement tmp;
lock (locker)
{
tmp = batchStatement;
tmp.EnableTracing();
batchStatement = new BatchStatement();
}
Session.ExecuteAsync(tmp).ContinueWith(t =>
{
if (t.Exception != null)
{
ErrorCount++;
Log.Error(t.Exception.Message + tmp.ToString());
}
else
InsertCount++;
});
}
}
catch (Exception ex)
{
Log.Error("Exception:" + ex);
Active = false;
}

How can I pass variable to Action?

below is the code sample, I want the components in the queue to take the local variable "entity", how can I achive this? thx
private void DoComparison(StuffEntity entity)
{
try
{
bool dataFlag = CheckIsNewData(entity.PickingTime, entity.WarningPeriod);
if (dataFlag)
{
Queue<Action<StuffEntity>> queue = new Queue<Action<StuffEntity>>();
//How can I let the queue stuff take the entity?
queue.Enqueue(DelaySendingMessageOut);
if (!QueueItem.ContainsKey(entity.FridgeID))
{
QueueItem.Add(entity.FridgeID, queue);
}
}
}
catch (Exception ex)
{
CommonUnity.WriteLog(ex.Message);
CommonUnity.WriteLog(ex.StackTrace);
}
}
private void DelaySendingMessageOut(StuffEntity entity)
{
int pendingPeroid = entity.PendingTime.ToInt();
if (pendingPeroid <= 0)
pendingPeroid = 5;
Thread.Sleep(pendingPeroid * 60 * 1000);
TriggerCheckingBeforeSendMessageOut(entity);
}

queue.Enqueue((e) => DelaySendingMessageOut(entity));
But you can use Queue<Action> since you will not use the argument e:
Queue<Action> queue = new Queue<Action>();
queue.Enqueue(() => DelaySendingMessageOut(entity));

What is the best scenario for one fast producer multiple slow consumers?

I'm looking for the best scenario to implement one producer multiple consumer multithreaded application.
Currently I'm using one queue for shared buffer but it's much slower than the case of one producer one consumer.
I'm planning to do it like this:
Queue<item>[] buffs = new Queue<item>[N];
object[] _locks = new object[N];
static void Produce()
{
int curIndex = 0;
while(true)
{
// Produce item;
lock(_locks[curIndex])
{
buffs[curIndex].Enqueue(curItem);
Monitor.Pulse(_locks[curIndex]);
}
curIndex = (curIndex+1)%N;
}
}
static void Consume(int myIndex)
{
item curItem;
while(true)
{
lock(_locks[myIndex])
{
while(buffs[myIndex].Count == 0)
Monitor.Wait(_locks[myIndex]);
curItem = buffs[myIndex].Dequeue();
}
// Consume item;
}
}
static void main()
{
int N = 100;
Thread[] consumers = new Thread[N];
for(int i = 0; i < N; i++)
{
consumers[i] = new Thread(Consume);
consumers[i].Start(i);
}
Thread producer = new Thread(Produce);
producer.Start();
}

Use a BlockingCollection
BlockingCollection<item> _buffer = new BlockingCollection<item>();
static void Produce()
{
while(true)
{
// Produce item;
_buffer.Add(curItem);
}
// eventually stop producing
_buffer.CompleteAdding();
}
static void Consume(int myIndex)
{
foreach (var curItem in _buffer.GetConsumingEnumerable())
{
// Consume item;
}
}
static void main()
{
int N = 100;
Thread[] consumers = new Thread[N];
for(int i = 0; i < N; i++)
{
consumers[i] = new Thread(Consume);
consumers[i].Start(i);
}
Thread producer = new Thread(Produce);
producer.Start();
}
If you don't want to specify number of threads from start you can use Parallel.ForEach instead.
static void Consume(item curItem)
{
// consume item
}
void Main()
{
Thread producer = new Thread(Produce);
producer.Start();
Parallel.ForEach(_buffer.GetConsumingPartitioner(), Consumer)
}

Using more threads won't help. It may even reduce performance. I suggest you try to use ThreadPool where every work item is one item created by the producer. However, that doesn't guarantee the produced items to be consumed in the order they were produced.
Another way could be to reduce the number of consumers to 4, for example and modify the way they work as follows:
The producer adds the new work to the queue. There's only one global queue for all worker threads. It then sets a flag to indicate there is new work like this:
ManualResetEvent workPresent = new ManualResetEvent(false);
Queue<item> workQueue = new Queue<item>();
static void Produce()
{
while(true)
{
// Produce item;
lock(workQueue)
{
workQueue.Enqueue(newItem);
workPresent.Set();
}
}
}
The consumers wait for work to be added to the queue. Only one consumer will get to do its job. It then takes all the work from the queue and resets the flag. The producer will not be able to add new work until that is done.
static void Consume()
{
while(true)
{
if (WaitHandle.WaitOne(workPresent))
{
workPresent.Reset();
Queue<item> localWorkQueue = new Queue<item>();
lock(workQueue)
{
while (workQueue.Count > 0)
localWorkQueue.Enqueue(workQueue.Dequeue());
}
// Handle items in local work queue
...
}
}
}
That outcome of this, however, is a bit unpredictable. It could be that one thread is doing all the work and the others do nothing.

I don't see why you have to use multiple queues. Just reduce the amount of locking. Here is an sample where you can have a large number of consumers and they all wait for new work.
public class MyWorkGenerator
{
ConcurrentQueue<object> _queuedItems = new ConcurrentQueue<object>();
private object _lock = new object();
public void Produce()
{
while (true)
{
_queuedItems.Enqueue(new object());
Monitor.Pulse(_lock);
}
}
public object Consume(TimeSpan maxWaitTime)
{
if (!Monitor.Wait(_lock, maxWaitTime))
return null;
object workItem;
if (_queuedItems.TryDequeue(out workItem))
{
return workItem;
}
return null;
}
}
Do note that Pulse() will only trigger one consumer at a time.
Example usage:
static void main()
{
var generator = new MyWorkGenerator();
var consumers = new Thread[20];
for (int i = 0; i < consumers.Length; i++)
{
consumers[i] = new Thread(DoWork);
consumers[i].Start(generator);
}
generator.Produce();
}
public static void DoWork(object state)
{
var generator = (MyWorkGenerator) state;
var workItem = generator.Consume(TimeSpan.FromHours(1));
while (workItem != null)
{
// do work
workItem = generator.Consume(TimeSpan.FromHours(1));
}
}
Note that the actual queue is hidden in the producer as it's imho an implementation detail. The consumers doesn't really have to know how the work items are generated.

A producer consumer queue with an additional thread for a periodic backup of data

I'm trying to implement a concurrent producer/consumer queue with multiple producers and one consumer: the producers add some data to the Queue, and the consumer dequeues these data from the queue in order to update a collection. This collection must be periodically backed up to a new file. For this purpose I created a custom serializable collection: serialization could be performed by using the DataContractSerializer.
The queue is only shared between the consumer and the producers, so access to this queue must be managed to avoid race conditions.
The custom collection is shared between the consumer and a backup thread.
The backup thread could be activated periodically using a System.Threading.Timer object: it may initially be scheduled by the consumer, and then it would be scheduled at the end of every backup procedure.
Finally, a shutdown method should stop the queuing by producers, then stop the consumer, perform the last backup and dispose the timer.
The dequeuing of an item at a time may not be efficient, so I thought of using two queues: when the first queue becomes full, the producers notify the consumer by invoking Monitor.Pulse. As soon as the consumer receives the notification, the queues are swapped, so while producers enqueue new items, the consumer can process the previous ones.
The sample that I wrote seems to work properly. I think it is also thread-safe, but I'm not sure about that. The following code, for simplicity I used a Queue<int>. I also used (again for simplicity) an ArrayList instead of collection serializable.
public class QueueManager
{
private readonly int m_QueueMaxSize;
private readonly TimeSpan m_BackupPeriod;
private readonly object m_SyncRoot_1 = new object();
private Queue<int> m_InputQueue = new Queue<int>();
private bool m_Shutdown;
private bool m_Pulsed;
private readonly object m_SyncRoot_2 = new object();
private ArrayList m_CustomCollection = new ArrayList();
private Thread m_ConsumerThread;
private Timer m_BackupThread;
private WaitHandle m_Disposed;
public QueueManager()
{
m_ConsumerThread = new Thread(Work) { IsBackground = true };
m_QueueMaxSize = 7;
m_BackupPeriod = TimeSpan.FromSeconds(30);
}
public void Run()
{
m_Shutdown = m_Pulsed = false;
m_BackupThread = new Timer(DoBackup);
m_Disposed = new AutoResetEvent(false);
m_ConsumerThread.Start();
}
public void Shutdown()
{
lock (m_SyncRoot_1)
{
m_Shutdown = true;
Console.WriteLine("Worker shutdown...");
Monitor.Pulse(m_SyncRoot_1);
}
m_ConsumerThread.Join();
WaitHandle.WaitAll(new WaitHandle[] { m_Disposed });
if (m_InputQueue != null) { m_InputQueue.Clear(); }
if (m_CustomCollection != null) { m_CustomCollection.Clear(); }
Console.WriteLine("Worker stopped!");
}
public void Enqueue(int item)
{
lock (m_SyncRoot_1)
{
if (m_InputQueue.Count == m_QueueMaxSize)
{
if (!m_Pulsed)
{
Monitor.Pulse(m_SyncRoot_1); // it notifies the consumer...
m_Pulsed = true;
}
Monitor.Wait(m_SyncRoot_1); // ... and waits for Pulse
}
m_InputQueue.Enqueue(item);
Console.WriteLine("{0} \t {1} >", Thread.CurrentThread.Name, item.ToString("+000;-000;"));
}
}
private void Work()
{
m_BackupThread.Change(m_BackupPeriod, TimeSpan.FromMilliseconds(-1));
Queue<int> m_SwapQueueRef, m_WorkerQueue = new Queue<int>();
Console.WriteLine("Worker started!");
while (true)
{
lock (m_SyncRoot_1)
{
if (m_InputQueue.Count < m_QueueMaxSize && !m_Shutdown) Monitor.Wait(m_SyncRoot_1);
Console.WriteLine("\nswapping...");
m_SwapQueueRef = m_InputQueue;
m_InputQueue = m_WorkerQueue;
m_WorkerQueue = m_SwapQueueRef;
m_Pulsed = false;
Monitor.PulseAll(m_SyncRoot_1); // all producers are notified
}
Console.WriteLine("Worker\t < {0}", String.Join(",", m_WorkerQueue.ToArray()));
lock (m_SyncRoot_2)
{
Console.WriteLine("Updating custom dictionary...");
foreach (int item in m_WorkerQueue)
{
m_CustomCollection.Add(item);
}
Thread.Sleep(1000);
Console.WriteLine("Custom dictionary updated successfully!");
}
if (m_Shutdown)
{
// schedule last backup
m_BackupThread.Change(0, Timeout.Infinite);
return;
}
m_WorkerQueue.Clear();
}
}
private void DoBackup(object state)
{
try
{
lock (m_SyncRoot_2)
{
Console.WriteLine("Backup...");
Thread.Sleep(2000);
Console.WriteLine("Backup completed at {0}", DateTime.Now);
}
}
finally
{
if (m_Shutdown) { m_BackupThread.Dispose(m_Disposed); }
else { m_BackupThread.Change(m_BackupPeriod, TimeSpan.FromMilliseconds(-1)); }
}
}
}
Some objects are initialized in the Run method to allow you to restart this QueueManager after it is stopped, as shown in the code below.
public static void Main(string[] args)
{
QueueManager queue = new QueueManager();
var t1 = new Thread(() =>
{
for (int i = 0; i < 50; i++)
{
queue.Enqueue(i);
Thread.Sleep(1500);
}
}) { Name = "t1" };
var t2 = new Thread(() =>
{
for (int i = 0; i > -30; i--)
{
queue.Enqueue(i);
Thread.Sleep(3000);
}
}) { Name = "t2" };
t1.Start(); t2.Start(); queue.Run();
t1.Join(); t2.Join(); queue.Shutdown();
Console.ReadLine();
var t3 = new Thread(() =>
{
for (int i = 0; i < 50; i++)
{
queue.Enqueue(i);
Thread.Sleep(1000);
}
}) { Name = "t3" };
var t4 = new Thread(() =>
{
for (int i = 0; i > -30; i--)
{
queue.Enqueue(i);
Thread.Sleep(2000);
}
}) { Name = "t4" };
t3.Start(); t4.Start(); queue.Run();
t3.Join(); t4.Join(); queue.Shutdown();
Console.ReadLine();
}

I would suggest using the BlockingCollection for a producer/consumer queue. It was designed specifically for that purpose. The producers add items using Add and the consumers use Take. If there are no items to take then it will block until one is added. It is already designed to be used in a multithreaded environment, so if you're just using those methods there's no need to explicitly use any locks or other synchronization code.