We Keep Coding

Potential race conditions with ConcurrentBag and multithreaded application

I've been wrestling for the past few months with how to improve a process where I'm using a DispatcherTimer to periodically check resources to see if they need to be updated/processed. After updating the resource("Product"), move the Product to the next step in the process, etc. The resource may or may not be available immediately.
The reason I have been struggling is two-fold. One reason is that I want to implement this process asynchronously, since it is just synchronous at the moment. The second reason is that I have identified the area where my implementation is stuck and it seems like not an uncommon design pattern but I have no idea how to describe it succinctly, so I can't figure out how to get a useful answer from google.
A rather important note is that I am accessing these Products via direct USB connection, so I am using LibUsbDotNet to interface with the devices. I have made the USB connections asyncronous so I can connect to multiple Products at the same time and process an arbitrary number at once.
public Class Product
{
public bool IsSoftwareUpdated = false;
public bool IsProductInformationCorrect = false;
public bool IsEOLProcessingCompleted = false;
public Product(){}
~Product()
}
public class ProcessProduct
{
List<Product> bagOfProducts = new List<Product>(new Product[10]);
ConcurrentBag<Product> UnprocessedUnits = new ConcurrentBag<Product>();
ConcurrentBag<Product> CurrentlyUpdating = new ConcurrentBag<Product>();
ConcurrentBag<Product> CurrentlyVerifyingInfo = new ConcurrentBag<Product>();
ConcurrentBag<Product> FinishedProcessing = new ConcurrentBag<Product>();
DispatcherTimer _timer = new DispatcherTimer();
public ProcessProduct()
{
_timer.Tick += Timer_Tick; //Every 1 second, call Timer_Tick
_timer.Interval = new TimeSpan(0,0,1); //1 Second timer
bagOfProducts.ForEach(o => UnprocessedUnits.Add(o)); //Fill the UnprocessedUnits with all products
StartProcessing();
}
private void StartProcessing()
{
_timer.Start();
}
private void Timer_Tick(object sender, EventArgs e)
{
ProductOrganizationHandler();
foreach(Product prod in CurrentlyUpdating.ToList())
{
UpdateProcessHandler(prod); //Async function that uses await
}
foreach(Product prod in CurrentlyVerifyingInfo.ToList())
{
VerifyingInfoHandler(prod); //Async function that uses Await
}
if(FinishedProcessing.Count == bagOfProducts.Count)
{
_timer.Stop(); //If all items have finished processing, then stop the process
}
}
private void ProductOrganizationHandler()
{
//Take(read REMOVE) Product from each ConcurrentBag 1 by 1 and moves that item to the bag that it needs to go
//depending on which process step is finished
//(or puts it back in the same bag if that step was not finished).
//E.G, all items are moved from UnprocessUnits to CurrentlyUpdating or CurrentlyVerifying etc.
//If a product is finished updating, it is moved from CurrentlyUpdating to CurrentlyVerifying or FinishedProcessing
}
private async void UpdateProcessHandler(Product prod)
{
await Task.Delay(1000).ConfigureAwait(false);
//Does some actual work validating USB communication and then running through the USB update
}
private async void VerifyingInfoHandler(Product prod)
{
await Task.Delay(1000).ConfigureAwait(false);
//Does actual work here and communicates with the product via USB
}
}
Full Compile-ready code example available via my code on Pastebin.
So, my question really is this: Are there any meaningful race conditions in this code? Specifically, with the ProductOrganizationHandler() code and the looping through the ConcurrentBags in Timer_Tick() (since a new call to Timer_Tick() happens every second). I'm sure this code works the majority of the time, but I am afraid of a hard-to-track bug later on that happens because of a rare race condition when, say, ProductOrganizationHandler() takes > 1 sec to run for some dumb reason.
As a secondary note: Is this even the best design pattern for this type of process? C# is my first OOP language and all self-taught on the job (nearly all of my job is Embedded C) so I don't have any formal experience with OOP design patterns.
My main goal is to asynchronously Update/Verify/Communicate with each device as it becomes available via USB. Once all products in the list are finished (or a timeout), then the process finishes. This project is in .NET 5.
EDIT: For anyone that comes along later with the same question, here's what I did.
I did not understand that DispatcherTimer add Ticks to the Dispatcher queue. This implies that a tick will only run if there is not already another instance of Tick already running or, worded another way, Timer_Tick will run to completion before the next Timer_Tick instance runs.
So, most(all?) of the Threading/concurrency concerns I had were unfounded and I can treat the Timer_Tick as a single-threaded non-concurrent function (which it is).
Also, to keep Ticks from piling up, I ran _timer.Stop() at the beginning of Timer_Tick and restarted the timer at the end of Timer_Tick.

First of all, you are using DispatchTimer, this will raise ticks on the UI thread. So as far as I can see there is no multi threading going on in the example. There are other timers, like System.Timers.Timer that raises events on a background thread if that is the intent. But if you just want to check and update status every so often, and are not running any code that blocks, just using the UI thread is fine and will simplify things a lot.
Even if we assume ProductOrganizationHandler did run on a worker thread, it would still be generally safe to remove items from one concurrent collection and putting them in another. But it would not guarantee that items are processed in any particular order, nor that any specific item is processed by a given tick of the timer. But since the timer will tick periodically all the items should eventually be processed. Keep in mind that most timers need to be disposed, so you need to handle that somehow, including if the processing is stopped prematurely.
Keep in mind that async does not mean concurrent, so I would not use it unless your USB library provides async methods. Even then I would avoid async void since this promotes exceptions to the captured synchronization context, potentially crashing the application, so it should mostly be used in the outermost layer, like button event handlers, or timers, and then you should probably handle exceptions somehow.
As for the best way to do it, I would take a look at DataFlow library.

System.Threading.Timer does not work correctly

I notice the timer is not correct.
This is a very simple C# code: it will print current date/time every 1 minute.
My expected result is: let it run at 3:30 PM then we will have: 3:31 PM, 3:32 PM, 3:33 PM, ...
But sometime don't receive above result: sometime it is 3:31 PM, 3:32 PM, 3:34 PM, ...
So it lost 1 row.
Could anyone point me what is problem?
class Program
{
static Timer m_Timer;
static int countDown;
static void Main(string[] args)
{
countDown = 60;
m_Timer = new Timer(TimerCallback, null, 0, 1000);
while (true) { System.Threading.Thread.Sleep(10); };
}
static void TimerCallback(Object o)
{
countDown -= 1;
if (countDown <= 0)
{
Console.WriteLine(" ===>>>>>" + System.DateTime.Now.ToString());
countDown = 60;
}
System.Threading.Thread.Sleep(10000); //long running code demo
}
}

System.Threading.Timer runs on threads from thread pool. You run callback function which runs on one thread in pool every 1s and block it for 10s using sleep. Depending on how many threads you have in thread pool at some timepoints they all may be blocked and wait or .NET should allocate new thread up to the maximum of threads in pool for you.
From comments extended answer.
Each function is independent and it does not wait until another processing finish. A simple task is: call a function to do something every 1 minutes. "do something" in my case is saving local variables into SQL server. This process is fast not slow. I use 1 timer for many functions because each function is schedule in different cycle. For example, function 1 is triggered every 1 minute, function 2 is triggered every 10 seconds ... That why I use the timer 1 second.
Your use case seems to be more complex as I read it from initial question. You have different tasks and try to implement sort of scheduler. Maybe each particular tasks is fast but all together some runs may be longer and blocking. Not sure how this logic was well implemented but there could be a lot of edge cases e.g. some run was missed etc.
How I would approach it?
I would not try to implement on my own if scheduler can be more complex. I would pick ready solution, e.g. Quartz.NET. They consider edge cases and help to scale on cluster with needed and help with config.
In any case I would refactor bigger schedule to have each task to run on its schedule based on configuration (custom implementation or Quartz) as smaller tasks
I would scale your "queue" of tasks first locally by introducing some queue, for example using ConcurrentQueue or BlockingCollection or any produce-consumer to limit number of threads and if performance of such execution is not good scale on cluster. By doing so you can at least guarantee that N tasks can be scheduled and executed locally and everything beyond is queued. Maybe having some priorities for tasks can also help because there might be execution which could be missed but there are execution which must run on schedule.
I doubt it is a good idea to start from thread timer execution other threads or tasks if most likely you already have problems with threading.
You problem is not with System.Threading.Timer, it does its job well. Your use case is more complex.

Windows - is not real time operating system. So, if you expect that timer waits ecactly 1 second - it's wrong. There are many reasonsm when timer can wait more time. Because of timer resolution or other high load operations.

If you like newer .NET TPL syntax yo can write it like this:
using System;
using System.Threading.Tasks;
namespace ConsoleApp1
{
internal class Program
{
private static void Main(string[] args)
{
Repeat(TimeSpan.FromSeconds(10));
Console.ReadKey();
}
private static void Repeat(TimeSpan period)
{
Task.Delay(period)
.ContinueWith(
t =>
{
//Do your staff here
Console.WriteLine($"Time:{DateTime.Now}");
Repeat(period);
});
}
}
}

The above code causes, that every second you run 10-second "demo" (sleep). You will run 10 worker threads simultanously.
Are you sure, this is what you are trying to achieve?
To see what really happens in your app, simply add:
Console.WriteLine($"Time:{DateTime.Now.ToString("hh:mm:ss.fff tt")},Thread:{Thread.CurrentThread.ManagedThreadId},countDown:{countDown}");
in the beginning of TimerCallback. You will notice, that timespan between following callbacks are not exactly 1000ms (usually it is a little bit more). This is perfectly normal in non-rtc OS, and, in most cases - it's not a problem. Just keep in mind, that Timer is not exact.
Moreover, if you are trying to use Timer that way, and trying to count ticks - these little errors cumulates in following ticks.

I just post what found here for people that have problem like me.
I found the answer from another thread.
I use "HighResolutionTimer.cs" and it works perfect:
https://gist.github.com/DraTeots/436019368d32007284f8a12f1ba0f545

Thread.Sleep - how to use it in a proper way?

System.Threading.thread.Sleep(1000); pauses a whole program for 1 second, but when this second is over it does everything what could be done for this period. For example:
Thread.Sleep(1000);
Console.WriteLine("A");
Thread.Sleep(1000);
Console.Writeline("B");
It will wait two seconds and write
A
B
How to use the pause properly?

If you want something to happen once per second, then create a timer. For example:
private System.Threading.Timer _timer;
void main()
{
_timer = new Timer(TimerTick, null, 1000, 1000);
// do other stuff in your main thread
}
void TimerTick(object state)
{
// do stuff here
}
There are several different types of timers. If you're writing a console program, then I would suggest using System.Threading.Timer. In a Windows Forms application, either System.Windows.Forms.Timer or System.Timers.Timer. See Timers for more information.

Thread.Sleep() behaves just like you would think; it just pauses the current thread for approximately the given number of milliseconds.
The problem here is that the standard output stream does not necessarily flush to the console (or wherever it is pointed at) to on each call to Write. Instead, it may buffer some content so as to write it out in larger chunks for efficiency. Try calling Console.Out.Flush(); after each WriteLine() and you should see the results you expect.

Perform a event certain so many times every so many seconds

I am currently working on a c# project where I need to perform a task so many times every so many seconds.
For example, write to the console 5 times every 1 second. How could I go about doing this.
Thanks for any help you can provide.

You can use a Timer and bind and event to the Timer.Elapsed event.
using System.Timers;
Sample code:
Timer timer = new Timer();
timer.Elapsed += (sender, eventArgs) =>
{
for (int i = 0; i < 5; i++)
{
Console.Write(i);
}
};
Is this an console application, or do you run this on another thread?

For short stuff like this, use a timer. There are two main ones to choose from:
System.Threading.Timer
And:
System.Windows.Forms.Timer
The former uses the ThreadPool, the latter uses UI events. They both expose the ability to specify and interval and either a callback method or event to attach to in order to put custom code.
For longer periods of inactivity, look into scheduling either with the Windows scheduler (the OS one) or a scheduling framework such as Quartz.NET.
Do note that the accuracy of the timers vary, but not really within margins that humans can detect :-)
Also note that the callback of the threaded timer will return on an arbitrary ThreadPool thread, so you could effectively end up "multi-threading" the code without realising it.
There is also System.Timers.Timer, it exposes an event. An article about the different timers available can be found here.

I don't think it will affect you, but it's still worth knowing - windows is not a real-time OS; if you ask for something to be done every X milliseconds, it won't be exact, as for how much it will be out depends on a variety of things.

You could create a thread to do it. Especially useful if you want to do a lot of processing! Here's an example of a thread doing work every 1s (1000ms):
public void Start()
{
running = true;
thread = new Thread(new ParameterizedThreadStart(ThreadFunction));
thread.Start();
}
public virtual void ThreadFunction(object o)
{
var insert = false;
while (running)
{
//Do work
Thread.Sleep(1000);
}
}

try this
While (true)
{
for (int i=0; i<5; i++)
{
console ("Hello");
}
//this will pause for 1 sec (1000msec)
Thread.sleep(1000);
}

contextswitchdeadlock

Whilst debugging my program in VS 2008 I have come across the following error:
The CLR has been unable to transition from COM context 0x34fc1a0 to COM context 0x34fc258 for 60 seconds. The thread that owns the destination context/apartment is most likely either doing a non pumping wait or processing a very long running operation without pumping Windows messages. This situation generally has a negative performance impact and may even lead to the application becoming non responsive or memory usage accumulating continually over time. To avoid this
It appears to be deadlocking even though the code only contains a simple C# timer: See Snippet Below:
private void RequestWork()
{
// The timer will be re-intialised if there are still no wating jobs in the database
StopTimer();
// assign all the threads some work
InitialiseTimer();
}
/// <summary>
/// Initialise a timer with a timer interval configured from app.config. Enable the timer and
/// register an appropriate event handler
/// </summary>
private void InitialiseTimer()
{
if (m_Timer == null)
{
// look up the default backoff time from the config
string backOffInt = ConfigurationSettings.AppSettings["BackOffInterval"];
int backoffInterval = 1000;
m_Timer = new System.Timers.Timer();
// set the timer interval to 5 seconds
m_Timer.Interval = backoffInterval;
m_Timer.Elapsed += new ElapsedEventHandler(m_Timer_Elapsed);
}
m_Timer.Enabled = true;
}
private void StopTimer()
{
if (m_Timer != null)
{
m_Timer.Enabled = false;
}
}
void m_Timer_Elapsed(object p_Sender, ElapsedEventArgs p_E)
{
RequestWork();
}
As far as I know the timer should run, elapse and then initialise again, I can see no local reason for a deadlock.
I am aware of how to turn this error msg off but feel that this is not a solution, instead it is masking the problem.

You can turn this off if you think you've definitely not got a deadlock situation:
Debug->Exceptions->Managed Debug Assistants menu in Visual Studio and uncheck the ContextSwitchDeadlock

This is an infinite loop. You need to let your application pump some messages at least once every 60 seconds to prevent this exception to happen.
Try calling System.Threading.Thread.CurrentThread.Join(10) once in a while. There are other calls you can do that let the messages pump.

It seems that you are adding a new event handler each time you call InitialiseTimer. That way m_Timer_Elapsed will be called as many times as it has been added.
You should add the event handler just one time.

If your application hangs or not reponse even after you uncheck the box against contextswitchdeadlock. Put the following line before call of method or for loop.
In C#
System.Windows.Forms.Application.DoEvents();
and VB.NET / VB / ASP.NET
DoEvents()

Couple thoughts/questions:
1) The code snippet looks like your interval is every 1 second (not 5 as mentioned in the comments).
2) The big question is what is RequestWork() doing?
Without knowing what RequestWork() is doing, we can't really comment on why you are seeing a ContextSwitchDeadlock.
Somethings to think about with respect to this method
a) how long does it take?
b) is it accessing GUI elements?
Some MSDN comments on Elapsed:
If you use the Timer with a user
interface element, such as a form or
control, assign the form or control
that contains the Timer to the
SynchronizingObject property, so that
the event is marshaled to the user
interface thread.
-and-
The Elapsed event is raised on a
ThreadPool thread. If processing of
the Elapsed event lasts longer than
Interval, the event might be raised
again on another ThreadPool thread.
Thus, the event handler should be
reentrant.
I'm thinking since you have a 1 second timer, you might want to look into what happens in RequestWork and see how long its taking.