We Keep Coding

How to concurrently take items from the BlockingCollection?

I have a BlockingCollection of integers.
Below is the code I am trying to develop to remove the items concurrently from the BlockingCollection.
static void Produce()
{
for (int i = 0; i < 100000; i++)
{
bc3.Add(i);
}
Run();
}
static void Run()
{
for (int i = 0; i < 5; i++)
{
Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning);
}
}
static void Process()
{
var stopWatch = new Stopwatch();
stopWatch.Start();
while (bc3.Count!= 0)
{
bc3.Take();
}
stopWatch.Stop();
Console.WriteLine("Thread {0}",
Thread.CurrentThread.ManagedThreadId);
Console.WriteLine("Elapsed Time {0}", stopWatch.Elapsed.Seconds);
}
Is this the correct way remove items in a faster way by creating 5 tasks?

Stopwatch issue
Your measurement results are wrong because you use
stopWatch.Elapsed.Seconds
instead of
stopWatch.ElapsedMilliseconds
You only display the seconds, but ignore the minutes, hours, etc.
Concurrency issue
No, this is not the correct way of removing items from a BlockingCollection. The statement that does not work is
bc3.Count != 0
All 5 tasks may check this condition at the same time, finding out that there is 1 item left. They all 5 go to
bc3.Take();
One task is able to remove the item, the other 4 tasks are waiting.
One way to solve this is to add
bc3.CompleteAdding();
in Produce().
Garbage collection issue
Once the first task finishes, the Run() method completes and all tasks in the method go out of scope and are garbage collected. This may make you see only 1 instead of 5 completion messages.
To fix this, use
static void Run()
{
var tasks = new List<Task>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning));
}
try
{
Task.WaitAll(tasks.ToArray());
}
catch (AggregateException)
{ }
}
Cost of synchronization
Here's one of my outputs with 5 tasks (and 100000000 items):
Thread 11
Thread 13
Thread 12
Thread 14
Thread 15
Elapsed Time 12878
Elapsed Time 13122
Elapsed Time 13128
Elapsed Time 13128
Elapsed Time 13128
Run: 13140
Compare this to a single task:
Thread 12
Elapsed Time 10147
Run: 10149
This is because only one Take() method can remove an item and it takes additional time for synchronization.

C# Multi Thread and Read/Write large text file

I wan to speed up my tool with multi thread to read/write text file
Here how I start - I think I need 7 thread
for (int i = 0; i < 7; i++)
{
Thread thread = new Thread(() => process(files[i], i));
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
}
And my func
void process(string file,int threadnumber)
{
// display thread number
// filter text
// read - write
}
But when I start:
Thread: 3 start
Thread: 3 start
Thread: 3 start
Thread: 4 start
Thread: 5 start
Thread: 6 start
Thread: 7 start
Why my tool don't start thread 1 2 3 4 5 6 7.. few threads duplicate - meaning read and write same file and error.
Please give me advise.

Depending on how fast the loop iterates and how fast a thread starts, there is a chance that by the time that a thread actually starts the i variable might have changed.
You must declare a new variable, for each thread, that is not changed:
for (int i = 0; i < 7; i++)
{
int j = i; //New variable set to the current 'i'.
Thread thread = new Thread(() => process(files[j], j));
thread.SetApartmentState(ApartmentState.STA);
thread.Start();
}
In the above the j-variable will get the same value as the current i, and won't change even if the loop continues to iterate.

Again on using interlocked

I read a lot of answers on this point but I did not found the solution yet.
I have a class with a counter attribute having a problem with cached values. Even volatile don't seems to work:
public class MyClass {
private Timer _timer;
private int _threadsCounter = 0;
public StreamWriter Tracer { get; set; }
public MyClass() {
_timer = new Timer(1000.0 * 10);
_timer.AutoReset = true;
_timer.Elapsed += new ElapsedEventHandler(OnTimer);
_timer.Start();
}
private void OnTimer(object sender, ElapsedEventArgs e) {
HashSet<Task> taskPool = new HashSet<Task>();
try {
if (Tracer != null) Tracer.WriteLine("[{0}] onTimer start. Current threads counter is {1}.", DateTime.Now, _threadsCounter);
if (_threadsCounter >= 10) return;
// create parallel tasks
for (int i = 0; i < 8; i++) {
// limit on the max num of parallel processing but the counter remains unchanged during this timer event!!!
if (_threadsCounter >= 10) break;
var timeout = (30 + i * 2);
var task = Task.Run(() => {
var localCounter = System.Threading.Interlocked.Increment(ref _threadsCounter);
try {
System.Threading.Thread.Sleep(timeout * 1000);
}
finally {
System.Threading.Interlocked.Decrement(ref _threadsCounter);
}
});
taskPool.Add(task);
}
}
finally {
if (Tracer != null)
Tracer.WriteLine("[{0}] onTimer end. Created {1} tasks. Current threads counter is {2}.", DateTime.Now, taskPool.Count, _threadsCounter);
}
}
Well, it seems that the onTimer caches the _threadsCounter variable as the output is:
[14:10:47] onTimer start. Current threads counter is 0.
[14:10:47] onTimer end. Created 8 tasks. Current threads counter is 0.
[14:10:57] onTimer start. Current threads counter is 8.
[14:10:57] onTimer end. Created 8 tasks. Current threads counter is 8.
[14:11:07] onTimer start. Current threads counter is 16.
[14:11:07] onTimer end. Created 0 tasks. Current threads counter is 16.
[14:11:17] onTimer start. Current threads counter is 15.
[14:11:17] onTimer end. Created 0 tasks. Current threads counter is 15.
[14:11:37] onTimer start. Current threads counter is 4.
[14:11:37] onTimer end. Created 8 tasks. Current threads counter is 4.
[14:11:47] onTimer start. Current threads counter is 8.
[14:11:47] onTimer end. Created 8 tasks. Current threads counter is 8.
[14:11:57] onTimer start. Current threads counter is 16.
[14:11:57] onTimer end. Created 0 tasks. Current threads counter is 16.
Why do I arrive to 16?
I solved the problem by changing a bit the code:
var localCounter = _threadsCounter;
...
if ((localCounter + taskPool.Count) >= 10) break;
But why this behavior?

Task.Run doesn't start the task immediately. It adds the task to the thread pool queue and returns.
In your case, the whole for loop is executed before the new tasks start running, so nothing changes _threadsCounter. That's why volatile doesn't help.

You're effectively testing for the number of tasks that have actually started and got as far as incrementing the counter. That takes a little while - so basically you're creating all 8 tasks and starting them, then they're incrementing the counter... by which time it's too late to notice that you've got more than 10.
The better solution would be to increment the counter before you start the task:
// Increment the counter in expectation of starting a task
var localCounter = Interlocked.Increment(ref _threadsCounter);
if (localCounter >= 10)
{
// Ah, we're not going to start a task after all, so undo
// the increment
Interlocked.Decrement(ref _threadsCounter);
break;
}
else
{
// Start a task, which will decrement the counter at the end.
// (You could add the "decrement" bit as a continuation, even...)
}

stopwatch c# behave differently in different threads?

I'm currently using a stopwatch as a global timer. I have main thread running, another thread, and an event method.
The main thread launches the other thread and the event method is triggered by events. Both methods will call the stopwatch and get its time. The thing is, the times are not consistent:
from main thread:
START REC AT 9282
STOp REC AT 19290
from another thread:
audio 1
audio 304
audio 354
audio 404
audio 444
audio 494
audio 544
audio 594
from event method:
video 4
video 5
video 29
video 61
video 97
video 129
video 161
I don't get why if i start my rec at 9282, the other two functions that call the stopwatch will have timers that start at zero? Is this a thread related issue? How can i fix this? Thanks
UPDATE:*********
when i save my frames i changed to:
long a = relogio.EllapseMilliseconds
i print out this value and its ok, as expected. but when i print the value stored in the lists, they come as starting from the beggining. strange huh?
SORRY FOR ALL THE TROUBLE, I PRINTED IT WITHOUT THE STARTING TIME,THATS WHY THEY ALL SEEMED TO START FROM ZERO! MANY THANKS AND SORRY!
main thread
private void Start_Recording_Click(object sender, RoutedEventArgs e)
{
rec_starting_time = relogio.ElapsedMilliseconds;
Console.WriteLine("START REC AT " + rec_starting_time);
write_stream.enableRecording();
Thread a = new Thread(scheduleAudioVideoFramePicks);
a.Start();
scheduleAudioVideoFramePicks - this thread just counts the time, so i know when to stop
//while....
if (rec_starting_time + time_Actual > rec_starting_time+recording_time * 1000)//1000 - 1s = 1000ms
{
totalRecordingTimeElapsed = true;
write_stream.disableRecording();
Console.WriteLine("STOp REC AT " + relogio.ElapsedMilliseconds);
}
//end while
lock (list_audio)
{
int b = 0;
//print time of frames gathered
foreach(AudioFrame item in list_audio){
Console.WriteLine("audio " + (item.getTime() - rec_starting_time));
}
lock (list_video)
{
}
foreach (VideoFrame item in list_video)
{
Console.WriteLine("video " + (item.getTime() - rec_starting_time));
}
}
the another thread, where i get the time
if (write_stream.isRecording())
{
list_audio.Enqueue(new AudioFrame(relogio.ElapsedMilliseconds, audioBuffer));
}
event method
if (write_stream.isRecording())
{
list_video.Add(new VideoFrame(relogio.ElapsedMilliseconds, this.colorPixels));
}~
i dont know if this is relevant, but i start my stopwatch like this
public MainWindow()
{
InitializeComponent();
//some code
this.relogio = new Stopwatch();
relogio.Start();
}

Stopwatch is not threadsafe, particularly for 32-bit programs.
It uses the Windows API call QueryPerformanceCounter() to update a private long field. On 32-bit systems you could get a "torn read" when one thread reads the long value while another thread is updating it.
To fix that, you'd have to put a lock around access to the Stopwatch.
Also note that one some older systems there were bugs where inconsistent values could be returned from different threads calling QueryPerformanceCounter(). From the documentation:
On a multiprocessor computer, it should not matter which processor is called. However, you can get different results on different processors due to bugs in the basic input/output system (BIOS) or the hardware abstraction layer (HAL). To specify processor affinity for a thread, use the SetThreadAffinityMask function.
I have never encountered this bug myself, and I don't think it's very common.
What results do you get with the following test program? The times should be mostly increasing in value, but you are liable to get one or two out of order just because their threads get rescheduled just after they've read a value and before they add it to the queue.
namespace Demo
{
class Program
{
Stopwatch sw = Stopwatch.StartNew();
object locker = new object();
ConcurrentQueue<long> queue = new ConcurrentQueue<long>();
Barrier barrier = new Barrier(9);
void run()
{
Console.WriteLine("Starting");
for (int i = 0; i < 8; ++i)
Task.Run(()=>test());
barrier.SignalAndWait(); // Make sure all threads start "simultaneously"
Thread.Sleep(2000); // Plenty of time for all the threads to finish.
Console.WriteLine("Stopped");
foreach (var elapsed in queue)
Console.WriteLine(elapsed);
Console.ReadLine();
}
void test()
{
barrier.SignalAndWait(); // Make sure all threads start "simultaneously".
for (int i = 0; i < 10; ++i)
queue.Enqueue(elapsed());
}
long elapsed()
{
lock (locker)
{
return sw.ElapsedTicks;
}
}
static void Main()
{
new Program().run();
}
}
}
Having said all that, the most obvious answer is that in fact you aren't sharing a single Stopwatch between the threads, but instead you have accidentally started a new one for each thread...

Run Tasks on a separate thread with return value

I'm developping an application that should run a new task , makes some stuff and return a list of objects...
when spawning the new task , i need to simulate in my UI thread a progressbar changing in a for loop... and after the task completion , i have to update my view ( by updating the values of my ViewModel properties as i'm using MVVM pattern).
the problem is that when the task is launched, the GUI freezes and the progress bar is not updated! i thought that tasks are run in background on a sperate thread ( as they are pooled). so why my GUI is blocked??
here is a snippet of my code :
Task<List<Items>> myTask = Task.Factory.StartNew(() => queryHandler.GetItemssStatistics(items, sec, buyer, seller, From, To));
//Here is the simulation of ProgressBar changing that i need to be done when task is started
for (int i = 0; i < 100; i++)
{
Thread.Sleep(100);
StatusValue = i+1;
}
//Here is the section i need to do once the task finishes its execution to update my VM
//Wait for the task to finish its execution and retreive results:
List<Items> _tempResults = myTask.Result;
foreach (Items d in _tempResults)
{
ResultsList.Add(d);
}
if (ResultsList.Count == 0)
{
MessageBox.Show("Sorry, No items matched the defined research criteria!", "Warning",
MessageBoxButton.OK,
MessageBoxImage.Warning, MessageBoxResult.OK, MessageBoxOptions.ServiceNotification);
}
else
{
//Show items:
ResultsAvailable = Visibility.Visible;
}

There are a few problems with your code:
You're using Thread.Sleep on the UI thread
You're calling Task<T>.Result again on the UI thread - which blocks until the task has been completed.
You can do a few things:
Use ProgressBar.IsIndeterminate = true to show a marquee-style progress bar instead of faking the progress update or use a DispatcherTimer to update the progress bar.
Use task continuations to perform whatever code that needs to happen after the task completes. Alternatively, if you're using .NET 4.5, you can use the new async/await features of the language.
To use async/await:
async Task GetItems()
{
List<Items> tempResults = await Task.Run(() => queryHandler.GetItemssStatistics(...));
// the following will be executed on the UI thread automatically
// after the task is done, keeping the UI responsive
foreach (Items d in _tempResults)
{
ResultsList.Add(d);
}
}
To use a continuation:
Task.Factory.StartNew(() => queryHandler.GetItemssStatistics(...))
.ContinueWith(task => ProcessResults(task.Result), TaskScheduler.FromCurrentSynchronizationContext());
The FromCurrentSynchronizationContext method creates a task scheduler that uses SynchronizationContext to marshal the call back to the UI thread.
Clearly using await/async is much easier, and should be your preference if it's available.

You are still stopping the UI from working, because you are calling Thread.Sleep on it. If you want your progress-bar to update, run another thread, or try to integrate the increasing of the progress-bar in your task.
Looking at your code, it would be better to do the update of the progress-bar in your task!
You are making the user wait for 10000 ms, if your task finishes before, the user has to wait...
Access the dispatcher of the progress-bar in your task, when one items has been progressed and do like
//process item
if(progressBar.Dispatcher.CheckAccess()){
progressBar.Value += 1; //maybe calculate the number of items you are processing
}
else {
//send a delegate to the dispatcher
MyDelegate myDel = new MyDelegate(delegate() {
progressBar.Value += 1;
});
progressBar.Dispatcher.Invoke(myDel);
}
The delegate's signature could be something like:
public void MyDelegate();