We Keep Coding

c# lock not working as expected

This class uses lock and Interlocked.
Both increaseCount.with_lock.Run(); and increaseCount.with_interlock.Run(); prints between 96-100.
I am expecting both of them to print always 100. What did I make mistake?
public static class increaseCount {
public static int counter = 0;
public static readonly object myLock = new object();
public static class with_lock {
public static void Run() {
List<Thread> pool = new List<Thread>();
for(int i = 0; i < 100; i++) {
pool.Add(new Thread(f));
}
Parallel.ForEach(pool, x => x.Start());
Console.WriteLine(counter); //should print 100
}
static void f() {
lock(myLock) {
counter++;
}
}
}
public static class with_interlock {
public static void Run() {
List<Thread> pool = new List<Thread>();
for(int i = 0; i < 100; i++) {
pool.Add(new Thread(f));
}
Parallel.ForEach(pool, x => x.Start());
Console.WriteLine(counter);//should print 100
}
static void f() {
Interlocked.Add(ref counter, 1);
}
}
}

In both cases, you start up your threads but you don't wait for them to complete so you don't reach the 100 before you print the result and the app closes.
If after you start all thread you would wait for all these threads to complete with Thread.Join you would always get the correct result:
List<Thread> pool = new List<Thread>();
for (int i = 0; i < 100; i++)
{
pool.Add(new Thread(f));
}
Parallel.ForEach(pool, x => x.Start());
foreach (var thread in pool)
{
thread.Join();
}
Console.WriteLine(counter);
Note: This seems like a test of some kind, but you should know that blocking multiple threads on a single lock is a huge waste of resources.

I believe it's because your Parallel.Foreach call simply calls start on all the threads in pool but they haven't necessarily completed by the time the loops finished and the Console.WriteLine is called. If you were to insert a Thread.Sleep(5000); // 5s sleep or similar before the Console.WriteLine it would likely always print out what you expect.

Your code is fine. The only problem is your expectation. Basically, not all 100 threads get to run untill the counter is displayed. Try putting a Thread.Sleep(1000) before the Console.WriteLine(counter) and you shall see what I mean.
Edit: wrongly posted as a comment the first time.

Get Notification when all threads finish work

I am new to threading concept, using threading first time in my application. One of my application processing multiple data, without threading it is taking approx 2 minutes while with help of threading it is taking just 25 seconds, but i want notification when all threads finishes work.
private int z = 0 ;
startfunction()
{
z= 250;
start1step(z);
}
private void start1step(int i)
{
if (i < 0)
return;
else
{
Thread thread = new Thread(new ThreadStart(WorkThreadFunction));
thread.Start();
start1step( --i);
}
}
public void WorkThreadFunction( )
{
try
{
int x = z ;
z-- ;
// do some background work
if(x ==0)
MessageBox.Show("All Thread finished");
}
catch
{
//
}
}
Above sample code is working perfectly except that notification part, I want notification when all threads finishes background work. There is one last step left which sums up work finished by all these threads.
Please help

There are lots of ways to do this. I would say the two most convenient methods that remain similar to your current implementation involve using a CountDownEvent or switching to the Task class for your threading. A third method involves using the Parallel.ForEach() method, and that might actually suit your specific scenario better.
CountDownEvent looks like this:
CountDownEvent countDown = new CountDownEvent(250);
startfunction()
{
countDown.Reset();
start1step(250);
countDown.Wait();
}
private void start1step(int i)
{
while (i-- > 0)
{
new Thread(WorkThreadFunction, i).Start();
}
}
public void WorkThreadFunction(object o)
{
int x = (int)o;
try
{
// do some background work
}
catch
{
//
}
finally
{
countDown.Signal();
}
}
Task looks like this:
startfunction()
{
Task[] tasks = new Task[250];
start1step(tasks);
Task.WaitAll(tasks);
}
private void start1step(Task[] tasks)
{
for (int i = 0; i < tasks.Length; i++)
{
int taskParam = i;
tasks[i] = Task.Run(() => WorkThreadFunction(taskParam));
}
}
public void WorkThreadFunction(int x)
{
try
{
// do some background work
}
catch
{
//
}
}
Parallel.ForEach() looks like this:
startfunction()
{
Parallel.ForEach(Enumerable.Range(0, 250), i => WorkThreadFunction(i));
}
public void WorkThreadFunction(int x)
{
try
{
// do some background work
}
catch
{
//
}
}

In your main thread you can do something like this...
List<Thread> threads = new List<Thread>();
// CREATE THREADS and add them to the list of threads
while (threads.Any(x => x.IsAlive))
{
Thread.Sleep(500);
}
Console.WriteLine("done");
Essentially, keep track of the threads and check their status every now and then in a way that doesn't block the main thread.

.NET 2.0 Processing very large lists using ThreadPool

This is further to my question here
By doing some reading .... I moved away from Semaphores to ThreadPool.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace ThreadPoolTest
{
class Data
{
public int Pos { get; set; }
public int Num { get; set; }
}
class Program
{
static ManualResetEvent[] resetEvents = new ManualResetEvent[20];
static void Main(string[] args)
{
int s = 0;
for (int i = 0; i < 100000; i++)
{
resetEvents[s] = new ManualResetEvent(false);
Data d = new Data();
d.Pos = s;
d.Num = i;
ThreadPool.QueueUserWorkItem(new WaitCallback(Process), (object)d);
if (s >= 19)
{
WaitHandle.WaitAll(resetEvents);
Console.WriteLine("Press Enter to Move forward");
Console.ReadLine();
s = 0;
}
else
{
s = s + 1;
}
}
}
private static void Process(object o)
{
Data d = (Data) o;
Console.WriteLine(d.Num.ToString());
Thread.Sleep(10000);
resetEvents[d.Pos].Set();
}
}
}
This code works and I am able to process in the sets of 20. But I don't like this code because of WaitAll. So let's say I start a batch of 20, and 3 threads take longer time while 17 have finished. Even then I will keep the 17 threads as waiting because of the WaitAll.
WaitAny would have been good... but it seems rather messy that I will have to build so much of control structures like Stacks, Lists, Queues etc in order to use the pool efficiently.
The other thing I don't like is that whole global variable in the class for resetEvents. because this array has to be shared between the Process method and the main loop.
The above code works... but I need your help in improving it.
Again... I am on .NET 2.0 VS 2008. I cannot use .NET 4.0 parallel/async framework.

There are several ways you can do this. Probably the easiest, based on what you've posted above, would be:
const int MaxThreads = 4;
const int ItemsToProcess = 10000;
private Semaphore _sem = new Semaphore(MaxThreads, MaxThreads);
void DoTheWork()
{
int s = 0;
for (int i = 0; i < ItemsToProcess; ++i)
{
_sem.WaitOne();
Data d = new Data();
d.Pos = s;
d.Num = i;
ThreadPool.QueueUserWorkItem(Process, d);
++s;
if (s >= 19)
s = 0;
}
// All items have been assigned threads.
// Now, acquire the semaphore "MaxThreads" times.
// When counter reaches that number, we know all threads are done.
int semCount = 0;
while (semCount < MaxThreads)
{
_sem.WaitOne();
++semCount;
}
// All items are processed
// Clear the semaphore for next time.
_sem.Release(semCount);
}
void Process(object o)
{
// do the processing ...
// release the semaphore
_sem.Release();
}
I only used four threads in my example because that's how many cores I have. It makes little sense to be using 20 threads when only four of them can be processing at any one time. But you're free to increase the MaxThreads number if you like.

So I'm pretty sure this is all .NET 2.0.
We'll start out defining Action, because I'm so used to using it. If using this solution in 3.5+, remove that definition.
Next, we create a queue of actions based on the input.
After that we define a callback; this callback is the meat of the method.
It first grabs the next item in the queue (using a lock since the queue isn't thread safe). If it ended up having an item to grab it executes that item. Next it adds a new item to the thread pool which is "itself". This is a recursive anonymous method (you don't come across uses of that all that often). This means that when the callback is called for the first time it will execute one item, then schedule a task which will execute another item, and that item will schedule a task that executes another item, and so on. Eventually the queue will run out, and they'll stop queuing more items.
We also want the method to block until we're all done, so for that we keep track of how many of these callbacks have finished through incrementing a counter. When that counter reaches the task limit we signal the event.
Finally we start N of these callbacks in the thread pool.
public delegate void Action();
public static void Execute(IEnumerable<Action> actions, int maxConcurrentItems)
{
object key = new object();
Queue<Action> queue = new Queue<Action>(actions);
int count = 0;
AutoResetEvent whenDone = new AutoResetEvent(false);
WaitCallback callback = null;
callback = delegate
{
Action action = null;
lock (key)
{
if (queue.Count > 0)
action = queue.Dequeue();
}
if (action != null)
{
action();
ThreadPool.QueueUserWorkItem(callback);
}
else
{
if (Interlocked.Increment(ref count) == maxConcurrentItems)
whenDone.Set();
}
};
for (int i = 0; i < maxConcurrentItems; i++)
{
ThreadPool.QueueUserWorkItem(callback);
}
whenDone.WaitOne();
}
Here's another option that doesn't use the thread pool, and just uses a fixed number of threads:
public static void Execute(IEnumerable<Action> actions, int maxConcurrentItems)
{
Thread[] threads = new Thread[maxConcurrentItems];
object key = new object();
Queue<Action> queue = new Queue<Action>(actions);
for (int i = 0; i < maxConcurrentItems; i++)
{
threads[i] = new Thread(new ThreadStart(delegate
{
Action action = null;
do
{
lock (key)
{
if (queue.Count > 0)
action = queue.Dequeue();
else
action = null;
}
if (action != null)
{
action();
}
} while (action != null);
}));
threads[i].Start();
}
for (int i = 0; i < maxConcurrentItems; i++)
{
threads[i].Join();
}
}

high cpu usage on windows service application

I recently started on a new job and there is a windows service here that consumes messages from a private windows queue. This service consumes the messages only from 9am to 6pm. So, during 7pm to 8:59am it accumulates a lot of messages on the queue. When it starts processing at 9pm, the cpu usage of the service goes to high(98, 99 percent), screwing with the server's performance.
This service use threads to process the messages of the queue, but as I had never worked with threads before I am a little lost.
Here's the part of code that I am sure this is happening:
private Thread[] th;
//in the constructor of the class, the variable th is initialized like this:
this.th = new Thread[4];
//the interval of this method calling is 1sec, it only goes high cpu usage when there is a lot of messages in the queue
public void Exec()
{
try
{
AutoResetEvent autoEvent = new AutoResetEvent(false);
int vQtd = queue.GetAllMessages().Length;
while (vQtd > 0)
{
for (int y = 0; y < th.Length; y++)
{
if (this.th[y] == null || !this.th[y].IsAlive)
{
this.th[y] = new Thread(new ParameterizedThreadStart(ProcessMessage));
this.th[y].Name = string.Format("Thread_{0}", y);
this.th[y].Start(new Controller(queue.Receive(), autoEvent));
vQtd--;
}
}
}
}
catch (Exception ex)
{
ExceptionPolicy.HandleException(ex, "RECOVERABLE");
}
}
EDIT: I am trying the second approach posted by Brian Gideon. But I'll by honest: I'm deeply confused with the code and I don't have a clue about what it's doing.
I haven't changed the way the 4 threads are created and the other code I showed, just changed my Exec(exec is the method called every second when it's 9am to 6pm) method to this:
public void Exec()
{
try
{
AutoResetEvent autoEvent = new AutoResetEvent(false);
int vQtd = queue.GetAllMessages().Length;
while (vQtd > 0)
{
for (int i = 0; i < 4; i++)
{
var thread = new Thread(
(ProcessMessage) =>
{
while (true)
{
Message message = queue.Receive();
Controller controller = new Controller(message, autoEvent);
//what am I supposed to do with the controller?
}
});
thread.IsBackground = true;
thread.Start();
}
vQtd--;
}
}
catch (Exception ex)
{
ExceptionPolicy.HandleException(ex, "RECOVERABLE");
}
}

Ouch. I have to be honest. That is not a very good design. It could very well be spinning around that while loop waiting for previous threads to finish processing. Here is a much better way of doing it. Notice that the 4 threads are only created once and hang around forever. The code below uses the BlockingCollection from the .NET 4.0 BCL. If you are using an earlier version you can replace it with Stephen Toub's BlockingQueue.
Note: Further refactoring may be warranted in your case. This code tries to preserve some common elements from the original.
public class Example
{
private BlockingCollection<Controller> m_Queue = new BlockingCollection<Controller>();
public Example()
{
for (int i = 0; i < 4; i++)
{
var thread = new Thread(
() =>
{
while (true)
{
Controller controller = m_Queue.Take();
// Do whatever you need to with Contoller here.
}
});
thread.IsBackground = true;
thread.Start();
}
}
public void Exec()
{
try
{
AutoResetEvent autoEvent = new AutoResetEvent(false);
int vQtd = Queue.GetAllMessages().Length
while (vQtd > 0)
{
m_Queue.Add(new Controller(Queue.Receive(), autoEvent));
}
}
catch (Exception ex)
{
ExceptionPolicy.HandleException(ex, "RECOVERABLE");
}
}
}
Edit:
Or better yet since MessageQueue is thread-safe:
public class Example
{
public Example()
{
for (int i = 0; i < 4; i++)
{
var thread = new Thread(
() =>
{
while (true)
{
if (/* between 9am and 6pm */)
{
Message message = queue.Receive();
Controller controller = new Controller(message, /* AutoResetEvent? */);
// Do whatever you need to with Contoller here.
// Is the AutoResetEvent really needed?
}
}
});
thread.IsBackground = true;
thread.Start();
}
}
}

The method you show runs in a tight loop when all threads are busy. Try something like this:
while (vQtd > 0)
{
bool full = true;
for (int y = 0; y < th.Length; y++)
{
if (this.th[y] == null || !this.th[y].IsAlive)
{
this.th[y] = new Thread(new ParameterizedThreadStart(ProcessMessage));
this.th[y].Name = string.Format("Thread_{0}", y);
this.th[y].Start(new Controller(queue.Receive(), autoEvent));
vQtd--;
full = false;
}
}
if (full)
{
Thread.Sleep(500); // Or whatever it may take for a thread to become free.
}
}

You have two options. Either you insert delays after each message with Thread.Sleep() or lower the thread priority of the polling threads. If you lower the thread priority the CPU usage will still be high, but should not affect performance that much.
Edit: or you can lower the number of threads from 4 to 3 to leave one core for other processing (assuming you have a quad core). This of course reduces your dequeuing throughput.
Edit2: or you could rewrite the whole think with task parallel library if you are running .NET 4. Look for Parallel.ForEach(). That should save you from some of the footwork if you are not familiar with threads.

Threaded simultaneous jobs

There is a string array myDownloadList containing 100 string URIs. I want to start 5 thread jobs that will pop next URI from myDownloadList (like a stack) and do something with it (download it), until there is no URIs left on a stack (myDownloadList).
What would be the best practice to do this?

Use the ThreadPool, and just setup all of your requests. The ThreadPool will automatically schedule them appropriately.
This will get easier with .NET 4, using the Task Parallel Library. Setting up each request as a Task is very efficient and easy.

Make sure each thread locks the myDownloadList when accessing it. You could use recursion to keep getting the latest one, then when the list is 0 it can just stop the function.
See the example below.
public static List<string> MyList { get; set; }
public static object LockObject { get; set; }
static void Main(string[] args)
{
Console.Clear();
Program.LockObject = new object();
// Create the list
Program.MyList = new List<string>();
// Add 100 items to it
for (int i = 0; i < 100; i++)
{
Program.MyList.Add(string.Format("Item Number = {0}", i));
}
// Start Threads
for (int i = 0; i < 5; i++)
{
Thread thread = new Thread(new ThreadStart(Program.PopItemFromStackAndPrint));
thread.Name = string.Format("Thread # {0}", i);
thread.Start();
}
}
public static void PopItemFromStackAndPrint()
{
if (Program.MyList.Count == 0)
{
return;
}
string item = string.Empty;
lock (Program.LockObject)
{
// Get first Item
item = Program.MyList[0];
Program.MyList.RemoveAt(0);
}
Console.WriteLine("{0}:{1}", System.Threading.Thread.CurrentThread.Name, item);
// Sleep to show other processing for examples only
System.Threading.Thread.Sleep(10);
Program.PopItemFromStackAndPrint();
}