static List<int> sharedCollection = new List<int>();
static readonly Object obj = new Object();
static void Main(string[] args)`enter code here`
{
var writeThread = new Thread(() =>
{
for (int i = 0; i < 10; i++)
{
lock (obj)
{
Write();
}
}
});
var readThread = new Thread(() =>
{
for (int i = 0; i < 10; i++)
{
lock (obj)
{
Read();
}
}
});
writeThread.Start();
readThread.Start();
Console.ReadLine();
}
static void Read()
{
Console.Write("Current collection state: ");
sharedCollection.ForEach((e) => Console.Write($"{e} "));
Console.WriteLine();
}
static void Write()
{
Random generator = new Random();
var addedValue = generator.Next(1, 20);
sharedCollection.Add(addedValue);
Console.WriteLine($"Added value is: {addedValue}");
}
I spend a lot of time trying to understand why I receive this:
console result
Could someone explain to me what is wrong with this code?
Mutex works fine but I need to illustrate lock statement too...
I expect that after every adding in 1st thread I obtain a collection state from the 2nd thread. Like this:
Added value: 1
Collection state: 1
Added value: 15
Collection state: 1 15
Added value: 4
Collection state: 1 15 4
I understand you expeected those threasd to run somewhat in paralell, but instead they executed sequentially. You expectation is correct.
I do not think it has anything to do with lock, however. lock will only prevent a read and a write from happening at the same time, not produce this behavior. Try it without the lock to verify. (However due to things like the JiT Compiler, CPU cache invalidations and Optimisations, results may still differet if there is a lock, even if it has no direct effect).
My best bet is that the read thread is simply so slow, it does not finish once before the write is through all it's itteartions. Writing the UI is expensive, even on something as trivial as the console. Or even especially there. I do a lot of backups of userprofiles using robocopy. And if it hits a lot of very small files, just writing the Console becomes the actuall programm bottleneck, ever over disk access. And something out-bottlenecking disk acess is not something that happens often.
If you write the UI only once per user triggerd event, you will not notice the cost. But do it from any form of loop - especially one running in another thread - and you will start to notice it. I was particualry informed that a foreach is apparently half as slow at ittearting as a for loop.
I even made a example for this, albeit in a Windows Forms Environment:
using System;
using System.Windows.Forms;
namespace UIWriteOverhead
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
int[] getNumbers(int upperLimit)
{
int[] ReturnValue = new int[upperLimit];
for (int i = 0; i < ReturnValue.Length; i++)
ReturnValue[i] = i;
return ReturnValue;
}
void printWithBuffer(int[] Values)
{
textBox1.Text = "";
string buffer = "";
foreach (int Number in Values)
buffer += Number.ToString() + Environment.NewLine;
textBox1.Text = buffer;
}
void printDirectly(int[] Values){
textBox1.Text = "";
foreach (int Number in Values)
textBox1.Text += Number.ToString() + Environment.NewLine;
}
private void btnPrintBuffer_Click(object sender, EventArgs e)
{
MessageBox.Show("Generating Numbers");
int[] temp = getNumbers(10000);
MessageBox.Show("Printing with buffer");
printWithBuffer(temp);
MessageBox.Show("Printing done");
}
private void btnPrintDirect_Click(object sender, EventArgs e)
{
MessageBox.Show("Generating Numbers");
int[] temp = getNumbers(1000);
MessageBox.Show("Printing directly");
printDirectly(temp);
MessageBox.Show("Printing done");
}
}
}
But even this overhead is pretty unlikey to have a presistent result. At some time the read thread should get the lock first, blocking write. But still, there are too many variables to say for sure. You should propably try a simpler example, with more consistent (and a whole lot less) writework. What about writing "A" and "B" to the console, instead of complex stuff like this?
Related
I've been struggling with Parallel.For and local variable. I'm trying to update percentages in a console app from a parallel for. I add page in multiple document and I would like an update on the percentage for each document.
Here's my attempt so far :
I'm taking the number of the line with Console.CursorTop, and I want to pass it to a method who's gonna override the line.
Loop from Program.cs
Parallel.For(0, generationFile.nbOfFile, optionsParallel,
i =>
{
string fileName = $"{tmpDirectoryPath}/{i + 1}_{guid}.pdf";
AddPage(fileName, generationFile, i);
});
The AddPage method
private static void AddPage(string fileName, GenerationFile generationFile, int i)
{
var cursorPosition = Console.CursorTop;
//Ajout des pages
for (int j = 0; j < generationFile.SizeMaxFile; j++)
{
Page page = Page.Create(outDoc, size);
AddText(outDoc, page, font, 14, i, fileName, j, generationFile.SizeMaxFile);
for (int k = 0; k < 292; k++)
{
AddImage(outDoc, page, 30, 30);
}
outDoc.Pages.Add(page);
ConsoleManager.UpdateConsole(i, j, cursorPosition, generationFile);
}
}
The UpdateConsole method
public static void UpdateConsole(int fileNumber, double progression, int cursorPosition, GenerationFile generationFile)
{
progression = (progression / 100) * generationFile.ArchiveESC;
Console.ForegroundColor = ConsoleColor.White;
Console.SetCursorPosition(0, cursorPosition);
Console.WriteLine($"\rFichier n°{fileNumber + 1}/{generationFile.SizeMaxFile} en cours de création : {progression}% ", Console.ForegroundColor);
}
I think everything works fine, except for the cursorPosition who take one value at the beginning and never change, so the same line is updated. I understand that there is something to do with local and/or shared variable, but I'm fairly new in parallel processing so even with the other threads on this topic and the MSDN, I don't understand what to do.
The way I prefer to handle progress reporting is to have all worker threads report to a shared progress field, and have a separate timer that reads this fields and reports the progress to the user. This lets me control how often progress is reported, regardless of how fast items are processed. I also want an abstraction layer that allows different ways to report progress. After all, the same method might be used from console, the UI, or not at all.
For example something like this:
public interface IMyProgress
{
void Increment(int incrementValue);
}
public sealed class MyProgress : IMyProgress, IDisposable
{
private readonly int totalItems;
private readonly Timer myTimer;
private volatile int progress;
private int lastUpdatedProgress;
public MyProgress(TimeSpan progressFrequency, int totalItems)
{
this.totalItems = totalItems;
myTimer = new Timer(progressFrequency.TotalMilliseconds);
myTimer.Elapsed += OnElapsed;
myTimer.Start();
}
private void OnElapsed(object sender, ElapsedEventArgs e)
{
var currentProgress = progress;
if (lastUpdatedProgress != currentProgress)
{
lastUpdatedProgress = currentProgress;
var percent = currentProgress * 100 / (double)totalItems;
Console.Write($"\rWork progress: {percent}%");
}
}
public void Increment(int incrementValue) => Interlocked.Add(ref progress, incrementValue);
public void Dispose() => myTimer?.Dispose();
}
This can be called from a parallel method like:
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
var myWorkItems = Enumerable.Range(1, 10000).ToList();
using var progress = new MyProgress(TimeSpan.FromSeconds(1), myWorkItems.Count);
DoProcessing(myWorkItems, progress);
}
private static void DoProcessing(IEnumerable<int> items, IMyProgress progress)
{
Parallel.ForEach(items, item =>
{
Thread.Sleep(20);
progress.Increment(1);
});
}
I would be a bit careful when using carriage returns. In my experience console applications tend to be used by other programs as much as by humans, and then it is likely that the output will be redirected to a file, and that does not support carriage returns. So I would try to make the output look good.
I would avoid trying to move the cursor around. I have tried that, but the result was unsatisfactory, YMMV.
I want to create an array of BackgroundWorkers which work alongside each other to take advantage of multicore processors. Let's say I want eight of them working simultaneously. I also want a separate Timer thread to report in the GUI on some data which each of the threads has been processing.
My problem is this. It takes around 3-4 seconds for each worker to start work. So the first worker element in the array starts straight away. The second starts a few seconds after. The third starts a few seconds after that etc. etc.
I want them all to start straight away. How can I fix this?
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
BackgroundWorker[] bw;
int threads = 8;
data[] d;
private void Form1_Load(object sender, EventArgs e)
{
d = new data[threads];
for (int i = 0; i < threads; i++) d[i] = new data(i);
bw = new BackgroundWorker[threads];
for (int i = 0; i < threads; i++)
{
bw[i] = new BackgroundWorker();
bw[i].DoWork += new DoWorkEventHandler(Work);
}
timer1.Enabled = true;
for (int i = 0; i < threads; i++) bw[i].RunWorkerAsync(d[i]);
}
private void Work(object sender, DoWorkEventArgs e)
{
data o = (data)e.Argument;
while (true) o.count += o.id;
}
private void timer1_Tick(object sender, EventArgs e)
{
StringBuilder sb = new StringBuilder();
long total = 0;
for (int n = 0; n < threads; n++) {
sb.Append(d[n].id + ": " + d[n].count + "\n");
total+=d[n].count;
}
sb.Append("TOTAL: "+total);
richTextBox1.Text = sb.ToString();
}
}
public class data {
public int id;
public long count;
public data(int id)
{
this.id = id;
this.count = 0;
}
}
-------------------------------EDIT: Found out later than the 3-4 second delay only applies beyond the maximum number of logical cores you have. So if you have 20 cores, and set 20 threads that's fine. However, if you have 4 cores, and set 20 threads, there will a delay between creating each of the 16 threads beyond the first 4.
I want them all to start straight away.
Then Backgroundworkers are not the right tool. And maybe even Windows is not the right OS.
Backgroundworkers run on the ThreadPool, the guideline is to use that only for short (< 500ms) tasks. The Threadpool creates new threads at 2/second (roughly).
The short solution is to increase ThreadPool.Minthreads (look up the actual name/method) but that's still a whacky solution.
You did not provide enough (real) information for a better advice.
I suppose, that the line while (true) o.count += o.id; consumes so much CPU that other operations may be blocked.
Try to add Thread.Sleep(100); before o.count += o.id; or reduce the number of threads (8 threads + GUI thread => 9 Threads just for your application).
I am building a small application simulating a horse race in order to gain some basic skill in working with threads.
My code contains this loop:
for (int i = 0; i < numberOfHorses; i++)
{
horsesThreads[i] = new Thread(horsesTypes[i].Race);
horsesThreads[i].Start(100);
}
In order to keep the race 'fair', I've been looking for a way to make all newly created threads wait until the rest of the new threads are set, and only then launch all of them to start running their methods (Please note that I understand that technically the threads can't be launched at the 'same time')
So basically, I am looking for something like this:
for (int i = 0; i < numberOfHorses; i++)
{
horsesThreads[i] = new Thread(horsesTypes[i].Race);
}
Monitor.LaunchThreads(horsesThreads);
Threading does not promise fairness or deterministic results, so it's not a good way to simulate a race.
Having said that, there are some sync objects that might do what you ask. I think the Barrier class (Fx 4+) is what you want.
The Barrier class is designed to support this.
Here's an example:
using System;
using System.Threading;
namespace Demo
{
class Program
{
private void run()
{
int numberOfHorses = 12;
// Use a barrier with a participant count that is one more than the
// the number of threads. The extra one is for the main thread,
// which is used to signal the start of the race.
using (Barrier barrier = new Barrier(numberOfHorses + 1))
{
var horsesThreads = new Thread[numberOfHorses];
for (int i = 0; i < numberOfHorses; i++)
{
int horseNumber = i;
horsesThreads[i] = new Thread(() => runRace(horseNumber, barrier));
horsesThreads[i].Start();
}
Console.WriteLine("Press <RETURN> to start the race!");
Console.ReadLine();
// Signals the start of the race. None of the threads that called
// SignalAndWait() will return from the call until *all* the
// participants have signalled the barrier.
barrier.SignalAndWait();
Console.WriteLine("Race started!");
Console.ReadLine();
}
}
private static void runRace(int horseNumber, Barrier barrier)
{
Console.WriteLine("Horse " + horseNumber + " is waiting to start.");
barrier.SignalAndWait();
Console.WriteLine("Horse " + horseNumber + " has started.");
}
private static void Main()
{
new Program().run();
}
}
}
[EDIT] I just noticed that Henk already mentioned Barrier, but I'll leave this answer here because it has some sample code.
I'd be looking at a ManualResetEvent as a gate; inside the Thread, decrement a counter; if it is still non-zero, wait on the gate; otherwise, open the gate. Basically:
using System;
using System.Threading;
class Program
{
static void Main()
{
ManualResetEvent gate = new ManualResetEvent(false);
int numberOfThreads = 10, pending = numberOfThreads;
Thread[] threads = new Thread[numberOfThreads];
ParameterizedThreadStart work = name =>
{
Console.WriteLine("{0} approaches the tape", name);
if (Interlocked.Decrement(ref pending) == 0)
{
Console.WriteLine("And they're off!");
gate.Set();
}
else gate.WaitOne();
Race();
Console.WriteLine("{0} crosses the line", name);
};
for (int i = 0; i < numberOfThreads; i++)
{
threads[i] = new Thread(work);
threads[i].Start(i);
}
for (int i = 0; i < numberOfThreads; i++)
{
threads[i].Join();
}
Console.WriteLine("all done");
}
static readonly Random rand = new Random();
static void Race()
{
int time;
lock (rand)
{
time = rand.Next(500,1000);
}
Thread.Sleep(time);
}
}
The following nunit test compares performance between running a single thread versus running 2 threads on a dual core machine. Specifically, this is a VMWare dual core virtual Windows 7 machine running on a quad core Linux SLED host with is a Dell Inspiron 503.
Each thread simply loops and increments 2 counters, addCounter and readCounter. This test was original testing a Queue implementation which was discovered to perform worse on a multi-core machine. So in narrowing down the problem to the small reproducible code, you have here no queue only incrementing variables and to shock and dismay, it's far slower with 2 threads then one.
When running the first test, the Task Manager shows 1 of the cores 100% busy with the other core almost idle. Here's the test output for the single thread test:
readCounter 360687000
readCounter2 0
total readCounter 360687000
addCounter 360687000
addCounter2 0
You see over 360 Million increments!
Next the dual thread test shows 100% busy on both cores for the whole 5 seconds duration of the test. However it's output shows only:
readCounter 88687000
readCounter2 134606500
totoal readCounter 223293500
addCounter 88687000
addCounter2 67303250
addFailure0
That's only 223 Million read increments. What is god's creation are those 2 CPU's doing for those 5 seconds to get less work done?
Any possible clue? And can you run the tests on your machine to see if you get different results? One idea is that perhaps the VMWare dual core performance isn't what you would hope.
using System;
using System.Threading;
using NUnit.Framework;
namespace TickZoom.Utilities.TickZoom.Utilities
{
[TestFixture]
public class ActiveMultiQueueTest
{
private volatile bool stopThread = false;
private Exception threadException;
private long addCounter;
private long readCounter;
private long addCounter2;
private long readCounter2;
private long addFailureCounter;
[SetUp]
public void Setup()
{
stopThread = false;
addCounter = 0;
readCounter = 0;
addCounter2 = 0;
readCounter2 = 0;
}
[Test]
public void TestSingleCoreSpeed()
{
var speedThread = new Thread(SpeedTestLoop);
speedThread.Name = "1st Core Speed Test";
speedThread.Start();
Thread.Sleep(5000);
stopThread = true;
speedThread.Join();
if (threadException != null)
{
throw new Exception("Thread failed: ", threadException);
}
Console.Out.WriteLine("readCounter " + readCounter);
Console.Out.WriteLine("readCounter2 " + readCounter2);
Console.Out.WriteLine("total readCounter " + (readCounter + readCounter2));
Console.Out.WriteLine("addCounter " + addCounter);
Console.Out.WriteLine("addCounter2 " + addCounter2);
}
[Test]
public void TestDualCoreSpeed()
{
var speedThread1 = new Thread(SpeedTestLoop);
speedThread1.Name = "Speed Test 1";
var speedThread2 = new Thread(SpeedTestLoop2);
speedThread2.Name = "Speed Test 2";
speedThread1.Start();
speedThread2.Start();
Thread.Sleep(5000);
stopThread = true;
speedThread1.Join();
speedThread2.Join();
if (threadException != null)
{
throw new Exception("Thread failed: ", threadException);
}
Console.Out.WriteLine("readCounter " + readCounter);
Console.Out.WriteLine("readCounter2 " + readCounter2);
Console.Out.WriteLine("totoal readCounter " + (readCounter + readCounter2));
Console.Out.WriteLine("addCounter " + addCounter);
Console.Out.WriteLine("addCounter2 " + addCounter2);
Console.Out.WriteLine("addFailure" + addFailureCounter);
}
private void SpeedTestLoop()
{
try
{
while (!stopThread)
{
for (var i = 0; i < 500; i++)
{
++addCounter;
}
for (var i = 0; i < 500; i++)
{
readCounter++;
}
}
}
catch (Exception ex)
{
threadException = ex;
}
}
private void SpeedTestLoop2()
{
try
{
while (!stopThread)
{
for (var i = 0; i < 500; i++)
{
++addCounter2;
i++;
}
for (var i = 0; i < 500; i++)
{
readCounter2++;
}
}
}
catch (Exception ex)
{
threadException = ex;
}
}
}
}
Edit: I tested the above on a quad core laptop w/o vmware and got similar degraded performance. So I wrote another test similar to the above but which has each thread method in a separate class. My purpose in doing that was to test 4 cores.
Well that test showed excelled results which improved almost linearly with 1, 2, 3, or 4 cores.
With some experimentation now on both machines it appears that the proper performance only happens if main thread methods are on different instances instead of the same instance.
In other words, if multiple threads main entry method on on the same instance of a particular class, then the performance on a multi-core will be worse for each thread you add, instead of better as you might assume.
It almost appears that the CLR is "synchronizing" so only one thread at a time can run on that method. However, my testing says that isn't the case. So it's still unclear what's happening.
But my own problem seems to be solved simply by making separate instances of methods to run threads as their starting point.
Sincerely,
Wayne
EDIT:
Here's an updated unit test that tests 1, 2, 3, & 4 threads with them all on the same instance of a class. Using arrays with variables uses in the thread loop at least 10 elements apart. And performance still degrades significantly for each thread added.
using System;
using System.Threading;
using NUnit.Framework;
namespace TickZoom.Utilities.TickZoom.Utilities
{
[TestFixture]
public class MultiCoreSameClassTest
{
private ThreadTester threadTester;
public class ThreadTester
{
private Thread[] speedThread = new Thread[400];
private long[] addCounter = new long[400];
private long[] readCounter = new long[400];
private bool[] stopThread = new bool[400];
internal Exception threadException;
private int count;
public ThreadTester(int count)
{
for( var i=0; i<speedThread.Length; i+=10)
{
speedThread[i] = new Thread(SpeedTestLoop);
}
this.count = count;
}
public void Run()
{
for (var i = 0; i < count*10; i+=10)
{
speedThread[i].Start(i);
}
}
public void Stop()
{
for (var i = 0; i < stopThread.Length; i+=10 )
{
stopThread[i] = true;
}
for (var i = 0; i < count * 10; i += 10)
{
speedThread[i].Join();
}
if (threadException != null)
{
throw new Exception("Thread failed: ", threadException);
}
}
public void Output()
{
var readSum = 0L;
var addSum = 0L;
for (var i = 0; i < count; i++)
{
readSum += readCounter[i];
addSum += addCounter[i];
}
Console.Out.WriteLine("Thread readCounter " + readSum + ", addCounter " + addSum);
}
private void SpeedTestLoop(object indexarg)
{
var index = (int) indexarg;
try
{
while (!stopThread[index*10])
{
for (var i = 0; i < 500; i++)
{
++addCounter[index*10];
}
for (var i = 0; i < 500; i++)
{
++readCounter[index*10];
}
}
}
catch (Exception ex)
{
threadException = ex;
}
}
}
[SetUp]
public void Setup()
{
}
[Test]
public void SingleCoreTest()
{
TestCores(1);
}
[Test]
public void DualCoreTest()
{
TestCores(2);
}
[Test]
public void TriCoreTest()
{
TestCores(3);
}
[Test]
public void QuadCoreTest()
{
TestCores(4);
}
public void TestCores(int numCores)
{
threadTester = new ThreadTester(numCores);
threadTester.Run();
Thread.Sleep(5000);
threadTester.Stop();
threadTester.Output();
}
}
}
That's only 223 Million read increments. What is god's creation are those 2 CPU's doing for those 5 seconds to get less work done?
You're probably running into cache contention -- when a single CPU is incrementing your integer, it can do so in its own L1 cache, but as soon as two CPUs start "fighting" over the same value, the cache line it's on has to be copied back and forth between their caches each time each one accesses it. The extra time spent copying data between caches adds up fast, especially when the operation you're doing (incrementing an integer) is so trivial.
A few things:
You should probably test each setup at least 10 times and take the average
As far as I know, Thread.sleep is not exact - it depends on how the OS switches your threads
Thread.join is not immediate. Again, it depends on how the OS switches your threads
A better way to test would be to run a computationally intensive operation (say, sum from one to a million) on two configurations and time them. For example:
Time how long it takes to sum from one to a million
Time how long it takes to sum one to 500000 on one thread and one 500001 to 1000000 on another
You were right when you thought that two threads would work faster than one thread. But yours are not the only threads running - the OS has threads, your browser has threads, and so on. Keep in mind that your timings will not be exact and may even fluctuate.
Lastly, there are other reasons(see slide 24) why threads work slower.
I have a problem with multithreading in .net. With the following code:
class Program
{
private static ManualResetEvent[] resetEvents;
private void cs(object o)
{
int xx = 0;
while (true)
{
xx++;
System.Xml.XmlDocument document = new System.Xml.XmlDocument();
document.Load("ConsoleApplication6.exe.config");
MSScriptControl.ScriptControlClass s =
newMSScriptControl.ScriptControlClass();
s.Language = "JScript";
object res = s.Eval("1+2");
Console.WriteLine("thread {0} execution {1}" , o , xx);
System.Threading.Thread.Sleep(5000);
}
}
static void Main(string[] args)
{
Program c = new Program();
for (int i = 0; i < 1000; i++)
{
System.Threading.Thread t = new System.Threading.Thread(
new System.Threading.ParameterizedThreadStart(c.cs));
t.Start((object)i);
}
}
}
When this code executed, it crashes after some minutes. Why is it crashing? What can I do to prevent the crashes?
You're starting 1000 threads. That is 1000 MB in stack space alone plus all the objects the threads create. My guess is that you're running out of memory. What are you trying to accomplish?
It rarely makes sense to create more threads than processor cores b/c the app will spend alot of time context switching and you will not see much if any performance gain. The only number of threads that will run at the same time equals the number of cores your machine has. Why does this have to be done with 1000 threads? (Obviously I am assuming you don't have a 1000 core machine)
There are probably some aspects of this app that aren't apparent given the provided code. That being said, I haven't been able to reproduce the crash but I am able to run up an obscene amount of memory usage (160MB at 29 "xx++"s).
I suspect that the ScriptControlClass is staying alive in memory. While there may be better ways of solving this problem, I was able to keep the memory at a consistent 60-70MB with this addition:
Console.WriteLine("thread {0} execution {1}" , o , xx);
s = null;
UPDATE
I had similar results with this code memory-wise but the code below was slightly faster. After several minutes the memory usage was a consistent 51MB when xx < 30 was changed back to true.
class Program
{
private static ManualResetEvent[] resetEvents;
[ThreadStatic]
static ScriptControlClass s;
private void cs(object o)
{
int xx = 0;
if (s == null)
{
s = new ScriptControlClass();
}
/* you should be able to see consistent memory usage after 30 iterations */
while (xx < 30)
{
xx++;
//Unsure why this is here but doesn't seem to affect the memory usage
// like the ScriptControlClass object.
System.Xml.XmlDocument document = new System.Xml.XmlDocument();
document.Load("ConsoleApplication6.exe.config");
s.Language = "JScript";
object res = s.Eval("1+2");
Console.WriteLine("thread {0} execution {1}", o, xx);
System.Threading.Thread.Sleep(5000);
}
s = null;
}
static void Main(string[] args)
{
Program c = new Program();
for (int i = 0; i < 1000; i++)
{
System.Threading.Thread t = new System.Threading.Thread(
new System.Threading.ParameterizedThreadStart(c.cs));
t.Start((object)i);
}
}
}
UPDATE 2
My Googling can't find anything about using the ScriptControlClass in a multithreaded environment.