I have a BlockingCollection of integers.
Below is the code I am trying to develop to remove the items concurrently from the BlockingCollection.
static void Produce()
{
for (int i = 0; i < 100000; i++)
{
bc3.Add(i);
}
Run();
}
static void Run()
{
for (int i = 0; i < 5; i++)
{
Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning);
}
}
static void Process()
{
var stopWatch = new Stopwatch();
stopWatch.Start();
while (bc3.Count!= 0)
{
bc3.Take();
}
stopWatch.Stop();
Console.WriteLine("Thread {0}",
Thread.CurrentThread.ManagedThreadId);
Console.WriteLine("Elapsed Time {0}", stopWatch.Elapsed.Seconds);
}
Is this the correct way remove items in a faster way by creating 5 tasks?
Stopwatch issue
Your measurement results are wrong because you use
stopWatch.Elapsed.Seconds
instead of
stopWatch.ElapsedMilliseconds
You only display the seconds, but ignore the minutes, hours, etc.
Concurrency issue
No, this is not the correct way of removing items from a BlockingCollection. The statement that does not work is
bc3.Count != 0
All 5 tasks may check this condition at the same time, finding out that there is 1 item left. They all 5 go to
bc3.Take();
One task is able to remove the item, the other 4 tasks are waiting.
One way to solve this is to add
bc3.CompleteAdding();
in Produce().
Garbage collection issue
Once the first task finishes, the Run() method completes and all tasks in the method go out of scope and are garbage collected. This may make you see only 1 instead of 5 completion messages.
To fix this, use
static void Run()
{
var tasks = new List<Task>();
for (int i = 0; i < 5; i++)
{
tasks.Add(Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning));
}
try
{
Task.WaitAll(tasks.ToArray());
}
catch (AggregateException)
{ }
}
Cost of synchronization
Here's one of my outputs with 5 tasks (and 100000000 items):
Thread 11
Thread 13
Thread 12
Thread 14
Thread 15
Elapsed Time 12878
Elapsed Time 13122
Elapsed Time 13128
Elapsed Time 13128
Elapsed Time 13128
Run: 13140
Compare this to a single task:
Thread 12
Elapsed Time 10147
Run: 10149
This is because only one Take() method can remove an item and it takes additional time for synchronization.
Related
Why without Thread.Sleep threads will work asynchronously (1), and with it - synchronously (2)?
class A
{
static object locker = new object();
static void M0()
{
for(int i = 0; i < 5; i++)
lock (locker)
{
Console.WriteLine("Secondary");
//Thread.Sleep(100);
}
}
static void Main()
{
ThreadStart ts = new ThreadStart(M0);
Thread t = new Thread(ts);
t.Start();
for (int i = 0; i < 5; i++)
lock (locker)
{
Console.WriteLine("Primary");
//Thread.Sleep(100);
}
}
}
Both ways are working asynchronously, after all, you've got 2 threads.
When you start the thread and don't have the sleep it executes so quickly that it manages to output 5 Secondary lines within its allocated timeslice.
The Sleep call slows everything down, and because you are sleeping for a period that is reasonably large (compared to a threads timeslice) it gives the appearance of the threads running in step.
However, this is just luck. There's nothing to stop the thread scheduler deciding to suspend M0 just before it enters the lock for a period of time that allows the main thread to fully execute its loop.
I have to loop through code for specified time .I achieved it with DateTime
var time=DateTime.Now.AddMinutes((Convert.ToDouble(1)));
while(DateTime.Compare(DateTime.Now, time) <= 0)
{
console.write("some message..")
}
How do i achieve the same with Timer.Timer or thread.timer which is best approach..
Is it possible to write 10 times per sec?
Can anyone suggest. thank you
You could always use StopWatch, which is accurate and most appropriate for your scenario.
Action<long> action = (milliseconds) =>
{
Console.WriteLine("Running for {0}ms", milliseconds);
Stopwatch watch = new Stopwatch();
watch.Start();
while (watch.Elapsed.TotalMilliseconds <= milliseconds)
{
Console.WriteLine("ticks:{0}", DateTime.Now.Ticks);
Thread.Sleep(100);
}
Console.WriteLine("Done");
watch.Stop();
};
Task.Run(() => action(1000));
if you are going to make this work you need to make your program multithreaded,
See System.Threading and System.Threading.Task
Once you have your code executing in it's own thread, (using Thread, Task, Timer or any of the other variations in those namespaces) you can tell it to stop executing for a set amount of time, this is done by calling the Thread.Sleep or Task.Delay methods.
e.g.
Task.Run(()=>
{
do
{
//do something
await Task.Delay(100);
}
while(! exitCondition)
});
however you shouldn't count on this for exact timing as what you are doing is saying to the OS that this thread doesn't need to be executed for that amount of time, it doesn't mean the OS will pass it to the processor immediately on the time running out. depending on how busy the CPU is there can be quite a delay before your thread reaches the top of the waiting to process queue. if the timing is vitally important then i would set a lower time and check the clock before running
static void Main(string[] args)
{
System.Threading.Timer timer = null;
int counts = 0;
timer = new Timer((obj) =>
{
Console.WriteLine(counts);
if (++counts > 10)
timer.Dispose();
}, null, 100, 100);
for (;;) ;
}
will call the method dosomething() after 100ms, every 100ms
in the background, till timer.Dispose() is called;
this implementation will ofc never terminate as it is written here ;)
This question is in two parts.
I have a console application that gets info from several servers and saves this info to a DB. In order to enabel that this executes simultaniously i have used threads. I am trying to make this execution automatic every minute.
Searching stackoverflow i found that this could work:
var timer = new System.Threading.Timer((e) =>
{
var models = ServerHandler.GetServerModels();
foreach (var m in models)
{
ServerHandler.MakeThreads(m);
}
Console.WriteLine("Running...");
Console.WriteLine("Press 'X' to exit or close the window, i : " + i);
i++;
}, null, 0, TimeSpan.FromMinutes(1).Seconds);
However this is not working as anticipated, it only executes once. If i change to for example this:
TimeSpan.FromMinutes(0.5).Seconds
Or:
TimeSpan.FromSeconds(30).Seconds
Then it works.
What am I doing wrong?
Second part of this question:
When this actually works as I showed above something else happens.
The process runs continuously and after 474 threads it crashes and says that the system is out of memory.
I tried using thread sleep for this but when i do that it stops executing after it has runed once.
Including this if it might help:
public static void MakeThreads(ServerModel model)
{
Thread thread = new Thread(() => SaveServerInfo(model));
thread.Start();
//Thread.Sleep(1);
//thread.Join();
}
How can I make this work?
In your first problem using the .Seconds will only return the seconds value, but you are defining the minutes value as .5, so seconds will always be zero.
If you want to return the seconds you need to use TotalSeconds
TimeSpan.FromMinutes(0.5).TotalSeconds
and in the timespan you are using you are supposed to define the milliseconds. So you're getting a huge number of threads because its running every 30 millseconds instead of every 30000 milliseconds.
So use
TimeSpan.FromMinutes(0.5).TotalMilliseconds
or what i always find easier
(int)(1000 * 60 * 0.5) // Then you just replace the 0.5 with the number of seconds.
Basically a timer does exactly what it's supposed to do: run your code every 0.5 seconds. :) An in your case, that's a problem...
(Please check for syntax errors etc, I'm writing this in notepad)
Long solution
Your problem seems to be that you don't control your threads. Here's how I'd solve it: (This long solution shows how it more or less works)
while (true)
{
// we want to run it again in 0.5 seconds.
DateTime start = DateTime.UtcNow.AddSeconds(0.5);
Thread[] threads = new Thread[models.Count];
for (int i=0; i<models.Count; ++i)
{
threads[i] = new Thread((a) => SaveServerInfo((ServerModel)a));
threads[i].Start(models[i]);
}
for (int i=0; i<models.Count; ++i)
{
threads[i].Join();
}
DateTime current = DateTime.UtcNow;
if (current < start)
{
Thread.Sleep(start.Subtract(current));
}
}
Short solution
However, this might give you issues as well: you might spawn too many threads. This can be solved with a mechanism called thread pooling. As it turns out, there's a simple way to solve this:
static void DoStuff(string s)
{
// change to a value > 0.5 as well to ensure everything works
Thread.Sleep(TimeSpan.FromSeconds(0.1));
Console.WriteLine(s);
}
static void Handle(List<string> models)
{
while (true)
{
// we want to run it again in 0.5 seconds.
DateTime start = DateTime.UtcNow.AddSeconds(0.5);
Parallel.ForEach(models, (a) => DoStuff(a));
DateTime current = DateTime.UtcNow;
if (current < start)
{
Thread.Sleep(start.Subtract(current));
}
}
}
static void Main(string[] args)
{
List<string> models = new List<string>();
for (int i = 0; i < 10; ++i)
{
models.Add(i.ToString());
}
Handle(models);
}
I read a lot of answers on this point but I did not found the solution yet.
I have a class with a counter attribute having a problem with cached values. Even volatile don't seems to work:
public class MyClass {
private Timer _timer;
private int _threadsCounter = 0;
public StreamWriter Tracer { get; set; }
public MyClass() {
_timer = new Timer(1000.0 * 10);
_timer.AutoReset = true;
_timer.Elapsed += new ElapsedEventHandler(OnTimer);
_timer.Start();
}
private void OnTimer(object sender, ElapsedEventArgs e) {
HashSet<Task> taskPool = new HashSet<Task>();
try {
if (Tracer != null) Tracer.WriteLine("[{0}] onTimer start. Current threads counter is {1}.", DateTime.Now, _threadsCounter);
if (_threadsCounter >= 10) return;
// create parallel tasks
for (int i = 0; i < 8; i++) {
// limit on the max num of parallel processing but the counter remains unchanged during this timer event!!!
if (_threadsCounter >= 10) break;
var timeout = (30 + i * 2);
var task = Task.Run(() => {
var localCounter = System.Threading.Interlocked.Increment(ref _threadsCounter);
try {
System.Threading.Thread.Sleep(timeout * 1000);
}
finally {
System.Threading.Interlocked.Decrement(ref _threadsCounter);
}
});
taskPool.Add(task);
}
}
finally {
if (Tracer != null)
Tracer.WriteLine("[{0}] onTimer end. Created {1} tasks. Current threads counter is {2}.", DateTime.Now, taskPool.Count, _threadsCounter);
}
}
Well, it seems that the onTimer caches the _threadsCounter variable as the output is:
[14:10:47] onTimer start. Current threads counter is 0.
[14:10:47] onTimer end. Created 8 tasks. Current threads counter is 0.
[14:10:57] onTimer start. Current threads counter is 8.
[14:10:57] onTimer end. Created 8 tasks. Current threads counter is 8.
[14:11:07] onTimer start. Current threads counter is 16.
[14:11:07] onTimer end. Created 0 tasks. Current threads counter is 16.
[14:11:17] onTimer start. Current threads counter is 15.
[14:11:17] onTimer end. Created 0 tasks. Current threads counter is 15.
[14:11:37] onTimer start. Current threads counter is 4.
[14:11:37] onTimer end. Created 8 tasks. Current threads counter is 4.
[14:11:47] onTimer start. Current threads counter is 8.
[14:11:47] onTimer end. Created 8 tasks. Current threads counter is 8.
[14:11:57] onTimer start. Current threads counter is 16.
[14:11:57] onTimer end. Created 0 tasks. Current threads counter is 16.
Why do I arrive to 16?
I solved the problem by changing a bit the code:
var localCounter = _threadsCounter;
...
if ((localCounter + taskPool.Count) >= 10) break;
But why this behavior?
Task.Run doesn't start the task immediately. It adds the task to the thread pool queue and returns.
In your case, the whole for loop is executed before the new tasks start running, so nothing changes _threadsCounter. That's why volatile doesn't help.
You're effectively testing for the number of tasks that have actually started and got as far as incrementing the counter. That takes a little while - so basically you're creating all 8 tasks and starting them, then they're incrementing the counter... by which time it's too late to notice that you've got more than 10.
The better solution would be to increment the counter before you start the task:
// Increment the counter in expectation of starting a task
var localCounter = Interlocked.Increment(ref _threadsCounter);
if (localCounter >= 10)
{
// Ah, we're not going to start a task after all, so undo
// the increment
Interlocked.Decrement(ref _threadsCounter);
break;
}
else
{
// Start a task, which will decrement the counter at the end.
// (You could add the "decrement" bit as a continuation, even...)
}
im trying to use few threads to make the following:
List<Thread> threads = new List<Thread>();
for (int i = 0; i < 5; i++)
threads.Add(new Thread(CheckTradeOneForOne));
foreach (Thread t in threads)
t.Start();
progressBarTrades.Value = 0;
count = 0;
while count is defined at class-level,and progressbar is progressbar(winforms^_^).
private void CheckTradeOneForOne()
{
int current;
while (count < Trades.Count)
{
lock (locker)
{
current = count;
count++;
}
temppage = HelpClass.GetSourceCodeForTrade(Trades[current], sessid, profilenum);
//if the trainer has requested anything?
if (!HelpClass.RequestAnything(temppage))
{
}
lock (locker)
{
progressBarTrades.Value = (100 * count) / Trades.Count;
//buttonAction.Text = count.ToString();
}
}
}
Trades is List(length around 1000).
i want to pass of all the trades and make one httpwebrequest each,
therefore i want to use multithreading,
the problem is that although i use lock im getting error about the second lock that another thread is using buttonaction\progressbar.
one more thing, is this a proper way to use multithreading?
how much threads is ideal to use?
and do the lock make sure that no moe then one thread take the same trade string?
tyvm for your help:)
With windows form the best way to work with the progress bar and threads is BackgroundWorkers in MSDN is a good example.
You cannot access windows form or wpf controls from a thread that is not the form thread.
To do that you should use BeginInvoke or the class SynchronizationContext.
However since you are just updating a progressbar I would suggest you to use a simple timer, if you have a lot of threads, it will be faster.
Instead threads I would also suggest you to use ThreadPool, this will avoid the instantiation of too many threads.
Just as an example I post some code.
private volatile int TradesCount;
private List<...> Trades;
void MyFunction()
{
List<Thread> threads = new List<Thread>();
for (int i = 0; i < 5; i++)
threads.Add(new Thread(CheckTradeOneForOne));
timer1.Enabled = true;
progressBarTrades.Value = 0;
this.TradesCount = 0;
foreach (Thread t in threads)
t.Start();
}
void timer1_Tick(object sender, EventArgs e)
{
int count = this.TradesCount;
if (count >= Trades.Count)
{
count = Trades.Count;
timer1.Enabled = false;
}
progressBarTrades.Value = (100 * count) / Trades.Count;
buttonAction.Text = count.ToString();
}
private void CheckTradeOneForOne()
{
int current;
for (;;)
{
// This will give you a warning, but you can ignore it with a #pragma, it is allowed to use Interlocked.Increment and volatile fields.
current = Interlocked.Increment(ref TradesCount) - 1;
if (current >= Trades.Count)
break; // We can exit the loop.
temppage = HelpClass.GetSourceCodeForTrade(Trades[current], sessid, profilenum);
//if the trainer has requested anything?
if (!HelpClass.RequestAnything(temppage))
{
...
}
}
}
I'm using Interlocked.Increment instead of lock, it is faster.
Look on google about Interlocked.Increment, it is a very good function: atomic increment.
I agree with the other answers recommending the use of .NET thread-pool instead of starting new threads yourself. If you have the luxury of using .NET 4 or newer, I would go even further and recommend you use the Task Parallel Library (and even if you are not currently using .NET 4, building a task-oriented abstraction on top of .NET thread-pool is very simple).
One of the biggest advantages of using tasks IMHO is that they are the foundation of the coming C# 5 await built-in language feature. So if you use tasks for your background processing today, you are already preparing your program for the future :-).
To show how you can use tasks to solve your immediate problem, I wrote a simple WinForms program that illustrates the technique of doing background processing and updating a progress bar to track how many tasks have already finished:
// file "Program.cs"
using System;
using System.Diagnostics;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace ProgressDialog
{
static class Program
{
[STAThread] static void Main()
{
// build UI
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
var rootForm = new Form { Text = #"Test Form", Width = 300, Height = 100 };
var btn = new Button { Text = #"Start", Parent = rootForm, Dock = DockStyle.Top };
var progress = new ProgressBar { Minimum = 0, Parent = rootForm, Dock = DockStyle.Top, Style = ProgressBarStyle.Continuous };
new Label { Text = #"Progress:", Parent = rootForm, Dock = DockStyle.Top, AutoSize = true };
// define parameters
const int sourcesCount = 20; // how many sources do we want to process
var completedCount = 0;
var randomGenerator = new Random();
var timer = new Stopwatch();
// callback that will be invoked on UI thread each time a task finishes
Action<int> onTaskCompleted = source =>
{
++completedCount; // we're modifying "completedCount" closure on purpose
progress.Value = completedCount;
System.Diagnostics.Debugger.Log(0, null, string.Concat(#"UI notified that task for source ", source, #" finished; overall ", completedCount, #" tasks finished", Environment.NewLine));
if (completedCount == sourcesCount)
{
timer.Stop();
btn.Enabled = true;
btn.Text = string.Concat(#"Finished (took ", timer.ElapsedMilliseconds, #" milliseconds). Start again");
}
};
// task itself (the hard part :) )
Action<int> task = source =>
{
System.Diagnostics.Debugger.Log(0, null, string.Concat(#" > Starting task for source ", source, Environment.NewLine));
Thread.Sleep(randomGenerator.Next(100, 200)); // simulate some workload (taking between 100 and 200 milliseconds)
System.Diagnostics.Debugger.Log(0, null, string.Concat(#" < Finished task for source ", source, Environment.NewLine));
rootForm.BeginInvoke(new Action(() => onTaskCompleted(source)));
};
// start button handler (kick-starts the background tasks)
btn.Click += (src, args) =>
{
btn.Enabled = false;
btn.Text = #"Running...";
progress.Maximum = sourcesCount;
progress.Value = 0;
timer.Restart();
completedCount = 0;
var sources = Enumerable.Range(1, sourcesCount); // simulate getting data for each task
var tasks = sources
.Select(s => Task.Factory.StartNew(() => task(s))) // at this point we only have an enumerable that is able to start all the tasks, nothing is running yet
.ToArray(); // now the tasks are started
if (tasks.Length != sourcesCount) { throw new InvalidOperationException(); } // assert that we created one task for each source
};
// show the form now, let the user interact with it
Application.Run(rootForm);
}
}
}
You can compile the program by either creating a new (console or winforms) project in Visual Studio and copying the code to Program.cs or by using the csc.exe on a command line.
When tasks are running, progress bar tracks number of finished tasks:
When all the tasks are finished, start button displays overall time taken:
Note that the time taken by each task is random (between 100 and 200 milliseconds), and number of tasks running concurrently will depend on how many processors/cores you have available (Task Parallel Library does this automatically), so the time displayed will vary between runs.
Also note that diagnostic messages are sent to the "Output" view in Visual Studio when running the program in debug mode (or you can use the SysInternals DebugView to see them). One sample run on my (2-core) machine produced the following:
> Starting task for source 1
> Starting task for source 2
> Starting task for source 3
< Finished task for source 3
> Starting task for source 4
UI notified that task for source 3 finished; overall 1 tasks finished
< Finished task for source 2
> Starting task for source 5
UI notified that task for source 2 finished; overall 2 tasks finished
< Finished task for source 1
> Starting task for source 6
UI notified that task for source 1 finished; overall 3 tasks finished
< Finished task for source 4
> Starting task for source 7
UI notified that task for source 4 finished; overall 4 tasks finished
< Finished task for source 5
> Starting task for source 8
UI notified that task for source 5 finished; overall 5 tasks finished
< Finished task for source 6
> Starting task for source 9
UI notified that task for source 6 finished; overall 6 tasks finished
< Finished task for source 8
> Starting task for source 10
UI notified that task for source 8 finished; overall 7 tasks finished
< Finished task for source 7
> Starting task for source 11
UI notified that task for source 7 finished; overall 8 tasks finished
< Finished task for source 9
> Starting task for source 12
UI notified that task for source 9 finished; overall 9 tasks finished
< Finished task for source 10
< Finished task for source 11
> Starting task for source 13
UI notified that task for source 10 finished; overall 10 tasks finished
UI notified that task for source 11 finished; overall 11 tasks finished
> Starting task for source 14
< Finished task for source 14
> Starting task for source 15
UI notified that task for source 14 finished; overall 12 tasks finished
< Finished task for source 13
> Starting task for source 16
UI notified that task for source 13 finished; overall 13 tasks finished
< Finished task for source 12
> Starting task for source 17
UI notified that task for source 12 finished; overall 14 tasks finished
< Finished task for source 16
> Starting task for source 18
UI notified that task for source 16 finished; overall 15 tasks finished
< Finished task for source 15
UI notified that task for source 15 finished; overall 16 tasks finished
> Starting task for source 19
< Finished task for source 17
UI notified that task for source 17 finished; overall 17 tasks finished
< Finished task for source 18
> Starting task for source 20
UI notified that task for source 18 finished; overall 18 tasks finished
< Finished task for source 19
UI notified that task for source 19 finished; overall 19 tasks finished
< Finished task for source 20
UI notified that task for source 20 finished; overall 20 tasks finished
As already mentioned by some, then use the ThreadPool instead of creating the threads yourself. By using the ThreadPool you don't have to worry about the number of threads to spawn - the ThreadPool will do that for you.
Assuming you're using .NET 4, you could take advantage of TPL:
public partial class Form1 : Form
{
private volatile int count;
private readonly int total;
public Form1()
{
InitializeComponent();
var urls = new List<string> { "http://something.com", "http://another.com" };
total = urls.Count;
// Execute the Parallel loop in a thread from the threadpool,
// in order not to block the UI thread.
ThreadPool.QueueUserWorkItem(o =>
{
Parallel.ForEach(urls, x => MakeRequest(x));
});
// other UI stuff here?
}
public void MakeRequest(string url)
{
// code for web request here...
int newCount = Interlocked.Increment(ref count);
Invoke(new Action(() => progressBar.Value = (100 * newCount) / total));
}
}