I having problem managed thread parallel in console application.
I am running 10 threads parallel & all thread doing some specific task.
In case if any task is over/completed then doing stop/end thread and immediate I started new thread instance. I want 10 threads so anyone thread is going to stop/end then It generates new thread. but every time I want 10 threads in running mode in console application & It should be parallel work using C# console application.
How I can running 10 threads in C# console application?
At the end of each thread put a lock on some shared object (lock (obj) {}).
Then remove the current thread from a collection of threads you have.
If the collection.Count is less than 10 create a new one and put inside the collection.
Release the lock.
private List<Thread> threads = new List<Thread>();
private void ThreadFunction() {
// do something
// here before the lock
lock (threads) {
threads.Remove(Thread.CurrentThread);
if (thread.Count < 10) {
Thread t = new Thread(ThreadFunction);
threads.Add(t);
t.Start();
}
}
}
Be sure to catch all exception inside the thread or you code will fail when a thread exception happens. That is make sure that the lock part of the code is always called (except on a Thread abord exception but that will not matter).
But as stated I think you should use a ThreadPool for such a task...
The book on threads in .Net is: http://www.albahari.com/threading/
This alone will probably answer any questions you have.
Depending on what you are using these threads for (I am guessing that you may be talking about running transactions in the background) you may want to use BackgroundWorker.
http://msdn.microsoft.com/en-us/library/system.componentmodel.backgroundworker.aspx
BackgroundWorker lets you deal with Begin/End/Progress Events only, making debugging much less error prone.
Related
This question already has answers here:
When does a multithreaded console application exit?
(2 answers)
Closed last year.
Sorry if my question seems naive. I'm new to C# and .Net, and still wrap my head around them.
I come from a Go-Lang background, and try to learn C#/.Net multithreading.
In Go the main thread will run and complete its logic regardless of other threads if no wait is used.
I thought it should be the same in C#, however, the code below allow all threads to run completely.
Which means Main thread waits other threads to complete, without using join() or any other wait techniques.
Could you please, let me know what I missed here or misunderstood.
namespace TestThread
{
internal class Program
{
static void Main(string[] args)
{
Thread T1 = new Thread(PrintY);
T1.Start();
// The following is the funtion of the Main thread.
for (int i = 0; i < 10; i++) Console.Write("x");
}
static void PrintY()
{
for (int i = 0; i < 100; i++)
{
Console.Write("Y");
Thread.Sleep(100);
}
}
}
}
The output is like the following:
xxxxxxxxxYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYYY
I expected at most one Y in the results before the Main method finishes and therefore the process terminates. What is keeping the process alive when the main thread is completed?
When you are creating Thread via constructor by default it's Thread.IsBackground is set to false:
A thread is either a background thread or a foreground thread. Background threads are identical to foreground threads, except that background threads do not prevent a process from terminating. Once all foreground threads belonging to a process have terminated, the common language runtime ends the process. Any remaining background threads are stopped and do not complete.
And:
By default, the following threads execute in the foreground (that is, their IsBackground property returns false):
The primary thread (or main application thread).
All threads created by calling a Thread class constructor.
So it will prevent the app from terminating. If you want your threads to allow the application to terminate you can set this property to false:
Thread T1 = new Thread(PrintY);
T1.IsBackground = true;
...
Also note that manually creating threads is rarely needed operation in modern C#. After introduction of Task Parallel Library (TPL) usually Task APIs are used instead of directly manipulating the threads (_ = Task.Run(PrintY); for example can be used in this case).
I have a little c# app with multiple threads runing, but my main thread has to wait for all of threads to finish then it can do the rest.
problem now is that im using .join() for each thread, this seems wait for each thread to finish then it goes to next thread, which makes app not really multi-threading and take long time to finish.
so I wonder if there is any way I can get around this problem or just a way to check if there are no more threads is active.
thanks
If you're hanging on to the Thread object, you can use Thread.IsAlive.
Alternately, you might want to consider firing an event from your thread when it is done.
Thread.Join() doesn't mean your application isn't multithreaded - it tells the current thread to wait for the other thread to finish, which is exactly what you want.
Doing the following:
List<Thread> threads = new List<Thread>();
/** create each thread, Start() it, and add it to the list **/
foreach (Thread thread in threads)
{
thread.Join()
}
will continue to run the other threads, except the current/main thread (it will wait until the other threads are done).
Just use Thread.Join()
Ye, as said by Cuong Le, using Task Parallel Library would be much efficient.
However, you can Create a list of Threads and then check if they are alive or not.
var threadsList = new List<Thread>();
threadsList.Add(myThread); // to add
bool areDone = true;
foreach (Thread t in threadsList) {
if (t.IsAlive)
{
areDone = false;
break;
}
}
if (areDone)
{
// Everything is finished :O
}
Run multiple at same time but wanted to wait for all of them to finish, here's a way of doing the same with Parallel.ForEach:
var arrStr = new string[] {"1st", "2nd", "3rd"};
Parallel.ForEach<string>(arrStr, str =>
{
DoSomething(str); // your custom method you wanted to use
Debug.Print("Finished task for: " + str);
});
Debug.Print("All tasks finished");
That was the most simplest and efficient i guess it can go if in C# 4.0 if you want all tasks to run through same method
Try using BackgroundWorker
It raises an event in the main thread (RunWorkerCompleted) after its work is done
Here is one sample from previously answered question
https://stackoverflow.com/a/5551376/148697
I have the following code, could anyone please clarify my doubt below.
public static void Main() {
Thread thread = new Thread(Display);
thread.Start();
Thread.Sleep(5000);
// Throws exception, thread is terminated, cannot be restarted.
thread.Start()
}
public static void Display() {
}
It seems like in order to restart the thread I have to re-instantiate the thread again. Does this means I am creating a new thread? If I keep on creating 100 re-instiation will it create 100 threads and cause performance issue?
Yes, you either have to create a new thread or give the task to the thread pool each time to avoid a genuinely new thread being created. You can't restart a thread.
However, I'd suggest that if your task has failed to execute 100 times in a row, you have bigger problems than the performance overhead of starting new tasks.
You do not need to start the thread after sleep, the thread wake up automatically. It's the same thread.
first of all, you can't start the thread if it has already started. In your example, thread has finished it is work, that's why it is in terminated state.
you can check status using:
Thread.ThreadState
Are you trying to wake the thread up before the 5 seconds in complete? In which case you could try using Monitor (Wait, Pulse etc)
I would like to start x number of threads from my .NET application, and I would like to keep track of them as I will need to terminate them manually or when my application closes my application later on.
Example ==> Start Thread Alpha, Start Thread Beta .. then at any point in my application I should be able to say Terminate Thread Beta ..
What is the best way to keep track of opened threads in .NET and what do I need to know ( an id ? ) about a thread to terminate it ?
You could save yourself the donkey work and use this Smart Thread Pool. It provides a unit of work system which allows you to query each thread's status at any point, and terminate them.
If that is too much bother, then as mentioned anIDictionary<string,Thread> is probably the simplest solution. Or even simpler is give each of your thread a name, and use an IList<Thread>:
public class MyThreadPool
{
private IList<Thread> _threads;
private readonly int MAX_THREADS = 25;
public MyThreadPool()
{
_threads = new List<Thread>();
}
public void LaunchThreads()
{
for (int i = 0; i < MAX_THREADS;i++)
{
Thread thread = new Thread(ThreadEntry);
thread.IsBackground = true;
thread.Name = string.Format("MyThread{0}",i);
_threads.Add(thread);
thread.Start();
}
}
public void KillThread(int index)
{
string id = string.Format("MyThread{0}",index);
foreach (Thread thread in _threads)
{
if (thread.Name == id)
thread.Abort();
}
}
void ThreadEntry()
{
}
}
You can of course get a lot more involved and complicated with it. If killing your threads isn't time sensitive (for example if you don't need to kill a thread in 3 seconds in a UI) then a Thread.Join() is a better practice.
And if you haven't already read it, then Jon Skeet has this good discussion and solution for the "don't use abort" advice that is common on SO.
You can create a Dictionary of threads and assign them id's, like:
Dictionary<string, Thread> threads = new Dictionary<string, Thread>();
for(int i = 0 ;i < numOfThreads;i++)
{
Thread thread = new Thread(new ThreadStart(MethodToExe));
thread.Name = threadName; //Any name you want to assign
thread.Start(); //If you wish to start them straight away and call MethodToExe
threads.Add(id, thread);
}
If you don't want to save threads against an Id you can use a list and later on just enumerate it to kill threads.
And when you wish to terminate them, you can abort them. Better have some condition in your MethodToExe that allows that method to leave allowing the thread to terminate gracefully. Something like:
void MethodToExe()
{
while(_isRunning)
{
//you code here//
if(!_isRunning)
{
break;
}
//you code here//
}
}
To abort you can enumerate the dictionary and call Thread.Abort(). Be ready to catch ThreadAbortException
I asked a similar questions and received a bunch of good answers: Shutting down a multithreaded application
Note: my question did not require a graceful exit, but people still recommended that I gracefully exit from the loop of each thread.
The main thing to remember is that if you want to avoid having your threads prevent your process from terminating you should set all your threads to background:
Thread thread = new Thread(new ThreadStart(testObject.RunLoop));
thread.IsBackground = true;
thread.start();
The preferred way to start and manage threads is in a ThreadPool, but just about any container out there can be used to keep a reference to your threads. Your threads should always have a flag that will tell them to terminate and they should continually check it.
Furthermore, for better control you can supply your threads with a CountdownLatch: whenever a thread is exiting its loop it will signal on a CountdownLatch. Your main thread will call the CountdownLatch.Wait() method and it will block until all the threads have signaled... this allows you to properly cleanup and ensures that all your threads have shutdown before you start cleaning up.
public class CountdownLatch
{
private int m_remain;
private EventWaitHandle m_event;
public CountdownLatch(int count)
{
Reset(count);
}
public void Reset(int count)
{
if (count < 0)
throw new ArgumentOutOfRangeException();
m_remain = count;
m_event = new ManualResetEvent(false);
if (m_remain == 0)
{
m_event.Set();
}
}
public void Signal()
{
// The last thread to signal also sets the event.
if (Interlocked.Decrement(ref m_remain) == 0)
m_event.Set();
}
public void Wait()
{
m_event.WaitOne();
}
}
It's also worthy to mention that the Thread.Abort() method does some strange things:
When a thread calls Abort on itself,
the effect is similar to throwing an
exception; the ThreadAbortException
happens immediately, and the result is
predictable. However, if one thread
calls Abort on another thread, the
abort interrupts whatever code is
running. There is also a chance that a
static constructor could be aborted.
In rare cases, this might prevent
instances of that class from being
created in that application domain. In
the .NET Framework versions 1.0 and
1.1, there is a chance the thread could abort while a finally block is
running, in which case the finally
block is aborted.
The thread that calls Abort might
block if the thread that is being
aborted is in a protected region of
code, such as a catch block, finally
block, or constrained execution
region. If the thread that calls Abort
holds a lock that the aborted thread
requires, a deadlock can occur.
After creating your thread, you can set it's Name property. Assuming you store it in some collection you can access it conveniently via LINQ in order to retrieve (and abort) it:
var myThread = (select thread from threads where thread.Name equals "myThread").FirstOrDefault();
if(myThread != null)
myThread.Abort();
Wow, there are so many answers..
You can simply use an array to hold the threads, this will only work if the access to the array will be sequantial, but if you'll have another thread accessing this array, you will need to synchronize access
You can use the thread pool, but the thread pool is very limited and can only hold fixed amount of threads.
As mentioned above, you can create you own thread pool, which in .NET v4 becomes much easier with the introduction of safe collections.
you can manage them by holding a list of mutex object which will determine when those threads should finish, the threads will query the mutex each time they run before doing anything else, and if its set, terminate, you can manage the mutes from anywhere, and since mutex are by defenition thread-safe, its fairly easy..
i can think of another 10 ways, but those seems to work. let me know if they dont fit your needs.
Depends on how sophisticated you need it to be. You could implement your own type of ThreadPool with helper methods etc. However, I think its as simple as just maintaining a list/array and adding/removing the threads to/from the collection accordingly.
You could also use a Dictionary collection and use your own type of particular key to retrieve them i.e. Guids/strings.
As you start each thread, put it's ManagedThreadId into a Dictionary as the key and the thread instance as the value. Use a callback from each thread to return its ManagedThreadId, which you can use to remove the thread from the Dictionary when it terminates. You can also walk the Dictionary to abort threads if needed. Make the threads background threads so that they terminate if your app terminates unexpectedly.
You can use a separate callback to signal threads to continue or halt, which reflects a flag set by your UI, for a graceful exit. You should also trap the ThreadAbortException in your threads so that you can do any cleanup if you have to abort threads instead.
I post a lot here regarding multithreading, and the great stackoverflow community have helped me alot in understand multithreading.
All the examples I have seen online only deal with one thread.
My application is a scraper for an insurance company (family company ... all free of charge). Anyway, the user is able to select how many threads they want to run. So lets say for example the user wants the application to scrape 5 sites at one time, and then later in the day he choses 20 threads because his computer isn't doing anything else so it has the resources to spare.
Basically the application builds a list of say 1000 sites to scrape. A thread goes off and does that and updates the UI and builds the list.
When thats finished another thread is called to start the scraping. Depending on the number of threads the user has set to use it will create x number of threads.
Whats the best way to create these threads? Should I create 1000 threads in a list. And loop through them? If the user has set 5 threads to run, it will loop through 5 at a time.
I understand threading, but it's the application logic which is catching me out.
Any ideas or resources on the web that can help me out?
You could consider using a thread pool for that:
using System;
using System.Threading;
public class Example
{
public static void Main()
{
ThreadPool.SetMaxThreads(100, 10);
// Queue the task.
ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
Console.WriteLine("Main thread does some work, then sleeps.");
Thread.Sleep(1000);
Console.WriteLine("Main thread exits.");
}
// This thread procedure performs the task.
static void ThreadProc(Object stateInfo)
{
Console.WriteLine("Hello from the thread pool.");
}
}
This scraper, does it use a lot of CPU when its running?
If it does a lot of communication with these 1000 remote sites, downloading their pages, that may be taking more time than the actual analysis of the pages.
And how many CPU cores does your user have? If they have 2 (which is common these days) then beyond two simultaneous threads performing analysis, they aren't going to see any speed up.
So you probably need to "parallelize" the downloading of the pages. I doubt you need to do the same for the analysis of the pages.
Take a look into asynchronous IO, instead of explicit multi-threading. It lets you launch a bunch of downloads in parallel and then get called back when each one completes.
If you really just want the application, use something someone else already spent time developing and perfecting:
http://arachnode.net/
arachnode.net is a complete and comprehensive .NET web crawler for
downloading, indexing and storing
Internet content including e-mail
addresses, files, hyperlinks, images,
and Web pages.
Whether interested or involved in
screen scraping, data mining, text
mining, research or any other
application where a high-performance
crawling application is key to the
success of your endeavors,
arachnode.net provides the solution
you need for success.
If you also want to write one yourself because it's a fun thing to write (I wrote one not long ago, and yes, it is alot of fun ) then you can refer to this pdf provided by arachnode.net which really explains in detail the theory behind a good web crawler:
http://arachnode.net/media/Default.aspx?Sort=Downloads&PageIndex=1
Download the pdf entitled: "Crawling the Web" (second link from top). Scroll to Section 2.6 entitled: "2.6 Multi-threaded Crawlers". That's what I used to build my crawler, and I must say, I think it works quite well.
I think this example is basically what you need.
public class WebScraper
{
private readonly int totalThreads;
private readonly List<System.Threading.Thread> threads;
private readonly List<Exception> exceptions;
private readonly object locker = new object();
private volatile bool stop;
public WebScraper(int totalThreads)
{
this.totalThreads = totalThreads;
threads = new List<System.Threading.Thread>(totalThreads);
exceptions = new List<Exception>();
for (int i = 0; i < totalThreads; i++)
{
var thread = new System.Threading.Thread(Execute);
thread.IsBackground = true;
threads.Add(thread);
}
}
public void Start()
{
foreach (var thread in threads)
{
thread.Start();
}
}
public void Stop()
{
stop = true;
foreach (var thread in threads)
{
if (thread.IsAlive)
{
thread.Join();
}
}
}
private void Execute()
{
try
{
while (!stop)
{
// Scrap away!
}
}
catch (Exception ex)
{
lock (locker)
{
// You could have a thread checking this collection and
// reporting it as you see fit.
exceptions.Add(ex);
}
}
}
}
The basic logic is:
You have a single queue in which you put the URLs to scrape then you create your threads and use a queue object to which every thread has access. Let the threads start a loop:
lock the queue
check if there are items in the queue, if not, unlock queue and end thread
dequeue first item in the queue
unlock queue
process item
invoke an event that updates the UI (Remember to lock the UI Controller)
return to step 1
Just let the Threads do the "get stuff from the queue" part (pulling the jobs) instead of giving them the urls (pushing the jobs), that way you just say
YourThreadManager.StartThreads(numberOfThreadsTheUserWants);
and everything else happens automagically. See the other replies to find out how to create and manage the threads .
I solved a similar problem by creating a worker class that uses a callback to signal the main app that a worker is done. Then I create a queue of 1000 threads and then call a method that launches threads until the running thread limit is reached, keeping track of the active threads with a dictionary keyed by the thread's ManagedThreadId. As each thread completes, the callback removes its thread from the dictionary and calls the thread launcher.
If a connection is dropped or times out, the callback reinserts the thread back into the queue. Lock around the queue and the dictionary. I create threads vs using the thread pool because the overhead of creating a thread is insignificant compared to the connection time, and it allows me to have a lot more threads in flight. The callback also provides a convenient place with which to update the user interface, even allowing you to change the thread limit while it's running. I've had over 50 open connections at one time. Remember to increase your MacConnections property in your app.config (default is two).
I would use a queue and a condition variable and mutex, and start just the requested number of threads, for example, 5 or 20 (and not start 1,000).
Each thread blocks on the condition variable. When woken up, it dequeues the first item, unlocks the queue, works with the item, locks the queue and checks for more items. If the queue is empty, sleep on the condition variable. If not, unlock, work, repeat.
While the mutex is locked, it can also check if the user has requested the count of threads to be reduced. Just check if count > max_count, and if so, the thread terminates itself.
Any time you have more sites to queue, just lock the mutex and add them to the queue, then broadcast on the condition variable. Any threads that are not already working will wake up and take new work.
Any time the user increases the requested thread count, just start them up and they will lock the queue, check for work, and either sleep on the condition variable or get going.
Each thread will be continually pulling more work from the queue, or sleeping. You don't need more than 5 or 20.
Consider using the event-based asynchronous pattern (AsyncOperation and AsyncOperationManager Classes)
You might want to take a look at the ProcessQueue article on CodeProject.
Essentially, you'll want to create (and start) the number of threads that are appropriate, in your case that number comes from the user. Each of these threads should process a site, then find the next site needed to process. Even if you don't use the object itself (though it sounds like it would suit your purposes pretty well, though I'm obviously biased!) it should give you some good insight into how this sort of thing would be done.