I have a C# Windows Service that starts up various objects (Class libraries). Each of these objects has its own "processing" logic that start up multiple long running processing threads by using the ThreadPool. I have one example, just like this:
System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(WorkerThread_Processing));
This works great. My app works with no issues, and my threads work well.
Now, for regression testing, I am starting those same objects up, but from a C# Console app rather than a Windows Service. It calls the same exact code (because it is invoking the same objects), however the WorkerThread_Processing method delays for up to 20 seconds before starting.
I have gone in and switched from the ThreadPool to a Thread, and the issue goes away. What could be happening here? I know that I am not over the MaxThreads count (I am starting 20 threads max).
The ThreadPool is specifically not intended for long-running items (more specifically, you aren't even necessarily starting up new threads when you use the ThreadPool, as its purpose is to spread the tasks over a limited number of threads).
If your task is long running, you should either break it up into logical sections that are put on the ThreadPool (or use the new Task framework), or spin up your own Thread object.
As to why you're experiencing the delay, the MSDN Documentation for the ThreadPool class says the following:
As part of its thread management strategy, the thread pool delays before creating threads. Therefore, when a number of tasks are queued in a short period of time, there can be a significant delay before all the tasks are started.
You only know that the ThreadPool hasn't reached its maximum thread count, not how many threads (if any) it actually has sitting idle.
The thread pool's maximum number of threads value is the maximum number that it can create. It is not the maximum number that are already created. The thread pool has logic that prevents it from spinning up a whole bunch of threads instantly.
If you call ThreadPool.QueueUserWorkItem 10 times in quick succession, the thread pool will not create 10 threads immediately. It will start a thread, delay, start another, etc.
I seem to recall that the delay was 500 milliseconds, but I can't find the documentation to verify that.
Here it is: The Managed Thread Pool:
The thread pool has a built-in delay (half a second in the .NET
Framework version 2.0) before starting new idle threads. If your
application periodically starts many tasks in a short time, a small
increase in the number of idle threads can produce a significant
increase in throughput. Setting the number of idle threads too high
consumes system resources needlessly.
You can control the number of idle threads maintained by the thread
pool by using the GetMinThreads and SetMinThreads
Note that this quote is taken from the .NET 3.5 version of the documentation. The .NET 4.0 version does not mention a delay.
Related
As far as I understand, .NET CLR creates a thread pool for each process. So each process has its own thread pool. And in every thread pool, there exist a certain number of threads available. It might be increasd or decreased as deemed necessary by the framework, but it starts with a predetermined number of threads for each process.
I wanted to find out the number of threads it will start with for a simple WPF application. When I used the System.Threading.ThreadPool.GetMaxThreads(out worker, out io) and System.Threading.ThreadPool.GetAvailableThreads(out worker, out io), I got the same result of 2047 worker threads and 1000 io threads. But I assume this can't be right, so this is not the right way to find the currently reserved threads in the thread pool.
So I looked at the thread count using Windows Task Manager and it showed 10 threads for the application. That seemed sensible and I came to the conclusion that the thread pool has 9 threads since one of the 10 is the main UI thread.
First of all, is my conclusion of 9 threads in thread pool correct? Second, what is the right way of querying it using c#?
I am trying to understand the impact of setting ThreadPool.SetMinthreads. I have multiple virtual applications running in one Azure App Service. My understanding is that all these virtual applications will share the App Pool, and will have only one worker process (Assuming the App Pool's max worker process will be 1).
I have the below two questions.
In this setup, if I set ThreadPool.SetMinThreads to let's say 100 worker threads and IO threads, can I safely assume that each app domain will have 100 worker threads and 100 IO threads when it is loaded? To be precise, the ThreadPool.SetMinThreads applies within the AppDomain, or Worker Process or App Pool? What is the scope of ThreadPool?
I also assume there is no limitation on the max threads the system can spawn as it is determined by the underlying host's capacity. This means, if I do not explicitly set ThreadPool.SetMaxThreads, the system will spawn new threads and will continue to do it if there is a continuous load till CPU/Memory spikes to the max. I am basing on the below statement to support my assumption:
Process and threads, for example, require physical memory, virtual
memory, and pool memory, so the number of processes or threads that
can be created on a given Windows system is ultimately determined by
one of these resources, depending on the way that the processes or
threads are created and which constraint is hit first.
https://blogs.technet.microsoft.com/markrussinovich/2009/07/05/pushing-the-limits-of-windows-processes-and-threads/
The MinThreads governs how many worker threads will be spawned without a delay.
Whenever you do something that requires a thread from the thread pool (whether worker or IOCP pool), the system will first see if there is a free thread.
If not, it looks to see how many threads are currently spawned. If that number is less than MinThreads, it immediately spawns a new thread. Otherwise it waits a short time, usually around 300-500ms, though that is system dependent. If there is still no free thread, it will then spawn a new thread.
Of course, this all is still limited by MaxThreads.
All that said, IIS is very good at figuring out a sensible number based on your machine and in most cases you are best to leave it alone; if you are just worried about serving requests then I wouldn't touch it personally. If, on the other hand, you are spawning a lot of background tasks yourself then it may be sensible. I'd strongly encourage you to measure it before you actually make changes.
Though... Setting MinThreads to 100 is rarely harmful, especially as the system will only start the number of threads it actually needs anyway
I've helped a client with an application which stopped doing it's work after a while (it eventually started doing work again).
The problem was that when a Task failed it used Thread.Sleep for 5 seconds (in the task). As there could be up to 800 tasks queued every two second you can imagine the problem if many of those jobs fails and invoked Thread.Sleep. None of those jobs were marked with TaskCreationOptions.LongRunning.
I rewrote the tasks so that Thread.Sleep (or Task.Delay for that matter) wasn't nessacary.
However, I'm interested in what the TaskScheduler (the default) did in that scenario. How and when do it increase the number of threads?
According to MSDN
Behind the scenes, tasks are queued to the ThreadPool, which has been
enhanced with algorithms (like hill-climbing) that determine and
adjust to the number of threads that maximizes throughput.
The ThreadPool will have a maximum number of threads depending on the environment. As you create more Tasks the pool can run them concurrently until it reaches its maximum number of threads, at which point any further tasks will be queued.
If you want to find out the maximum number of ThreadPool threads you can use System.Threading.ThreadPool.GetMaxThreads (you need to pass in two out int parameters, one that will be populated with the number of maximum worker threads and another that will be populated with the maximum number of asynchronous I/O threads).
If you want to get a better idea of what is happening in your application at runtime you can use Visual Studio's threads window by going to Debug -> Windows -> Threads (The entry will only be there when you are debugging so you'll need to set a break point in your application first).
This post is possibly of interest. It would seem the default task scheduler simply queues the task up in the ThreadPool's queue unless you use TaskCreationOptions.LongRunning. This means that it's up to the ThreadPool to decide when to create new threads.
There have been quite a few discussions on the issue, but they can't seem to explain my particular problem. I'm having serious performence issues when threading useing ThreadPool instead of Thread class.
The details:
I have built a tcp server, when the tcp server accepts a new client it spawns a new thread to handle that client. All Fairly simple, but it takes my server way too long to handle many concurrent clients. 30 seconds for about 35 simple clients that just send a 2048 byte buffer, recieve it and close.
After many stopwathes I found that ThreadPool.QueueUserWorkItem takes up to 26 seconds. I use it to spawn new threads for handling new clients.
After replacing ThreadPool.QueueUserWorkItem with new Thread() my performance improved to less then a second.
I would love some explanation to why this is happening.
Clarification:
The Delay has nothing to do with the client code, from the moment ThreadPool.QueueUserWorkItem is called until clientMsgHandler.HandleIncomingMsgs is started 20 seconds can pass.
The lag starts at very first threads, and actually improves slightly as the test continues. I'm less interested in solutions and more interested in an explanation to why it's happening.
The client does block, but for a very short while.
The server code:
private void AddTcpClientMsgHandler(TcpClient tcpClient)
{
//lock so no addition of client and closure can occur concurrently
Stopwatch watch = new Stopwatch();
watch.Start();
Monitor.Enter(this);
int pWatchIdx = watchIDX++;
if (!isOpen)
throw new ObjectDisposedException(ResourceAlreadyClosed);
TcpClientMsgHandler clientMsgHandler = CreateClientHandler(tcpClient);
clientMsgHandlerManager.AddTcpClientMsgHandler(clientMsgHandler);
//ThreadPool.QueueUserWorkItem(clientMsgHandler.HandleIncomingMsgs); takes 20 seconds to run
Thread thread = new Thread(clientMsgHandler.HandleIncomingMsgs);
thread.Start();
watch.Stop();
Monitor.Exit(this);
Console.WriteLine(string.Format("Iteration {0} took {1} Client {2}", pWatchIdx.ToString(),watch.Elapsed, tcpClient.Client.RemoteEndPoint));
}
Blocking code is the enemy of the ThreadPool. From the example you posted, it's not possible to tell where the blocking is happening, but I suggest that you review the code path to find out where the code is blocking. Run your server in the debugger until it starts showing high latency then break execution and take a look in the threads panel of VS. This will show you where the threads have blocked. Most likely this is on synchronous IO. Consider replacing with asynchronous code.
ThreadPool.QueueUserWorkItem - Queues a method for execution. The method executes when a thread pool thread becomes available.
The ThreadPool waits for an idle thread.
When an idle thread is found the ThreadPool uses it to execute your method.
Why ThreadPool is slow ?
The above two is not really a reason until you are running out of Threads in ThreadPool.
Under .NET 3.5 there are 2000 worker threads and 1000 IO completion port threads. (even more in 4.0,4.5) . Check Jon Skeet's answer (Active Thread Number in Thread Pool).
[For eg.]
.Net 2.0 defaults to 25 threads per available processor. This means that if you queue up 30 tasks, the last five will have to wait for threads to become available from the pool before being executed.
Solution :
SetMinThreads() to make the minimum number of threads 30 (for .Net 2.0).
This will increase performance since the ThreadPool won’t immediately create new threads when needed; it only does this on certain intervals.
Note : Using one thread per Client doesn't support more concurrency.
Use Asynchronous sockets - These are non-blocking sockets, except that you don’t have to poll: the stack sends the program a special window message whenever something "interesting" happens.
After the first few threads in the threadpool are used up, the system introduces a delay before starting each new threadpool thread (this doesn't affect re-used threads).
You can change the initial number of threadpool threads by setting ThreadPool.SetMinThreads to a suitably large value before you start any threads. BUT YOU ARE NOT SUPPOSED TO DO THAT! (So you didn't hear that from me... ;)
You should look into a way of reducing the number of threads instead of doing that.
I think it's depending about what you are doing in your clientMsgHandler.HandleIncomingMsgs() method.
The thread pool must be use only for really short treatments.
Furthermore, the thread pool has a default size of 25 worker threads per available processor, be careful of crosslock in your thread.
>> The Managed Thread Pool
I have a thread that I fire off every time the user scans a barcode.
Most of the time it is a fairly short running thread. But sometimes it can take a very long time (waiting on a invoke to the GUI thread).
I have read that it may be a good idea to use the ThreadPool for this rather than just creating my own thread for each scan.
But I have also read that if the ThreadPool runs out of threads then it will just wait until some other thread exits (not OK for what I am doing).
So, how likely is it that I am going to run out of threads? And is the benefit of the ThreadPool really worth it? (When I scan it does not seem to take too long for the scan to "run" the thread logic.)
It depends on what you mean by "a very long time" and how common that scenario is.
The MSDN topic "The Managed Thread Pool" offers good guidelines for when not to use thread pool threads:
There are several scenarios in which it is appropriate to create and manage your own threads instead of using thread pool threads:
You require a foreground thread.
You require a thread to have a particular priority.
You have tasks that cause the thread to block for long periods of time. The
thread pool has a maximum number of
threads, so a large number of blocked
thread pool threads might prevent
tasks from starting.
You need to place threads into a single-threaded apartment. All
ThreadPool threads are in the
multithreaded apartment.
You need to have a stable identity associated with the thread, or to
dedicate a thread to a task.
Since the user will never scan more than one barcode at a time, the memory costs of the threadpool might not be worth it - I'd stick with a single thread just waiting in the background.
The point of the thread pool is to amortize the cost of creating threads, which are not inexpensive to spin up and tear down. If you have a short-running task, the cost of creating/destroying the thread can be a significant portion of the overall run-time. The maximum number of threads in the thread pool depends on the version of the .NET Framework, typically dozens to hundreds per processor. The number of threads is scaled depending on available work.
Will you run out of threads and have to wait for a thread to become available? It depends on your workload. You can get the maximum number of threads available via ThreadPool.GetMaxThreads(). Chances are (based on the description of your problem) that this number is sufficiently high.
http://msdn.microsoft.com/en-us/library/system.threading.threadpool.getmaxthreads.aspx
Another option would be to manage your own pool of scan threads and assign them work rather than creating a new thread for every scan. Personally I would try the threadpool first and only manage your own threads if it proved necessary. Even better, I would look into async programming techniques in .NET. The methods will be run on the thread pool, but give you a much nicer programming experience than manual thread management.
If most of the time it is short running threads you could use the thread pool or a BackgroundWorker which draws threads from the pool.
An advantage I can see in your case is that threadpool class puts an upper limit on the amount of threads that may be active. It depends on the context of your application whether you will exhaust system resources. Exhausting a modern desktop system is VERY hard to do really.
If the software is used in a supermarket till it is highly unlikely that you will have more then 5 barcodes being analysed at the same time. If its run in a back-end server for a whole row of supermarket tills. Then perhaps 30-100 concurrent requests might be active.
With this sort of theory crafting it is highly unlikely that you will run out of threads, even on embedded hardware. If you have a dozen or so requests active at a time, and your code works, it's ok to just leave it as it is.
A thread pool is just an abstraction though, and you could have queue in the middle that queues request onto a thread-pool, in this scenario for the row-of-till example above, I'd feel comfortable queueing 100-1000 requests against a threadpool with 10 threads.
In .net (and on windows in general), the question should always be reversed: "Is creating a new thread worth it in this scenario?"
Creating a new thread is expensive, and doing it over and over again is almost certainly not worth it. The thread pool is cheap, and really should be the first thing you turn to when you need a new thread.
If you decide to spin up a new thread, soon you will start worrying about re-using the thread if it's already running. Then you will start worrying that sometimes the thread is running but it seems to be taking too long, and so you should make a new one. Then you're going to decide to have a thread not exit immediately upon finishing work, but to wait a little while in case new work comes in. And then... bam! You've created your own thread pool. At which point you should just back up and use the system-provided one.
The folks who mentioned that the thread pool might "run out of threads" were well-intentioned, but they did you a disservice. The limit on the number of threads in the thread pool is quite large. If you run into it, you have other problems.
(And, of course, since .net 2.0, you can set the maximum number of threads, so you can tweak the number if you absolutely have to.)
Others have directed you to MSDN: "The Managed Thread Pool". I will repeat that direction, as the article is good, but in my mind does not sell the thread pool hard enough. :)