In an effort to speed up the startup of my resource-hungry app, I've moved various startup tasks to background threads and marked those thread with 'Thread.Priority = Lowest`.
However, those low priority threads still execute pretty much in parallel with the application (as it loads its UI), as evidenced by the timeline on the ANTS Profiler. My understanding was that Lowest meant that the CPU will handle all higher priority threads first, then get the lower priority threads.
Is my understanding flawed?
The threads may be scheduled with the lowest priority, but they don't wait at the back of the line. They will probably still get enough CPU time slices to gobble up certain resources that are the real bottlenecks, like hard drive access. It really all depends on exactly what you are doing.
Is the initialization computation-intensive? Or web intensive/hard drive intensive. A multi-threading approach is going to be most effective when different tasks use different resources, or to allow computationally intensive operations run without blocking other operations.
A single-threaded approach could feasibly order the tasks to make the application appear to load faster, where-as the multithreaded approach may mean that everyone gets their hands in at the same time, possibly even getting in eachother's way.
Lowering the priority doesn't mean that the thread will always be the last one picked to get a time slot. If the lower priority thread hasn't got a time slot for a while, it will be more likely to get one. That way lower priority threads will run slower, but not completely stop.
Also, if the main thread is waiting for something, like for example waiting for the disk drive to return data, the other threads can run in that void. If the main thread does a lot of disk I/O, there will be a lot of holes to run other threads in.
If the CPU has more than a single core, the load will be more evenly distributed between threads. No matter how high priority a thread has, it will still only run on one core.
Is it possible to re-engineer your app so that the threads that you're trying to get to wait until the UI is loaded don't actually run at all until after the UI is loaded? This would do what you wish, forcing them to wait until the UI is loaded (because they're not even created/started), whereas the method you're employing causes them to execute less often, but still execute.
Related
I create about 5000 background workers that do intensive work in a console app. I'm also using an external library that instantiates an object, say ObjectX. At some point, say t0, ObjectX tries to obtain a thread from an os thread pool and start it, but I have no control on how it obtains this thread. Things work fine for 100 background workers. For 1000 background workers it takes about 10 minutes after t0 for ObjectX to obtain and start a thread.
Is there a way to set, in advance, a high priority for any threads that will be started in the future by an object?
As I think the answer to 1 is "no", is there a way to limit the priority of the background workers so as to somehow favor everything else? Even though I only want to 'favor' ObjectX.
The goal would be to always have available resources to run the thread launched by ObjectX, no matter how overloaded the machine is.
I'm using C# and the .Net fr 3.5, on a Windows 64bit machine.
The way threads work is that they are given processor time by the OS. When this happens this is called a context switch. A context switch takes about 2000-8000 cycles (i.e. depending on processor 2000-8000 instructions). If the OS has many CPUs or cores, it may not need to take the CPU away from one thread and give it to another--avoiding a context switch. There can only be one thread per CPU running at a time, when you have more threads that need CPU than CPUs then you're forcing a context switch. Context switches are performed no faster than the system quantum (every 20ms for client and 120ms for server).
If you have 5000 background workers you effectively have 5000 threads. Each of those threads is potentially vying for CPU time. On a client version of windows, that means 250,000 context switches per second. i.e. 500,000,000 to 2,000,000,000 cycles per second are devoted simply to switching between threads. (i.e. over and above the work your threads are performing) if it could even process that many context switches per second.
The recommended practice is to only have one CPU-bound thread per processor. A CPU-bound thread is one that spends very little time "waiting". The UI thread is not a CPU-bound thread. If your background workers are spending a lot of time waiting for locks, then they may not be CPU-bound either--but, in general, background worker threads are CPU-bound. (otherwise, what would be the point of using a background worker?).
Also, the OS spends a lot of time figuring out what thread needs to get the CPU next. When you start changing thread priorities you interfere with that and most of the time end up making your entire system slower (not just your application) rather than faster.
Update:
On a related not, it takes about 200,000 cycles to create a new thread and about 100,000 cycles to destroy a thread.
Update 2:
If the impetus of the question isn't simply "If it can be done" but to be able to scale workload, then as #JoshW/#Servy mention, using something like the Producer/Consumer Pattern would allow for scalability that could facilitate horizontal scaling to multiple computers/nodes via a queue or a service bus. Simple starting up an in ordinate amount of threads is not scalable beyond the # of CPUs. If what you truly want is an architecture that can scaled out because "available resources...how overloaded the machine is" is simply impossible.
Personally I think this is a bad idea, however... given the comments you have made on other answers and your request that "No matter how many background workers are create that ObjectX runs as soon as possible"... You could conceivably force your background workers to block using a ManualResetEvent.
For example at the top of your worker code you could block on a Manual reset event with the WaitOne method. This manual reset could be static or passed as an input parameter and wherever your ObjectX gets instantiated/called or whatever, you call the .Reset method on your ManualResetEvent. This would block all your workers at the WaitOne line. Next at the bottom of the code that runs ObjectX, call the ManualResetEvent.Set() method and that will unblock the workers.
Note this is NOT an efficient way to manage your threads, but if you "just have to make it work" and have time later to improve it... I suppose it's one possible solution.
The goal would be to always have available resources to run the thread launched by ObjectX, no matter how overloaded the machine is.
Then thread priorities might not be the right tool.. Remember, thread priorities are evil
In general, windows is not a real-time OS; especially, win32 does not even attempt to be soft real-time (IIRC, the NT kernel tried, at some point, to have at least support for soft real time subsystems, but I may be wrong). So there is no guarantee about available resources, or timing.
Also, are you worried about other threads in the system? Those threads are out of your control (what if the other threads are already at the system max priority?).
If you are worried about threads in your app... you can control and throttle them, using less threads/workers to do more work (batching work in bigger units, and submitting it to a worker, for example, or by using TPL or other tools that will handle and throttle thread usage for you)
That said, you could intercept when a thread is created (look for example this question https://stackoverflow.com/a/3802316/863564) see if it was created for ObjectX (for example, checking its name) and use SetThreadPriority to boost it.
I'm trying to execute a long-running piece of code in a "background" thread and by "background" I mean low-priority thread not .NET background term. I created a thread, set its priority to Lowest and still 100% CPU is used if no other threads are running. The situation improves when I manually call Thread.Sleep(1) but I don't want to change my code and controlling CPU utilization via Thread.Sleep is highly unaccurate.
I want some way of executing a piece of code which will be throttled to ensure CPU usage at 5%-10%. Also I don't want to change the code too much (it is quite complex).
Is it possible?
It sounds a bit like you need to solve the wrong issue - instead of trying to throttle the thread to a set CPU usage (which I'm not sure you can do), is there something wrong with what the thread is doing/how it's doing it? i.e. if it uses 100% CPU, is there nothing you could do to that logic to make it less CPU intensive in the first place?
You may find you can rework/optimise it to prevent it being this intensive in the first place, hence solving the problem at the root.
UPDATE: periodically read the external sources using a timer. A good reference is here. You will solve the root cause of the high CPU usage by not continually checking those sources, but by doing it periodically (e.g. even if you do it every 15 seconds say, that will make a vast difference)
Perhaps you should have your "background" thread check a shared boolean that is set when the high-priority thread is working and wait until it's unset.
Setting the thread priority is not an absolute thing. It's relative to other threads in the program.
Setting the thread priority to Normal will mean that it gets an equal "share" of the processor time (with other Normal threads).
Setting the priority to Low (or Lowest) will mean that it gets less of the processor time compared to Normal threads in the same program, and setting it to High (or Highest) will mean that it gets more.
If there is only one thread then it will get as much CPU as it needs regardless of the priority you set.
As AdaTheDev says, if it's using 100% CPU then the issue is with your algorithm not how Windows is allocating processor time to the thread.
I am working on maintaining someone else's code that is using multithreading, via two methods:
1: ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf ReadData), objUpdateItem)
2: Dim aThread As New Thread(AddressOf LoadCache)
aThread.Start()
However, on a dual core machine, I am only getting 50% CPU utlilization, and on a dual core with hyperthreadin enabled machine, I am only getting 25% CPU utilization.
Obviously threading is extremely complicated, but this behaviour would seem to indicate that I am not understanding some simple fundamental fact?
UPDATE
The code is too horribly complex to post here unfortunately, but for reference purposes, here is roughly what happens....I have approx 500 Accounts whose data is loaded from the database into an in memory cache...each account is loaded individually, and that process first calls a long running stored procedure, followed by manipulation and caching of the returned data. So, the point of threading in this situation is that there is indeed a bottleneck hitting the database (ie: the thread will be idled for up to 30 seconds waiting for the query to return), so we thread to allow others to begin processing the data they have received from Oracle.
So, the main thread executes:
ThreadPool.QueueUserWorkItem(New WaitCallback(AddressOf ReadData), objUpdateItem)
Then, the ReadData() then proceeds to execute (exactly once):
Dim aThread As New Thread(AddressOf LoadCache)
aThread.Start()
And this is occurring in a recursive function, so the QueueUserWorkItem can be executing multiple times, which in turn then executes exactly one new thread via the aThread.Start
Hopefully that gives a decent idea of how things are happening.
So, under this scenario, should this not theoretically pin both cores, rather than maxing out at 100% on one core, while the other core is essentially idle?
That code starts one thread that will go an do something. To get more than one core working you need to start more than one thread and get them both busy. Starting a thread to do some work, and then having your main thread wait for it won't get the task done any quicker. It is common to start a long running task on a background thread so that the UI remains responsive, which may be what this code was intended to do, but it won't make the task get done any quicker.
#Judah Himango - I had assumed that those two lines of code were samples of how multi-threading were being achieved in two different places in the program. Maybe the OP can clarify if this is the case or if these two lines really are in the one method. If they are part of one method then we will need to see what the two methods are actually doing.
Update:
That does sound like it should max out both cores. What do you mean by recursivly calling ReadData()? If each new thread is only calling ReadData at or near its end to start the next thread then that could explain the behaviour you are seeing.
I am not sure that this is actaully a good idea. If the stored proc takes 30 seconds to get the data then presumably it is placing a fair load on the database server. Running it 500 times in parallel is just going to make things worse. Obviously I don't know your database or data, but I would look at improving the performance of the stored proc.
If multi threading does look like the way forward, then I would have a loop on the main thread that calls ThreadPool.QueueUserWorkItem once for each account that needs loading. I would also remove the explicit thread creation and only use the thread pool. That way you are less likely to starve the local machine by creating too many threads.
How many threads are you spinning up? It may seem primitive (wait a few years, and you won't need to do this anymore), but your code has got to figure out an optimal number of threads to start, and spin up that many. Simply running a single thread won't make things any faster, and won't pin a physical processor, though it may be good for other reasons (a worker thread to keep your UI responsive, for instance).
In many cases, you'll want to be running a number of threads equal to the number of logical cores available to you (available from Environment.ProcessorCount, I believe), but it may have some other basis. I've spun up a few dozen threads, talking to different hosts, when I've been bound by remote process latency, for instance.
Multi-Threaded and Multi-Core are two different things. Doing things Multi-Threaded often won't offer you an enormous increase in performance, sometimes quite the opposite. The Operating System might do a few tricks to spread your cpu cycles over multiple cores, but that's where it ends.
What you are looking for is Parallelism. The .NET 4.0 framework will add a lot of new features to support Parallelism. Have a sneak-peak here:
http://www.danielmoth.com/Blog/2009/01/parallelising-loops-in-net-4.html
The CPU behavior would indicate that the application is only utilizing one logical processor. 50% would be one proc out of 2 (proc+proc). 25% would be one logical processor out of 4 (proc + HT + proc + HT)
How many threads to you have in total and do you have any locks in LoadCache. A SyncLock may a multi-thread system act as a single thread (by design). Also if your only spool one thread you will only get one worker thread.
CPU utilization is suggesting that you're only using one core; this may suggest that you've added threading to a portion where it is not beneficial (in this case, where CPU time is not a bottle neck).
If Loading the Cache or reading data happens very quickly, multi threading won't provide a massive improvement in speed performance. Similarly, if you're encountering a different bottleneck (slow bandwidth to a server, etc), it may not show up as CPU usage.
Are there any benefits to limiting the number of concurrent threads doing a given task to equal the number of processors on the host system? Or better to simply trust libraries such as .NET's ThreadPool to do the right thing ... even if there are 25 different concurrent threads happening at any one given moment?
Most threads are not CPU bound, they end up waiting on IO or other events. If you look at your system now, I imagine you have 100's (if not 1000's) of threads executing with no problems. By that measure, you're probably best just leaving the .NET thread pool to do the right thing!
However, if the threads were all CPU bound (e.g. something like ray tracing) then it would be a good idea to limit the number of threads to the number of cores, otherwise chances are that context switching will begin to hurt performance.
The threadpool already does a reasonably good job at this. It tries to limit the number of running threads to the number of CPU cores in your machine. When one thread ends, it immediately schedules another eligible thread for execution.
Every 0.5 seconds, it evaluates what is going on with the running threads. When the threads have been running too long, it assumes they are stalled and allows another thread to start executing. You'll now have more threads running than you have CPU cores. This can go up to the maximum number of allowed thread, as set by ThreadPool.SetMaxThreads().
Starting around .NET 2.0 SP1, the default maximum number of threads was increased considerably to 250 times the number of cores. You should never ever get there. If you do, you would have wasted about 2 minutes of time where a possibly non-optimal number of threads were running. Those threads however would all have to be blocking for that long, not exactly a typical execution pattern for a thread. On the other hand, if these threads are all waiting on the same kind of resource they are likely to just take turns, adding more threads cannot improve throughput.
Long story short, the thread pool will work well if you run threads that execute quickly (seconds at most) and don't block for a long time. You probably ought to consider creating your own Thread objects when your code doesn't match that pattern.
Well, if your bottleneck is ONLY processors, then it might make sense, but that would ignore all memory and other i/o bottlenecks, and chances are at least your cache memory is throwing page faults and other events that would slow the threads.
I'd trust the library myself. Threads wait for all kinds of things, and you don't want your application to slow down because it can't spawn a new thread, even though most of the rest are just sleeping, waiting for some event or resource.
Measure your application under a variety of thread:processor ratios. Come to conclusions based on hard data about your application. Accept no arguments from first principles about what performance you should get, only what you do get matters.
So I have a program that serves as a sort of 'shell' for other programs. At its core, it gets passed a class, a method name, and some args, and handles the execution of the function, with the idea of allowing other programmers to basically schedule their processes to run on this shell service. Everything works fine except for one issue. Frequently, these processes that are scheduled for execution are very CPU heavy. At times, the processes called start using so much of the CPU, that the threads that I have which are responsible for checking schedules and firing off other jobs don't get a chance to run for quite some time, resulting in scheduling issues and a lack of sufficient responsiveness. Unfortunately, I can't insert Thread.Sleep() calls in the actual running code, as I don't really 'own' it.
So my question is: Is it possible to force an arbitrary thread (that I started) to sleep (yield) every so often, without modifying the actual code running in that thread? Barring that, is there some way to 'inject' Thread.Sleep() calls into code that I'm about to run dynamically, at run-time?
Not really. You could try changing the priority of the other thread with the Priority property, but you can't really inject a "sleep" call.
There's Thread.Suspend() but I strongly recommend you don't use it. You don't really want to get another thread to suspend at arbitrary times, as it might hold important resources. (That's why the method is marked as obsolete and has all kinds of warnings around it :)
Reducing the priority of the other tasks and potentially boosting the priority of your own tasks is probably the best approach.
No there is not (to my knowledge), you can however change the scheduling priority of a thread - http://msdn.microsoft.com/en-us/library/system.threading.thread.priority.aspx.
Lower the priority of the thread that is running the problem code as you start the thread (or raise the priority of your scheduling threads)
You can suspend a thread with Thread.Suspend (deprecated and not recommended) or you can lower its priority by changing Thread.Priority.
Actually, what you are stating is very much a security risk and thus isn't going to be supported by most platforms. It's generally not a good policy to allow other programs to affect your status without your permission.
If possible, you could probably load the prog in question on its own process that you create in your code, and then you can set that processe's priority or make it sleep, or what have you. I am no expert in windows programming, so your mileage may vary.
You should not be doing this. Threads should not Sleep() unless they themselves decide it's a good idea.
It's not relevant to your problem anyway. Your problem apparently is that management threads do not run, due to CPU starvation. To fix that, it's sufficient to give those threads a higher priority. At the end of each timeslice, the OS will determine which threads should run next. Thread priority is somewhat dynamic: threads that haven't run for a while will move up relative to threasd that just ran. But the higher base thread priority of your management threads will mean they don't need this dynamic boost to still get scheduled.
If this is still not sufficient, lower the priority of the worker threads. This will mean the worker thread has to be dormant for even more time before it gets a new timeslice.
Finally, make sure that the thread kicking off new worker threads runs at a very low priority. This will mean that no new threads are created if the system is hitting CPU limits. This implements a simple but robust throttling mechanism.