So I have a program that serves as a sort of 'shell' for other programs. At its core, it gets passed a class, a method name, and some args, and handles the execution of the function, with the idea of allowing other programmers to basically schedule their processes to run on this shell service. Everything works fine except for one issue. Frequently, these processes that are scheduled for execution are very CPU heavy. At times, the processes called start using so much of the CPU, that the threads that I have which are responsible for checking schedules and firing off other jobs don't get a chance to run for quite some time, resulting in scheduling issues and a lack of sufficient responsiveness. Unfortunately, I can't insert Thread.Sleep() calls in the actual running code, as I don't really 'own' it.
So my question is: Is it possible to force an arbitrary thread (that I started) to sleep (yield) every so often, without modifying the actual code running in that thread? Barring that, is there some way to 'inject' Thread.Sleep() calls into code that I'm about to run dynamically, at run-time?
Not really. You could try changing the priority of the other thread with the Priority property, but you can't really inject a "sleep" call.
There's Thread.Suspend() but I strongly recommend you don't use it. You don't really want to get another thread to suspend at arbitrary times, as it might hold important resources. (That's why the method is marked as obsolete and has all kinds of warnings around it :)
Reducing the priority of the other tasks and potentially boosting the priority of your own tasks is probably the best approach.
No there is not (to my knowledge), you can however change the scheduling priority of a thread - http://msdn.microsoft.com/en-us/library/system.threading.thread.priority.aspx.
Lower the priority of the thread that is running the problem code as you start the thread (or raise the priority of your scheduling threads)
You can suspend a thread with Thread.Suspend (deprecated and not recommended) or you can lower its priority by changing Thread.Priority.
Actually, what you are stating is very much a security risk and thus isn't going to be supported by most platforms. It's generally not a good policy to allow other programs to affect your status without your permission.
If possible, you could probably load the prog in question on its own process that you create in your code, and then you can set that processe's priority or make it sleep, or what have you. I am no expert in windows programming, so your mileage may vary.
You should not be doing this. Threads should not Sleep() unless they themselves decide it's a good idea.
It's not relevant to your problem anyway. Your problem apparently is that management threads do not run, due to CPU starvation. To fix that, it's sufficient to give those threads a higher priority. At the end of each timeslice, the OS will determine which threads should run next. Thread priority is somewhat dynamic: threads that haven't run for a while will move up relative to threasd that just ran. But the higher base thread priority of your management threads will mean they don't need this dynamic boost to still get scheduled.
If this is still not sufficient, lower the priority of the worker threads. This will mean the worker thread has to be dormant for even more time before it gets a new timeslice.
Finally, make sure that the thread kicking off new worker threads runs at a very low priority. This will mean that no new threads are created if the system is hitting CPU limits. This implements a simple but robust throttling mechanism.
Related
Scenario
I have a Windows Forms Application. Inside the main form there is a loop that iterates around 3000 times, Creating a new instance of a class on a new thread to perform some calculations. Bearing in mind that this setup uses a Thread Pool, the UI does stay responsive when there are only around 100 iterations of this loop (100 Assets to process). But as soon as this number begins to increase heavily, the UI locks up into eggtimer mode and the thus the log that is writing out to the listbox on the form becomes unreadable.
Question
Am I right in thinking that the best way around this is to use a Background Worker?
And is the UI locking up because even though I'm using lots of different threads (for speed), the UI itself is not on its own separate thread?
Suggested Implementations greatly appreciated.
EDIT!!
So lets say that instead of just firing off and queuing up 3000 assets to process, I decide to do them in batches of 100. How would I go about doing this efficiently? I made an attempt earlier at adding "Thread.Sleep(5000);" after every batch of 100 were fired off, but the whole thing seemed to crap out....
If you are creating 3000 separate threads, you are pushing a documented limitation of the ThreadPool class:
If an application is subject to bursts
of activity in which large numbers of
thread pool tasks are queued, use the
SetMinThreads method to increase the
minimum number of idle threads.
Otherwise, the built-in delay in
creating new idle threads could cause
a bottleneck.
See that MSDN topic for suggestions to configure the thread pool for your situation.
If your work is CPU intensive, having that many separate threads will cause more overhead than it's worth. However, if it's very IO intensive, having a large number of threads may help things somewhat.
.NET 4 introduces outstanding support for parallel programming. If that is an option for you, I suggest you have a look at that.
More threads does not equal top speed. In fact too many threads equals less speed. If your task is simply CPU related you should only be using as many threads as you have cores otherwise you're wasting resources.
With 3,000 iterations and your form thread attempting to create a thread each time what's probably happening is you are maxing out the thread pool and the form is hanging because it needs to wait for a prior thread to complete before it can allocate a new one.
Apparently ThreadPool doesn't work this way. I have never checked it with threads before so I am not sure. Another possibility is that the tasks begin flooding the UI thread with invocations at which point it will give up on the GUI.
It's difficult to tell without seeing code - but, based on what you're describing, there is one suspect.
You mentioned that you have this running on the ThreadPool now. Switching to a BackgroundWorker won't change anything, dramatically, since it also uses the ThreadPool to execute. (BackgroundWorker just simplifies the invoke calls...)
That being said, I suspect the problem is your notifications back to the UI thread for your ListBox. If you're invoking too frequently, your UI may become unresponsive while it tries to "catch up". This can happen if you're feeding too much status info back to the UI thread via Control.Invoke.
Otherwise, make sure that ALL of your work is being done on the ThreadPool, and you're not blocking on the UI thread, and it should work.
If every thread logs something to your ui, every written log line must invoke the main thread. Better to cache the log-output and update the gui only every 100 iterations or something like that.
Since I haven't seen your code so this is just a lot of conjecture with some highly hopefully educated guessing.
All a threadpool does is queue up your requests and then fire new threads off as others complete their work. Now 3000 threads doesn't sounds like a lot but if there's a ton of processing going on you could be destroying your CPU.
I'm not convinced a background worker would help out since you will end up re-creating a manager to handle all the pooling the threadpool gives you. I think more you issue is you've got too much data chunking going on. I think a good place to start would be to throttle the amount of threads you start and maintain. The threadpool manager easily allows you to do this. Find a balance that allows you to process data while still keeping the UI responsive.
I see the following in Joseph Albahari's Threading book (http://www.albahari.com/threading/)
Thread.Sleep(0) relinquishes the thread’s current time slice
immediately, voluntarily handing over the CPU to other threads.
Framework 4.0’s new Thread.Yield() method does the same thing — except
that it relinquishes only to threads running on the same processor.
Is the context switch happen to some other thread within the same process or among the threads that are waiting to get CPU?
If the answer is the latter, is there any way to do context switch to some other thread that is in wait state in the same process?
I understand that the thread scheduling has been taken care by the operating system. But, got struck with a problem because of Thread.Sleep(0) and trying to find the solution for it.
Editing for more clarity about the problem:
The software has two threads (say A and B) and A will wait for a signal from B for 20 milliseconds and proceed regardless of the signal. A sets the signal and to let the processor continue with B, Thread.Sleep(0) applied as the software is a time critical application where every second maters. For a second both A and B didn't continued and restored (known with the help of the logs). We thought some other process in the same processor got the CPU time slice and now looking for alternatives.
The Thread.Yield method will switch to any thread which is ready to run on the current processor. It doesn't make any distinction about which process that Thread exists in
There is no way to yield to another thread in the same process, even by P/Invoke. Windows simply doesn't support it.
An alternative would be to use some kind of co-operative multitasking, such as TPL and async/await. When you await something, such as the awaitable object returned by Task.Yield(), it enables another task queued with the scheduler to start up. It's also quite a bit more efficient than using Thread.Yield(), but if you're not using it yet this will likely require a large overhaul of your app.
Thread.Yield() will just allow the scheduler to choose another thread within the same process that is ready to run, and resume it at whatever point it was stopped. It has nothing to do with time-slicing among processes, which is a completely different thing. (And rarely of concern unless you're programming the other process(es) as well.)
Note that the Yield() may have no effect at all, if the current thread is the only one able to run. It will just return (relatively immediately) from the Yield() call.
Your question about "context switching to another thread in the same process" is a bit mis-guided. You shouldn't think in those terms. If you need to wait for another thread to finish, use Join. If you need to signal to another thread that it should stop waiting and do something, there are a variety of mechanisms to use for that.
In short, your problem will get worse if you're trying to "outguess" the thread scheduler.
Perhaps you should be more explicit about the problem you're actually having.
Thread is a wrapper around the OS threads. Due to this scheduling of Threads is performed by OS kernel and Yield just a way to tell the kernel, that you want relinquish CPU but still stay runnable (unblocked). A kernel will consider your request as a good point to perform a rescheduling and give the CPU to some other waiting thread. OS is free to give CPU to any waiting thread from the runqueue disregard the process to which it belong. There is no way to affect to the scheduler decision unless it is your own scheduler and you use so called green threads and cooperative multitasking.
In regard to your problem: you need to use explicit synchronization if you want to achieve guaranteed results.
Yielding is a wrong way because it doesn't provide any guaranties to you.
There are a bunch of issues that can appear from its use.
For example, your thread B can simply have not enough time to accomplish its work and to send signal to A before A will be scheduled again, A can be scheduled immediately after Yield onto another CPU core, A even can be rescheduled again before the B will got a chance to be executed. Finally, other application can take a CPU. If you really care about time then raise priorities of both threads, but synchronize them explicitly.
I create about 5000 background workers that do intensive work in a console app. I'm also using an external library that instantiates an object, say ObjectX. At some point, say t0, ObjectX tries to obtain a thread from an os thread pool and start it, but I have no control on how it obtains this thread. Things work fine for 100 background workers. For 1000 background workers it takes about 10 minutes after t0 for ObjectX to obtain and start a thread.
Is there a way to set, in advance, a high priority for any threads that will be started in the future by an object?
As I think the answer to 1 is "no", is there a way to limit the priority of the background workers so as to somehow favor everything else? Even though I only want to 'favor' ObjectX.
The goal would be to always have available resources to run the thread launched by ObjectX, no matter how overloaded the machine is.
I'm using C# and the .Net fr 3.5, on a Windows 64bit machine.
The way threads work is that they are given processor time by the OS. When this happens this is called a context switch. A context switch takes about 2000-8000 cycles (i.e. depending on processor 2000-8000 instructions). If the OS has many CPUs or cores, it may not need to take the CPU away from one thread and give it to another--avoiding a context switch. There can only be one thread per CPU running at a time, when you have more threads that need CPU than CPUs then you're forcing a context switch. Context switches are performed no faster than the system quantum (every 20ms for client and 120ms for server).
If you have 5000 background workers you effectively have 5000 threads. Each of those threads is potentially vying for CPU time. On a client version of windows, that means 250,000 context switches per second. i.e. 500,000,000 to 2,000,000,000 cycles per second are devoted simply to switching between threads. (i.e. over and above the work your threads are performing) if it could even process that many context switches per second.
The recommended practice is to only have one CPU-bound thread per processor. A CPU-bound thread is one that spends very little time "waiting". The UI thread is not a CPU-bound thread. If your background workers are spending a lot of time waiting for locks, then they may not be CPU-bound either--but, in general, background worker threads are CPU-bound. (otherwise, what would be the point of using a background worker?).
Also, the OS spends a lot of time figuring out what thread needs to get the CPU next. When you start changing thread priorities you interfere with that and most of the time end up making your entire system slower (not just your application) rather than faster.
Update:
On a related not, it takes about 200,000 cycles to create a new thread and about 100,000 cycles to destroy a thread.
Update 2:
If the impetus of the question isn't simply "If it can be done" but to be able to scale workload, then as #JoshW/#Servy mention, using something like the Producer/Consumer Pattern would allow for scalability that could facilitate horizontal scaling to multiple computers/nodes via a queue or a service bus. Simple starting up an in ordinate amount of threads is not scalable beyond the # of CPUs. If what you truly want is an architecture that can scaled out because "available resources...how overloaded the machine is" is simply impossible.
Personally I think this is a bad idea, however... given the comments you have made on other answers and your request that "No matter how many background workers are create that ObjectX runs as soon as possible"... You could conceivably force your background workers to block using a ManualResetEvent.
For example at the top of your worker code you could block on a Manual reset event with the WaitOne method. This manual reset could be static or passed as an input parameter and wherever your ObjectX gets instantiated/called or whatever, you call the .Reset method on your ManualResetEvent. This would block all your workers at the WaitOne line. Next at the bottom of the code that runs ObjectX, call the ManualResetEvent.Set() method and that will unblock the workers.
Note this is NOT an efficient way to manage your threads, but if you "just have to make it work" and have time later to improve it... I suppose it's one possible solution.
The goal would be to always have available resources to run the thread launched by ObjectX, no matter how overloaded the machine is.
Then thread priorities might not be the right tool.. Remember, thread priorities are evil
In general, windows is not a real-time OS; especially, win32 does not even attempt to be soft real-time (IIRC, the NT kernel tried, at some point, to have at least support for soft real time subsystems, but I may be wrong). So there is no guarantee about available resources, or timing.
Also, are you worried about other threads in the system? Those threads are out of your control (what if the other threads are already at the system max priority?).
If you are worried about threads in your app... you can control and throttle them, using less threads/workers to do more work (batching work in bigger units, and submitting it to a worker, for example, or by using TPL or other tools that will handle and throttle thread usage for you)
That said, you could intercept when a thread is created (look for example this question https://stackoverflow.com/a/3802316/863564) see if it was created for ObjectX (for example, checking its name) and use SetThreadPriority to boost it.
I wrote some code that mass imports a high volume of users into AD. To refrain from overloading the server, I put a thread.sleep() in the code, executed at every iteration.
Is this a good use of the method, or is there a better alternative (.NET 4.0 applies here)?
Does Thread.Sleep() even aid in performance? What is the cost and performance impact of sleeping a thread?
The Thread.Sleep() method will just put the thread in a pause state for the specified amount of time. I could tell you there are 3 different ways to achieve the same Sleep() calling the method from three different Types. They all have different features. Anyway most important, if you use Sleep() on the main UI thread, it will stop processing messages during that pause and the GUI will look locked. You need to use a BackgroundWorker to run the job you need to sleep.
My opinion is to use the Thread.Sleep() method and just follow my previous advice. In your specific case I guess you'll have no issues. If you put some efforts looking for the same exact topic on SO, I'm sure you'll find much better explanations about what I just summarized before.
If you have no way to receive a feedback from the called service, like it would happen on a typical event driven system (talking in abstract..we could also say callback or any information to understand how the service is affected by your call), the Sleep may be the way to go.
I think that Thread.Sleep is one way to handle this; #cHao is correct that using a timer would allow you to do this in another fashion. Essentially, you're trying to cut down number of commands sent to the AD server over a period of time.
In using timers, you're going to need to devise a way to detect trouble (that's more intuitive than a try/catch). For instance, if your server starts stalling and responding slower, you're going to continue stacking commands that the server can't handle (which may cascade in other errors).
When working with AD I've seen the Domain Controller freak out when too many commands come in (similar to a DOS attack) and bring the server to a crawl or crash. I think by using the sleep method you're creating a manageable and measurable flow.
In this instance, using a thread with a low priority may slow it down, but not to any controllable level. The thread priority will only be a factor on the machine sending the commands, not to the server having to process them.
Hope this helps; cheers!
If what you want is not overload the server you can just reduce the priority of the thread.
Thread.Sleep() do not consume any resources. However, the correct way to do this is set the priority of thread to a value below than Normal: Thread.Current.Priority = ThreadPriority.Lowest for example.
Thread.Sleep is not that "evil, do not do it ever", but maybe (just maybe) the fact that you need to use it reflects some lack on solution design. But this is not a rule at all.
Personally I never find a situation where I have to use Thread.Sleep.
Right now I'm working on an ASP.NET MVC application that uses a background thread to load a lot of data from database into a memory cache and after that write some data to the database.
The only feature I have used to prevent this thread to eat all my webserver and db processors was reduce the thread priority to the Lowest level. That thread will get about to 35 minutes to conclude all the operations instead of 7 minutes if a use a Normal priority thread. By the end of process, thread will have done about 230k selects to the database server, but this do not has affected my database or webserver performance in a perceptive way for the user.
tip: remember to set the priority back to Normal if you are using a thread from ThreadPool.
Here you can read about Thread.Priority:
http://msdn.microsoft.com/en-us/library/system.threading.thread.priority.aspx
Here a good article about why not use Thread.Sleep in production environment:
http://msmvps.com/blogs/peterritchie/archive/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program.aspx
EDIT Like others said here, maybe just reduce your thread priority will not prevent the thread to send a large number of commands/data to AD. Maybe you'll get better results if you rethink all the thing and use timers or something like that. I personally think that reduce priority could resolve your problem, although I think you need to do some tests using your data to see what happens to your server and other servers involved in the process.
You could schedule the thread at BelowNormal priority instead. That said, that could potentially lead to your task never running if something else overloads the server. (Assuming Windows scheduling works the way the documentation on scheduling threads mentions for "some operating systems".)
That said, you said you're moving data into AD. If it's over the nework, it's entirely possible the CPU impact of your code will be negligible compared to I/O and processing on the AD side.
I don't see any issue with it except that during the time you put the thread to sleep then that thread will not be responsive. If that is your main thread then your GUI will become non responsive. If it is a background thread then you won't be able to communicate with it (eg to cancel it). If the time you sleep is short then it shouldn't matter.
I don't think reducing the priority of the thread will help as 1) your code might not even be running on the server and 2) most of the work being done by the server is probably not going to be on your thread anyway.
Thread.sleep does not aid performance (unless your thread has to wait for some resource). It incurs at least some overhead, and the amount of time that you sleep for is not guaranteed. The OS can decide to have your Thread sleep longer than the amount of time you specify.
As such, it would make more sense to do a significant batch of work between calls to Thread.Sleep().
Thread.Sleep() is a CPU-less wait state. Its overhead should be pretty minimal. If execute Thread.Sleep(0), you don't [necessarily] sleep, but you voluntarily surrender your time slice so the scheduler can let lower priority thread run.
You can also lower your thread's priority by setting Thread.Priority.
Another way of throttling your task is to use a Timer:
// instantiate a timer that 'ticks' 10 times per second (your ideal rate might be different)
Timer timer = new Timer( ImportUserIntoActiveDirectory , null , 0 , 100 ) ;
where ImportUserIntoActiveDirectory is an event handler that will import just user into AD:
private void ImportUserIntoActiveDirectory( object state )
{
// import just one user into AD
return
}
This lets you dial things in. The event handler is called on thread pool worker threads, so you don't tie up your primary thread. Let the OS do the work for you: all you do is decide on your target transaction rate.
In an effort to speed up the startup of my resource-hungry app, I've moved various startup tasks to background threads and marked those thread with 'Thread.Priority = Lowest`.
However, those low priority threads still execute pretty much in parallel with the application (as it loads its UI), as evidenced by the timeline on the ANTS Profiler. My understanding was that Lowest meant that the CPU will handle all higher priority threads first, then get the lower priority threads.
Is my understanding flawed?
The threads may be scheduled with the lowest priority, but they don't wait at the back of the line. They will probably still get enough CPU time slices to gobble up certain resources that are the real bottlenecks, like hard drive access. It really all depends on exactly what you are doing.
Is the initialization computation-intensive? Or web intensive/hard drive intensive. A multi-threading approach is going to be most effective when different tasks use different resources, or to allow computationally intensive operations run without blocking other operations.
A single-threaded approach could feasibly order the tasks to make the application appear to load faster, where-as the multithreaded approach may mean that everyone gets their hands in at the same time, possibly even getting in eachother's way.
Lowering the priority doesn't mean that the thread will always be the last one picked to get a time slot. If the lower priority thread hasn't got a time slot for a while, it will be more likely to get one. That way lower priority threads will run slower, but not completely stop.
Also, if the main thread is waiting for something, like for example waiting for the disk drive to return data, the other threads can run in that void. If the main thread does a lot of disk I/O, there will be a lot of holes to run other threads in.
If the CPU has more than a single core, the load will be more evenly distributed between threads. No matter how high priority a thread has, it will still only run on one core.
Is it possible to re-engineer your app so that the threads that you're trying to get to wait until the UI is loaded don't actually run at all until after the UI is loaded? This would do what you wish, forcing them to wait until the UI is loaded (because they're not even created/started), whereas the method you're employing causes them to execute less often, but still execute.