This has been bugging me for a while. Assuming that I am not using any explicit form of task parallelism (e.g. Paralell.ForEach), how many threads are natively used when I deploy a web application to a Windows Server?
In other words, if the web server has an eight-core processor, does the application only use one of those cores if I am not explicitly telling it to use more?
I'll bet I have missed something simple here, so flame on--but I still want to know the answer. Thanks.
First, you have to consider that a web server is, by its nature, a multi-threaded system because it has to be able to respond to multiple requests at the same time (there are ways to do this in a single threaded environment, but it's not very efficient, and IIS does not support it). If two users access the site at the same time, two separate threads are servicing each request in most cases.
Once you get into scalability issues, and async support, then a thread can actually service multiple requests (or rather, a request can be put on a queue and the IIS worker threads can be reused to process other requests).
So you are looking at this from the aspect of a single user. And that single user will likely run only a single thread of activity during that web request (ignoring async, io completion ports, etc..). i.e. you can think of a single web request as being equal to a single thread in the general sense (but there are so many things that can spin off other threads, that this isn't something that you can really count on).
When you use something like Parallel.ForEach it's so that your single users request can now execute things in multiple threads to improve efficiency.
An IIS web site has a worker process associated with it. That worker process can be configured and tuned to control how many threads it uses. But just remember, that your web site is running under the control of IIS, not as it's own application.
The topic of threads can get really confusing as you move along .NET versions.
But to over-simplify, it depends on your processModel configuration for ASP.NET. (machine.config) autoConfig=true/false etc.
at a bare minimum, in .NET 4.0, the min/max WorkerThreads and min/max IOCompletionThreads determine the amount of threads in play.
e.g. If the minWorkerThreads = 1 (default value), then the total minThreads = 1 * no:of cores..
same with maxWorkerThreads.
Based on the load, the worker process can ramp up the threads needed to service the requests, using an undisclosed algorith.. minWorkerThreads to maxWorkerThreads. (of course based on availability of ThreadPool threads etc.)
Related
We're developing a web application in Asp.Net MVC 4 intended for hundreds of users.
We need to have a background service per user to work in an interval of a few minutes.
We are not sure whether to use Windows Services (multiple windows services) or to use a Thread Pool of processes. We think of Windows Services cause they're maintainable easily via windows server and that approach can save the overhead of programming a UI and manage threads. It also can easily run in an interval of time.
Is it possible for a Windows Service to automatically initiate a new instance for a new user who has just signed up (so we have multiple background windows services instances, one for each user)? if not the Windows Services option falls.
If the upper is possible, should We choose Windows Services approach or make our own managed Thread Pool of processes?
Certainly, starting a process per user guarantees you high memory overhead and non-scalability when you get into the 1000s. I don't see what starting a process (as opposed to a thread) could possibly save because the new process will contain at least one thread. Also, Windows Services have nothing to do with "logged in users". They are not made for multi-instancing.
You seem to want to run background work in ASP.NET MVC. Be aware, that this is hard. Using one Windows Service can make sense here.
The hard thing about background work is that worker processes can exit for many reasons. You must tolerate this. A service has the same problem: You need to deploy new versions and restart the server regularly. You also need an HA strategy so you need multiple servers.
I'm not convinced that a Windows Service would be a better choice even for long-running background work.
With 100s of concurrent background workers you should probably use async IO to not have 100s of threads dedicated.
I assume your background work waits most of the time. You can make the waiting async (using a timer) and all the rest synchronous. That gives you a simple implementation and vast savings in memory usage.
We wrote a open-source project, Revalee, to handle this type of workload. It uses a Windows Service to manage the task scheduling, but leverages your existing ASP.NET MVC application to handle the action out-of-band. Tasks can be in the hundreds of thousands and are persisted until successfully dispatched.
Here is a simple workflow diagram:
user initiated action
|
| ......................
| : future callback :
V V :
====================== ===========================
| Your web app | | Revalee Windows Service |
| | | |
====================== ===========================
| ^
| registers a callback now |
|________________________________|
Using a separate service to perform background tasks might introduce additional burden:
Having two separate apps will certainly increase complexity of your project: if you need any communication between the web app and the service, it will be more complex (and slower) than having the whole thing inside the web app. You also need to deploy two separate projects, so there some plumbing overhead that will double this way.
Performance wise, there is nothing you gain from this approach. On the contrary, having your own managed pool inside the web app will allow better scheduling of these threads and quite possibly allow you to run them more efficiently than simply letting Windows take care of this. You really don't want to spawn hundreds of processes (or threads) which would compete for resources simultaneously on the same machine.
If nothing else, keeping the whole functionality inside the web app app might simplify your hosting options. Installing and managing a Windows service might require more privileges than a cheap hosting provider is prepared to give you.
Having said that, running background tasks in ASP.NET means that you need to be prepared to have your threads aborted abruptly, due to exceptions, recycling or any other reason IIS can think of. Running these background tasks in a separate process will certainly be less susceptible to these ASP.NET quirks, so it ultimately boils down to a compromise: how important it is for you to make sure these tasks are never interrupted, and does it justify additional programming and maintenance effort?
[Edit]
If your concern is how to schedule these tasks inside the service, take a look at some scheduling libraries for .NET, like Quartz. It allows better control over scheduling than simply using a timer and (should you ever need them) provides some advanced features like persisting jobs (useful if you want to make sure your jobs will finish after restarts) and clustering for large scale applications.
Using a simple timer will work, but make sure that you understand how each of .NET timers dispatches its events.
I have a console application(c#) where I have to call various third party API's and collect data. This I have to do simultaneously for different users. I am using threads for it. But as the number of users are increasing this service is eating into the CPU performance. It is affecting other processes. Is there a way we can use threads for parallel processing but do not affect the CPU performance in a huge way.
I assume from your question that you're creating threads manually, and so the quick way to answer this is to suggest that you use an API like the Task Parallel Library, because this will take an arbitrary number of tasks and try to use a sensible number of threads to process them - so given 500 API requests, it would limit itself to just a few threads.
However, to answer in more detail: the typical reason that you would see this problem is that code is creating too many threads. Threads are not free resources - they are expensive.
A made up example based on your question might be this:
you have 5 3rd party APIs that you need to call, and each is going to return ~1MB of data per user
you call each API on a separate background thread, for each user
you have 100 users
you therefore have created 500 threads in total, each of which is waiting on data from the network
The problem here is that there are 500 threads the program is trying to manage, and they are all waiting on the slowest piece of the system - the network.
More simply, we are trying to download 500 pieces of data at once (which in this example would mean everything finishes slowly), rather than downloading them one at a time so that individual items will finish earlier. Because each thread will be doing nothing (just waiting for the network), the CPU will switch between idle threads continually. As you increase your number of users, the number of threads increases - which increases the CPU usage just for switch between threads, even though each thread is actually downloading more slowly. This is (approximately) why you'll be seeing slower performance as your user count goes up.
A better example would be to take the same scenario and use just one background thread:
you have 5 3rd party APIs that you need to call, and each is going to return ~1MB of data per user
each API call is put into a queue and the queue is processed by a single thread
you have 100 users
you therefore have 1 thread running in the background which is using the full available bandwidth of the network for each request
In this example, your CPU usage will be pretty consistent - no matter how many users you have, there is only one background thread running, so context switching is minimised. Each individual API call runs at the maximum rate of the network card and so finishes as quickly as possible.
The reality is that one thread is probably not enough: a single request is unlikely to saturate the network, as there will be limiting factors elsewhere. But this is something you can tune later: maybe 2 or 3 threads would be more performant, but 4 threads would be slower again. The general rule when threading is to start small and work up, not to create a thread for each piece of work.
First, run a profiler and checkout some refactoring tools to see if you can perform code optimization to resolve the issue. If your application is still overloading the server then setup or purchase load balancing. In the meantime, if you are running the latest OS's you could try setting a hacky CPU rate limit...however, that may not work for the needs you described.
I know about tools for load testing a WCF service, so please don't suggest I use one.
If I wanted to write a tool for calling a service (or invoking any action really) X times a second for Y seconds, what things do I need to consider?
My initial approach would be to have a timer fire at the required interval and create a task when it fires, but I'm concerned that this will simply queue a lot of tasks up waiting for threads from the thread pool to execute on and they will not invoke the service at the required times.
Would creating individual threads to do the work be better? Then I'd be concerned about creating a large number of threads.
So what strategies can I use?
It depends on your goals for scalability. Running a single session per thread is easier - the code is much simpler to run a series of synchronous I/O transactions on a thread. You can spawn thousands of threads on Windows and Linux, if tuned properly. If you need to scale much farther than that, then you'll need to use asynchronous I/O APIs and setup a pools of threads to service groups of those I/O channels. I'd suggest making the thread-to-I/O-channel ratio configurable and monitor the idle time of the threads in those pools...perhaps even allowing the pools to add more threads when needed.
I’m looking for the best way of using threads considering scalability and performance.
In my site I have two scenarios that need threading:
UI trigger: for example the user clicks a button, the server should read data from the DB and send some emails. Those actions take time and I don’t want the user request getting delayed. This scenario happens very frequently.
Background service: when the app starts it trigger a thread that run every 10 min, read from the DB and send emails.
The solutions I found:
A. Use thread pool - BeginInvoke:
This is what I use today for both scenarios.
It works fine, but it uses the same threads that serve the pages, so I think I may run into scalability issues, can this become a problem?
B. No use of the pool – ThreadStart:
I know starting a new thread takes more resources then using a thread pool.
Can this approach work better for my scenarios?
What is the best way to reuse the opened threads?
C. Custom thread pool:
Because my scenarios occurs frequently maybe the best way is to start a new thread pool?
Thanks.
I would personally put this into a different service. Make your UI action write to the database, and have a separate service which either polls the database or reacts to a trigger, and sends the emails at that point.
By separating it into a different service, you don't need to worry about AppDomain recycling etc - and you can put it on an entire different server if and when you want to. I think it'll give you a more flexible solution.
I do this kind of thing by calling a webservice, which then calls a method using a delegate asynchronously. The original webservice call returns a Guid to allow tracking of the processing.
For the first scenario use ASP.NET Asynchronous Pages. Async Pages are very good choice when it comes to scalability, because during async execution HTTP request thread is released and can be re-used.
I agree with Jon Skeet, that for second scenario you should use separate service - windows service is a good choice here.
Out of your three solutions, don't use BeginInvoke. As you said, it will have a negative impact on scalability.
Between the other two, if the tasks are truly background and the user isn't waiting for a response, then a single, permanent thread should do the job. A thread pool makes more sense when you have multiple tasks that should be executing in parallel.
However, keep in mind that web servers sometimes crash, AppPools recycle, etc. So if any of the queued work needs to be reliably executed, then moving it out of process is a probably a better idea (such as into a Windows Service). One way of doing that, which preserves the order of requests and maintains persistence, is to use Service Broker. You write the request to a Service Broker queue from your web tier (with an async request), and then read those messages from a service running on the same machine or a different one. You can also scale nicely that way by simply adding more instances of the service (or more threads in it).
In case it helps, I walk through using both a background thread and Service Broker in detail in my book, including code examples: Ultra-Fast ASP.NET.
I know there are many cases which are good cases to use multi-thread in an application, but when is it the best to multi-thread a .net web application?
A web application is almost certainly already multi threaded by the hosting environment (IIS etc). If your page is CPU-bound (and want to use multiple cores), then arguably multiple threads is a bad idea, as when your system is under load you are already using them.
The time it might help is when you are IO bound; for example, you have a web-page that needs to talk to 3 external web-services, talk to a database, and write a file (all unrelated). You can do those in parallel on different threads (ideally using the inbuilt async operations, to maximise completion-port usage) to reduce the overall processing time - all without impacting local CPU overly much (here the real delay is on the network).
Of course, in such cases you might also do better by simply queuing the work in the web application, and having a separate service dequeue and process them - but then you can't provide an immediate response to the caller (they'd need to check back later to verify completion etc).
IMHO you should avoid the use of multithread in a web based application.
maybe a multithreaded application could increase the performance in a standard app (with the right design), but in a web application you may want to keep a high throughput instead of speed.
but if you have a few concurrent connection maybe you can use parallel thread without a global performance degradation
Multithreading is a technique to provide a single process with more processing time to allow it to run faster. It has more threads thus it eats more CPU cycles. (From multiple CPU's, if you have any.) For a desktop application, this makes a lot of sense. But granting more CPU cycles to a web user would take away the same cycles from the 99 other users who are doing requests at the same time! So technically, it's a bad thing.
However, a web application might use other services and processes that are using multiple threads. Databases, for example, won't create a separate thread for every user that connects to them. They limit the number of threads to just a few, adding connections to a connection pool for faster usage. As long as there are connections available or pooled, the user will have database access. When the database runs out of connections, the user will have to wait.
So, basically, the use of multiple threads could be used for web applications to reduce the number of active users at a specific moment! It allows the system to share resources with multiple users without overloading the resource. Instead, users will just have to stand in line before it's their turn.
This would not be multi-threading in the web application itself, but multi-threading in a service that is consumed by the web application. In this case, it's used as a limitation by only allowing a small amount of threads to be active.
In order to benefit from multithreading your application has to do a significant amount of work that can be run in parallel. If this is not the case, the overhead of multithreading may very well top the benefits.
In my experience most web applications consist of a number of short running methods, so apart from the parallelism already offered by the hosting environment, I would say that it is rare to benefit from multithreading within the individual parts of a web application. There are probably examples where it will offer a benefit, but my guess is that it isn't very common.
ASP.NET is already capable of spawning several threads for processing several requests in parallel, so for simple request processing there is rarely a case when you would need to manually spawn another thread. However, there are a few uncommon scenarios that I have come across which warranted the creation of another thread:
If there is some operation that might take a while and can run in parallel with the rest of the page processing, you might spawn a secondary thread there. For example, if there was a webservice that you had to poll as a result of the request, you might spawn another thread in Page_Init, and check for results in Page_PreRender (waiting if necessary). Though it's still a question if this would be a performance benefit or not - spawning a thread isn't cheap and the time between a typical Page_Init and Page_Prerender is measured in milliseconds anyway. Keeping a thread pool for this might be a little bit more efficient, and ASP.NET also has something called "asynchronous pages" that might be even better suited for this need.
If there is a pool of resources that you wish to clean up periodically. For example, imagine that you are using some weird DBMS that comes with limited .NET bindings, but there is no pooling support (this was my case). In that case you might want to implement the DB connection pool yourself, and this would necessitate a "cleaner thread" which would wake up, say, once a minute and check if there are connections that have not been used for a long while (and thus can be closed off).
Another thing to keep in mind when implementing your own threads in ASP.NET - ASP.NET likes to kill off its processes if they have been inactive for a while. Thus you should not rely on your thread staying alive forever. It might get terminated at any moment and you better be ready for it.