We Keep Coding

With C# and .NET, why doesn't the runtime automatically operate asynchronously?

This is such a basic (read noob) question regarding the .NET async/await library but I thought I'd ask it anyhow before rewriting our api's to be awaitable.
The Question
Why wouldn't the runtime simply evaluate any given thread that has a lot of idle time and automatically operate asynchronous whenever it gets to the blocking call.
Example of some routine
Web request to some app...
App starts database call...
Wait for response...(idle and long)
Receive recordset...
Return to client...
If I were the runtime environment, wouldn't it be wise to simply jot down that step 3 takes a while so I should use the current thread at that point, during its idle moments, to help out other routines that would normally be waiting for our current thread to be available?
Isn't it possible that at some point in the future we'll be able to toggle a flag in the app.config (or web.config) that says <system.runtime><asyncBehavior enableAsynchronousWhenIdle=true /></system.runtime>?

Sure it's possible but it completely breaks the current programming model. Before when you had a blocking call you were guaranteed that no other code would run on your thread. This change now allows re-entrant calls on the same thread.
For instance consider this case:
static int _processCount;
static object _lockObj = new object();
public Response ProcessRequest(Request request) {
lock (_lockObj) {
_processCount++;
var savedCount = _processCount;
// Make long running request
if (savedCount != _processCount)
throw new InvalidOperationException("Is my lock broken?");
}
}
Before we allow processing requests during the long running process this code is fine, but if we allow new requests to be processed on the thread while it is making a long running request we open up the possibility of this case.
Process Request A
Process Request A waits for the long running operation
The idle processing uses the thread to process Request B.
Request B enters the lock because locks have thread affinity
Request B waits for the long running operation
Request A returns from the long running operation and throws an exception because it's state has been corrupted.
So the code needs to be written in such a way that it is aware of the reentrancy potential. There is no way for the Framework to know if your code will break so that change will never happen.

.NET (nor any framework) isn't that smart. Unless you explicitly program code to run asynchronously, it has no way of knowing if any particularly code should run asynchronously. It can't look at your code and say, "Oh here's some code that runs awhile and blocking the UI thread, so I should run this in a separate thread so that the UI can update." As far as the framework is concerned--you intended it to operate that way--and it has no intelligence to override the way you coded into something more efficient.

Basically because both synchronous and asynchronous approaches are useful, and sometimes blocking is fine. Asynchrony isn't a silver bullet.
In the other hand, turning all synchronous code into asynchronous operations may break a lot of code base, both from the base class library (BCL) and third-party code, because asynchronous operations should synchronize access to shared resources and objects, and current synchronous code can be an actual bomb!

Resource usage of ThreadPool RegisterWaitForSingleObject

I am writing a server application which processes request from multiple clients. For the processing of requests I am using the threadpool.
Some of these requests modify a database record, and I want to restrict the access to that specific record to one threadpool thread at a time. For this I am using named semaphores (other processes are also accessing these records).
For each new request that wants to modify a record, the thread should wait in line for its turn.
And this is where the question comes in:
As I don't want the threadpool to fill up with threads waiting for access to a record, I found the RegisterWaitForSingleObject method in the threadpool.
But when I read the documentation (MSDN) under the section Remarks:
New wait threads are created automatically when required. ...
Does this mean that the threadpool will fill up with wait-threads? And how does this affect the performance of the threadpool?
Any other suggestions to boost performance is more than welcome!
Thanks!

Your solution is a viable option. In the absence of more specific details I do not think I can offer other tangible options. However, let me try to illustrate why I think your current solution is, at the very least, based on sound theory.
Lets say you have 64 requests that came in simultaneously. It is reasonable to assume that the thread pool could dispatch each one of those requests to a thread immediately. So you might have 64 threads that immediately begin processing. Now lets assume that the mutex has already been acquired by another thread and it is held for a really long time. That means those 64 threads will be blocked for a long time waiting for the thread that currently owns the mutex to release it. That means those 64 threads are wasted on doing nothing.
On the other hand, if you choose to use RegisterWaitForSingleObject as opposed to using a blocking call to wait for the mutex to be released then you can immediately release those 64 waiting threads (work items) and allow them to be put back into the pool. If I were to implement my own version of RegisterWaitForSingleObject then I would use the WaitHandle.WaitAny method which allows me to specify up to 64 handles (I did not randomly choose 64 for the number of requests afterall) in a single blocking method call. I am not saying it would be easy, but I could replace my 64 waiting threads for only a single thread from the pool. I do not know how Microsoft implemented the RegisterWaitForSingleObject method, but I am guessing they did it in a manner that is at least as efficient as my strategy. To put this another way, you should be able to reduce the number of pending work items in the thread pool by at least a factor of 64 by using RegisterWaitForSingleObject.
So you see, your solution is based on sound theory. I am not saying that your solution is optimal, but I do believe your concern is unwarranted in regards to the specific question asked.

IMHO you should let the database do its own synchronization. All you need to do is to ensure that you're sync'ed within your process.
Interlocked class might be a premature optimization that is too complex to implement. I would recommend using higher-level sync objects, such as ReaderWriterLockSlim. Or better yet, a Monitor.

An approach to this problem that I've used before is to have the first thread that gets one of these work items be responsible for any other ones that occur while it's processing the work item(s), This is done by queueing the work items then dropping into a critical section to process the queue. Only the 'first' thread will drop into the critical section. If a thread can't get the critical section, it'll leave and let the thread already operating in the critical section handle the queued object.
It's really not very complicated - the only thing that might not be obvious is that when leaving the critical section, the processing thread has to do it in a way that doesn't potentially leave a late-arriving workitem on the queue. Basically, the 'processing' critical section lock has to be released while holding the queue lock. If not for this one requirement, a synchronized queue would be sufficient, and the code would really be simple!
Pseudo code:
// `workitem` is an object that contains the database modification request
//
// `queue` is a Queue<T> that can hold these workitem requests
//
// `processing_lock` is an object use to provide a lock
// to indicate a thread is processing the queue
// any number of threads can call this function, but only one
// will end up processing all the workitems.
//
// The other threads will simply drop the workitem in the queue
// and leave
void threadpoolHandleDatabaseUpdateRequest(workitem)
{
// put the workitem on a queue
Monitor.Enter(queue.SyncRoot);
queue.Enqueue(workitem);
Monitor.Exit(queue.SyncRoot);
bool doProcessing;
Monitor.TryEnter(processing_queue, doProcessing);
if (!doProcessing) {
// another thread has the processing lock, it'll
// handle the workitem
return;
}
for (;;) {
Monitor.Enter(queue.SyncRoot);
if (queue.Count() == 0) {
// done processing the queue
// release locks in an order that ensures
// a workitem won't get stranded on the queue
Monitor.Exit(processing_queue);
Monitor.Exit(queue.SyncRoot);
break;
}
workitem = queue.Dequeue();
Monitor.Exit(queue.SyncRoot);
// this will get the database mutex, do the update and release
// the database mutex
doDatabaseModification(workitem);
}
}

ThreadPool creates a wait thread for ~64 waitable objects.
Good comments are here: Thread.sleep vs Monitor.Wait vs RegisteredWaitHandle?

Background consumer thread lifetime management best practices

I have a C# class library which starts up a background consumer thread (lazily) which listens for tasks to complete from a producer/consumer queue. This class library can be used from any type of .NET application and is currently in use under an ASP.NET MVC web site. The consumer thread is blocked most of the time until a request comes into the queue. There should be only one consumer thread for any given app domain.
I want to be able to gracefully shut down the consumer thread when the application exits, regardless of what type of application it is, e.g. Windows Forms, Console, or WPF application. The application code itself should be ignorant of the fact that this thread is lingering around waiting to be terminated.
How can I solve this thread lifetime issue from the class library's perspective? Is there some global application shutdown event I can tie into to tear down the thread gracefully then?
Right now, since it's in an ASP.NET MVC app domain, it really doesn't matter since those never get torn down gracefully anyway. But now that I'm starting to use it in scheduled console application tasks designed to terminate, I want to solve this problem once and for all. The console application does not terminate since the consumer thread is still active and blocked waiting for requests. I've exposed a public static Thread property on the class to issue an Abort() call when the console app is exiting but this is, quite frankly, disgusting.
Any pointers would be appreciated! Again, I don't want to have to write Windows Forms or WPF or console application specific code to solve the issue. A nice generic solution that every consumer of the class library can use would be best.

Set the thread's IsBackground property to true. Then it will not prevent the process from ending.
http://msdn.microsoft.com/en-us/library/system.threading.thread.isbackground.aspx
Alternately, instead of using Abort, use Interrupt. Much less disgusting.

This is a bit tricky. You are right on track that you want to avoid an abort as that might terminate the thread in the middle of an important operation (writing to a file, etc.). Setting the IsBackground property would have the same effect.
What you need to do is make your blocking queue cancellable. Too bad you are not using .NET 4.0 yet otherwise you could have used the BlockingCollection class. No worries though. You will just have to modify your custom queue so that it polls periodically for a stopping signal. Here is a quick and dirty implementation that I just whipped up using the canonical implementation of a blocking queue as a starting point.
public class CancellableBlockingCollection<T>
{
private Queue<T> m_Queue = new Queue<T>();
public T Take(WaitHandle cancel)
{
lock (m_Queue)
{
while (m_Queue.Count <= 0)
{
if (!Monitor.Wait(m_Queue, 1000))
{
if (cancel.WaitOne(0))
{
throw new ThreadInterruptedException();
}
}
}
return m_Queue.Dequeue();
}
}
public void Add(T data)
{
lock (m_Queue)
{
m_Queue.Enqueue(data);
Monitor.Pulse(m_Queue);
}
}
}
Notice how the Take method accepts a WaitHandle that can be tested for the cancel signal. This could be the same wait handle used in other parts of your code. The idea here is that the wait handle gets polled on a certain interval inside Take and if it is signaled then the whole method throws. The assumption is that throwing inside Take is okay since it is at a safe point because the consumer could not have been in the middle of an important operation. You can just allow the thread to disintegrate upon the exception at this point. Here is what your consumer thread may look like.
ManualResetEvent m_Cancel = new ManualResetEvent(false);
CancellableBlockingCollection<object> m_Queue = new CancellableBlockingCollection<object>();
private void ConsumerThread()
{
while (true)
{
object item = m_Queue.Take(m_Cancel); // This will throw if the event is signalled.
}
}
There are various other ways you could cancel the Take method. I chose to use a WaitHandle, but it could easily be done using a simple bool flag for example.
Update:
As Steven pointed out in the comments Thread.Interrupt does essentially this already. It will cause a thread to throw an exception on a blocking call...pretty much what I was after in this example, but with a lot more code. One caveat with Thread.Interrupt is that it only works during the canned blocking calls in the .NET BCL (like WaitOne, etc.). So you would not be able to cancel a long running computation in progress like you would if you used a more manual approach like the one in this answer. It is definitely great tool to keep in your back pocket and might just be useful in your specific scenario.

If you are using .net 4.0 a CancelationToken can be used to signal the thread that it is time to gracefully shut down. It is very useful as most Wait commands allow you to pass it a token and if the thread is in a wait when the thread is canceled it will automatically come out of the wait.

ASP.NET Threading: should I use the pool for DB and Emails actions?

I’m looking for the best way of using threads considering scalability and performance.
In my site I have two scenarios that need threading:
UI trigger: for example the user clicks a button, the server should read data from the DB and send some emails. Those actions take time and I don’t want the user request getting delayed. This scenario happens very frequently.
Background service: when the app starts it trigger a thread that run every 10 min, read from the DB and send emails.
The solutions I found:
A. Use thread pool - BeginInvoke:
This is what I use today for both scenarios.
It works fine, but it uses the same threads that serve the pages, so I think I may run into scalability issues, can this become a problem?
B. No use of the pool – ThreadStart:
I know starting a new thread takes more resources then using a thread pool.
Can this approach work better for my scenarios?
What is the best way to reuse the opened threads?
C. Custom thread pool:
Because my scenarios occurs frequently maybe the best way is to start a new thread pool?
Thanks.

I would personally put this into a different service. Make your UI action write to the database, and have a separate service which either polls the database or reacts to a trigger, and sends the emails at that point.
By separating it into a different service, you don't need to worry about AppDomain recycling etc - and you can put it on an entire different server if and when you want to. I think it'll give you a more flexible solution.

I do this kind of thing by calling a webservice, which then calls a method using a delegate asynchronously. The original webservice call returns a Guid to allow tracking of the processing.

For the first scenario use ASP.NET Asynchronous Pages. Async Pages are very good choice when it comes to scalability, because during async execution HTTP request thread is released and can be re-used.
I agree with Jon Skeet, that for second scenario you should use separate service - windows service is a good choice here.

Out of your three solutions, don't use BeginInvoke. As you said, it will have a negative impact on scalability.
Between the other two, if the tasks are truly background and the user isn't waiting for a response, then a single, permanent thread should do the job. A thread pool makes more sense when you have multiple tasks that should be executing in parallel.
However, keep in mind that web servers sometimes crash, AppPools recycle, etc. So if any of the queued work needs to be reliably executed, then moving it out of process is a probably a better idea (such as into a Windows Service). One way of doing that, which preserves the order of requests and maintains persistence, is to use Service Broker. You write the request to a Service Broker queue from your web tier (with an async request), and then read those messages from a service running on the same machine or a different one. You can also scale nicely that way by simply adding more instances of the service (or more threads in it).
In case it helps, I walk through using both a background thread and Service Broker in detail in my book, including code examples: Ultra-Fast ASP.NET.

Optimizing an ASMX web service with Multiple Long-Running Operations

I'm writing an ASP.NET web service using C# that has a DoLookup() function. For each call to the DoLookup() function I need my code to execute two separate queries: one to another web service at a remote site and one to a local database. Both queries have to complete before I can compile the results and return them as the response to the DoLookup method. The problem I'm dealing with is that I want to make this as efficient as possible, both in terms of response time and resource usage on the web server. We are expecting up to several thousand queries per hour. Here's a rough C#-like overview of what I have so far:
public class SomeService : System.Web.Services.WebService
{
public SomeResponse DoLookup()
{
// Do the lookup at the remote web service and get the response
WebResponse wr = RemoteProvider.DoRemoteLookup();
// Do the lookup at the local database and get the response
DBResponse dbr = DoDatabaseLookup();
SomeResponse resp = new SomeResponse( wr, dbr);
return resp;
}
}
The above code does everything sequentially and works great but now I want to make it more scalable. I know that I can call the DoRemoteLookup() function asynchronously ( RemoteProvider has BeginRemoteLookup / EndRemoteLookup methods) and that I can also do the database lookup asynchronously using the BeginExecuteNonQuery / EndExecuteNonQuery methods.
My question (finally) is this: how do I fire both the remote web service lookup AND the database lookup simultaneously on separate threads and ensure that they have both completed before returning the response?
The reason I want to execute both requests on separate threads is that they both potentially have long response times (1 or 2 seconds) and I'd like to free up the resources of the web server to handle other requests while it is waiting for responses. One additional note - I do have the remote web service lookup running asynchronously currently, I just didn't want to make the sample above too confusing. What I'm struggling with is getting both the remote service lookup AND the database lookup started at the same time and figuring out when they have BOTH completed.
Thanks for any suggestions.

You can use a pair of AutoResetEvents, one for each thread. At the end of thread execution, you call AutoResetEvents.Set() to trigger the event.
After spawning the threads, you use WaitAll() with the two AutoResetEvents. This will cause the thread to block until both events are set.
The caveat to this approach is that you must ensure the Set() is guarantee to be called, otherwise you will block forever. Additionally ensure that with threads you exercise proper exception handling, or you will inadvertently cause more performance issues when unhanded exceptions cause your web application to restart.
MSDN Has sample code regarding AutoResetEvent usage.

See Asynchronous XML Web Service Methods, How to: Create Asynchronous Web Service Methods and How to: Chain Asynchronous Calls with a Web Service Method.
But note the first paragraph of those articles:
This topic is specific to a legacy technology. XML Web services and XML Web service clients should now be created using Windows Communication Foundation (WCF).
BTW, doing things the way these articles say is important because it frees up the ASP.NET worker thread while the long-running task runs. Otherwise, you might be blocking the worker thread, preventing it from servicing further requests, and impacting scalability.

Assuming you can have a callback function for both the web request and the database lookup then something along these lines may work
bool webLookupDone = false;
bool databaseLookupDone = false;
private void FinishedDBLookupCallBack()
{
databaseLookupDone = true;
if(webLookupDone)
{
FinishMethod();
}
}
private void FinishedWebLookupCallBack()
{
webLookupDone = true;
if(databaseLookupDone)
{
FinishMethod();
}
}

I guess I don't have enough rep to upvote nor to comment. So this is a comment on John Saunders answer and Alan's comment on it.
You definitely want to go with John's answer if you are concerned about scalability and resource consumption.
There are two considerations here: Speeding up an individual request, and making your system handle many concurrent requests efficiently. The former both Alan's and John's answer achieve by performing the external calls in parallel.
The latter, and it sounds like that was your main concern, is achieved by not having threads blocked anywhere, i.e. John's answer.
Don't spawn your own threads. Threads are expensive, and there are already plenty of threads in the IO Threadpool that will handle your external calls for you if you use the asynch methods provided by the .net framework.
Your service's webmethod needs to be asynch as well. Otherwise a worker thread will be blocked until your external calls are done (it's still 1-2 seconds even if they run in parallel). And you only have 12 threads per CPU handling incoming requests (if your machine.config is set according to recommendation.) I.e. you would at most be able to handle 12 concurrent requests (times the # of CPUs). On the other hand if your web method is asynch the Begin will return pretty much instantenously and the thread returned to the worker thread pool ready to handle another incoming request, while your external calls are being waited on by the IO completion port, where they will be handled by threads from the IO thread pool, once they return.