We Keep Coding

UWP handling many threads

So I am developing a UWP application that has a large number of threads. Previously I would start all the threads with System.Threading.Tasks.Task.Run(), save their thread handles to an array, then Task.WaitAll() for completion and get the results. This currently is taking too much memory though.
I have changed my code to only wait for a smaller amount of threads and copy their results out before continuing on to more of the threads. Since UWP the UWP implementation of Task does not implement IDisposable, what is the proper way to signal the framework that I am done with a task so it can be garbage collected? I would like to read out the results of the treads after a certain number of them come in and dispose of the threads resources to make space for the next threads.
Thanks so much!

Just to point out an issue which might be degrading the performance of your application: You are deliberately blocking the thread until all Tasks complete rather than actually await for them. That would make sense, if you are not performing Asynchronous work inside them, but if you are, you should definitely switch to:
Task.WhenAll rather than Task.WaitAll , such as this:
List<Tasks> tasks = new List<Tasks> { Method1(), Method2(), ... };
Task result = await Task.WhenAll(tasks);
This way, you are actually leveraging the asynchrony of your app, and you will not block the current thread until all the tasks are completed, like Task.WaitAll() does.
Since you are utilizing the Task.Run() method, instead of the Task.Factory.StartNew(), the TaskScheduler used is the default, and utilizes Threads from the Thread Pool. So you will not actually end up blocking the UI thread, but blocking many Thread Pool threads, is also not good.
Taking from Microsoft documentation, for one of the cases where Thread Pools should not be used:
You have tasks that cause the thread to block for long periods of
time. The thread pool has a maximum number of threads, so a large
number of blocked thread pool threads might prevent tasks from
starting.
Edit:
I do not need anything else but I will look in to that! Thanks! So is
there any way I can get it to run the Tasks like a FIFO with just the
API's available with the default thread pool?
You should take a look, into Continuations
A continuation is nothing else other than a task which is activated whenever it's antecedent task/tasks have completed. If you have specific tasks which you only want to execute after another task has completed you should take a look into Continuations, since they are extremely flexible, and you can actually create really complex flow of Tasks to better suit your needs.

Garbage collection on a .Net application always works the same, when a variable is not needed anymore (out of scope) it is collected.
Why do you think the threads are consuming the memory? It is much likely than the process inside the threads is the one consuming the memory.

Deadlock when accessing StackExchange.Redis

I'm running into a deadlock situation when calling StackExchange.Redis.
I don't know exactly what is going on, which is very frustrating, and I would appreciate any input that could help resolve or workaround this problem.
In case you have this problem too and don't want to read all this;
I suggest that you'll try setting PreserveAsyncOrder to false.
ConnectionMultiplexer connection = ...;
connection.PreserveAsyncOrder = false;
Doing so will probably resolve the kind of deadlock that this Q&A is about and could also improve performance.
Our setup
The code is run as either a Console application or as an Azure Worker Role.
It exposes a REST api using HttpMessageHandler so the entry point is async.
Some parts of the code have thread affinity (is owned by, and must be run by, a single thread).
Some parts of the code is async-only.
We are doing the sync-over-async and async-over-sync anti-patterns. (mixing await and Wait()/Result).
We're only using async methods when accessing Redis.
We're using StackExchange.Redis 1.0.450 for .NET 4.5.
Deadlock
When the application/service is started it runs normally for a while then all of a sudden (almost) all incoming requests stop functioning, they never produce a response. All those requests are deadlocked waiting for a call to Redis to complete.
Interestingly, once the deadlock occur, any call to Redis will hang but only if those calls are made from an incoming API request, which are run on the thread pool.
We are also making calls to Redis from low priority background threads, and these calls continue to function even after the deadlock occurred.
It seems as if a deadlock will only occur when calling into Redis on a thread pool thread. I no longer think this is due to the fact that those calls are made on a thread pool thread. Rather, it seems like any async Redis call without continuation, or with a sync safe continuation, will continue to work even after the deadlock situation has occurred. (See What I think happens below)
Related
StackExchange.Redis Deadlocking
Deadlock caused by mixing await and Task.Result (sync-over-async, like we do). But our code is run without synchronization context so that doesn't apply here, right?
How to safely mix sync and async code?
Yes, we shouldn't be doing that. But we do, and we'll have to continue doing so for a while. Lots of code that needs to be migrated into the async world.
Again, we don't have a synchronization context, so this should not be causing deadlocks, right?
Setting ConfigureAwait(false) before any await has no effect on this.
Timeout exception after async commands and Task.WhenAny awaits in StackExchange.Redis
This is the thread hijacking problem. What's the current situation on this? Could this be the problem here?
StackExchange.Redis async call hangs
From Marc's answer:
...mixing Wait and await is not a good idea. In addition to deadlocks, this is "sync over async" - an anti-pattern.
But he also says:
SE.Redis bypasses sync-context internally (normal for library code), so it shouldn't have the deadlock
So, from my understanding StackExchange.Redis should be agnostic to whether we're using the sync-over-async anti-pattern. It's just not recommended as it could be the cause of deadlocks in other code.
In this case, however, as far as I can tell, the deadlock is really inside StackExchange.Redis. Please correct me if I'm wrong.
Debug findings
I've found that the deadlock seems to have its source in ProcessAsyncCompletionQueue on line 124 of CompletionManager.cs.
Snippet of that code:
while (Interlocked.CompareExchange(ref activeAsyncWorkerThread, currentThread, 0) != 0)
{
// if we don't win the lock, check whether there is still work; if there is we
// need to retry to prevent a nasty race condition
lock(asyncCompletionQueue)
{
if (asyncCompletionQueue.Count == 0) return; // another thread drained it; can exit
}
Thread.Sleep(1);
}
I've found that during the deadlock; activeAsyncWorkerThread is one of our threads that is waiting for a Redis call to complete. (our thread = a thread pool thread running our code). So the loop above is deemed to continue forever.
Without knowing the details, this sure feels wrong; StackExchange.Redis is waiting for a thread that it thinks is the active async worker thread while it is in fact a thread that is quite the opposite of that.
I wonder if this is due to the thread hijacking problem (which I don't fully understand)?
What to do?
The main two question I'm trying to figure out:
Could mixing await and Wait()/Result be the cause of deadlocks even when running without synchronization context?
Are we running into a bug/limitation in StackExchange.Redis?
A possible fix?
From my debug findings it seems as the problem is that:
next.TryComplete(true);
...on line 162 in CompletionManager.cs could under some circumstances let the current thread (which is the active async worker thread) wander off and start processing other code, possibly causing a deadlock.
Without knowing the details and just thinking about this "fact", then it would seem logical to temporarily release the active async worker thread during the TryComplete invocation.
I guess that something like this could work:
// release the "active thread lock" while invoking the completion action
Interlocked.CompareExchange(ref activeAsyncWorkerThread, 0, currentThread);
try
{
next.TryComplete(true);
Interlocked.Increment(ref completedAsync);
}
finally
{
// try to re-take the "active thread lock" again
if (Interlocked.CompareExchange(ref activeAsyncWorkerThread, currentThread, 0) != 0)
{
break; // someone else took over
}
}
I guess my best hope is that Marc Gravell would read this and provide some feedback :-)
No synchronization context = The default synchronization context
I've written above that our code does not use a synchronization context. This is only partially true: The code is run as either a Console application or as an Azure Worker Role. In these environments SynchronizationContext.Current is null, which is why I wrote that we're running without synchronization context.
However, after reading It's All About the SynchronizationContext I've learned that this is not really the case:
By convention, if a thread’s current SynchronizationContext is null, then it implicitly has a default SynchronizationContext.
The default synchronization context should not be the cause of deadlocks though, as UI-based (WinForms, WPF) synchronization context could - because it does not imply thread affinity.
What I think happens
When a message is completed its completion source is checked for whether it is considered sync safe. If it is, the completion action is executed inline and everything is fine.
If it is not, the idea is to execute the completion action on a newly allocated thread pool thread. This too works just fine when ConnectionMultiplexer.PreserveAsyncOrder is false.
However, when ConnectionMultiplexer.PreserveAsyncOrder is true (the default value), then those thread pool threads will serialize their work using a completion queue and by ensuring that at most one of them is the active async worker thread at any time.
When a thread becomes the active async worker thread it will continue to be that until it have drained the completion queue.
The problem is that the completion action is not sync safe (from above), still it is executed on a thread that must not be blocked as that will prevent other non sync safe messages from being completed.
Notice that other messages that are being completed with a completion action that is sync safe will continue to work just fine, even though the active async worker thread is blocked.
My suggested "fix" (above) would not cause a deadlock in this way, it would however mess with the notion of preserving async completion order.
So maybe the conclusion to make here is that it is not safe to mix await with Result/Wait() when PreserveAsyncOrder is true, no matter whether we are running without synchronization context?
(At least until we can use .NET 4.6 and the new TaskCreationOptions.RunContinuationsAsynchronously, I suppose)

These are the workarounds I've found to this deadlock problem:
Workaround #1
By default StackExchange.Redis will ensure that commands are completed in the same order that result messages are received. This could cause a deadlock as described in this question.
Disable that behavior by setting PreserveAsyncOrder to false.
ConnectionMultiplexer connection = ...;
connection.PreserveAsyncOrder = false;
This will avoid deadlocks and could also improve performance.
I encourage anyone that run into to deadlock problems to try this workaround, since it's so clean and simple.
You'll loose the guarantee that async continuations are invoked in the same order as the underlying Redis operations are completed. However, I don't really see why that is something you would rely on.
Workaround #2
The deadlock occur when the active async worker thread in StackExchange.Redis completes a command and when the completion task is executed inline.
One can prevent a task from being executed inline by using a custom TaskScheduler and ensure that TryExecuteTaskInline returns false.
public class MyScheduler : TaskScheduler
{
public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
return false; // Never allow inlining.
}
// TODO: Rest of TaskScheduler implementation goes here...
}
Implementing a good task scheduler may be a complex task. There are, however, existing implementations in the ParallelExtensionExtras library (NuGet package) that you can use or draw inspiration from.
If your task scheduler would use its own threads (not from the thread pool), then it might be a good idea to allow inlining unless the current thread is from the thread pool. This will work because the active async worker thread in StackExchange.Redis is always a thread pool thread.
public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
// Don't allow inlining on a thread pool thread.
return !Thread.CurrentThread.IsThreadPoolThread && this.TryExecuteTask(task);
}
Another idea would be to attach your scheduler to all of its threads, using thread-local storage.
private static ThreadLocal<TaskScheduler> __attachedScheduler
= new ThreadLocal<TaskScheduler>();
Ensure that this field is assigned when the thread starts running and cleared as it completes:
private void ThreadProc()
{
// Attach scheduler to thread
__attachedScheduler.Value = this;
try
{
// TODO: Actual thread proc goes here...
}
finally
{
// Detach scheduler from thread
__attachedScheduler.Value = null;
}
}
Then you can allow inlining of tasks as long as its done on a thread that is "owned" by the custom scheduler:
public override bool TryExecuteTaskInline(Task task, bool taskWasPreviouslyQueued)
{
// Allow inlining on our own threads.
return __attachedScheduler.Value == this && this.TryExecuteTask(task);
}

I am guessing a lot based on the detailed information above and not knowing the source code you have in place. It sounds like you may be hitting some internal, and configurable, limits in .Net. You shouldn't be hitting those, so my guess is that you are not disposing of objects since they are floating between threads which won't allow you to use a using statement to cleanly handle their object lifetimes.
This details the limitations on HTTP requests. Similar to the old WCF issue when you didn't dispose of the connection and then all WCF connections would fail.
Max number of concurrent HttpWebRequests
This is more of a debugging aid, since I doubt you really are using all the TCP ports, but good info on how to find how many open ports you have and to where.
https://msdn.microsoft.com/en-us/library/aa560610(v=bts.20).aspx

Are a .NET Task thread's resources returned back to the pool temporarily if the thread is waiting on an async operation to finish?

I have a TPL Task that does two things. First, it calls a web service. Second, it inserts some data into a database. I have up to 20 Tasks started at one time doing this same thing over and over again. All they do all day is call web services and insert data into a database.
I'm fairly new to TPL in .NET. I've done some stuff with background worker processes and async web services.
The web service call and the database insert are both blocking calls within the thread the Task is running in.
I understand that under the covers, when you use Tasks, .NET manages a thread pool for you. Yes?
Would the thread pool have more threads at its disposal if I made the service call and database call with async and await() instead of making them blocking calls?
My theory (and I'm not sure why I think this) is that the thread is busy doing nothing while waiting on the blocking web service and can't return its resources temporarily to the pool. But I wonder if the Tasks were waiting for async calls to finish whether the main Task thread would be able to switch to let other stuff process while waiting.
Is my theory right? Or am I making stuff up?
I'm using c# and .NET 4.0, but I could go to 4.5 if needed.

Would the thread pool have more threads at its disposal if I made the
service call and database call with async and await() instead of
making them blocking calls?
It depends on what you mean by "making use of async-await".
When you use Task.Run, behind the scenes, the Task class uses the ThreadPool to offload work using a ThreadPool thread.
If your service doesn't expose a true async api and you uses Task.Run to queue your work, you will still be blocking a threadpool thread to do IO bound work, regardless of the use of async-await. In your question you state that both calls are blocking calls, and in that case the answer is no, the threadpool thread used to make those blocking calls woul still be blocked.
If your service and database calls were true async APIs (one that doesn't consume any extra threads to do its work), you could advantage of async-await, as when you await on one of those calls (and you shouldn't need to use Task.Run with them at all), the current thread will yield control back to the caller, and can be used in the meanwhile to do more work. If this is the case, then yes.
My theory (and I'm not sure why I think this) is that the thread is busy doing nothing while waiting on the blocking web service and can't return its resources temporarily to the pool. But I wonder if the Tasks were waiting for async calls to finish whether the main Task thread would be able to switch to let other stuff process while waiting.
Your theory is correct. If the main job of the queued threadpool work is to make an IO bound request then its spending of most its time simply blocking until the request finishes.
When you await a Task, control yields back to caller. Lets assume your service call was a REST call, you could use HttpClient which exposes true non-thread consuming async methods such as GetAsync, PostAsync, and when you await these calls, your calling thread is released to do more work in the meanwhile.

If all an application's tasks block, each task will use a thread from the thread pool.
If all tasks regularly await, the thread pool does not need to use a thread for every task.
When your code awaits an operation that hasn't completed yet, the method's state is saved such that it can be resumed on any other thread.
Idle threadpool threads get released after a while, so the actual thread which hits an await can be released from the thread pool while the method calling await is still running.
Putting all this together, an async version of a routine can do the same work with less threads (assuming the workload has enough balance of time awaiting vs spinning the CPU).
This code runs 100 tasks doing a synchronous wait:
var numTasks = 100;
for (int i = 0; i < numTasks; i++)
{
Thread.Sleep(5);
Task.Run(() =>
{
Thread.Sleep(5000);
Interlocked.Decrement(ref numTasks);
});
}
while (numTasks > 0) Thread.Sleep(100);
For the async wait, change it to:
Task.Run(async () =>
{
await Task.Delay(5000);
Interlocked.Decrement(ref numTasks);
});
On my system the async version grows the peak thread count half as much, and takes 20% of the time to do the same 'work'.

The answer is yes. Although technically it's not "waiting" for the async operation to complete (otherwise there would be no benefit to async). Under the hood there is a callback delegate that is run when the async operation completes, which is what allows your calling thread to proceed without blocking. It's the async/await magic that turns these 'continuations' into a linear looking piece of code.
Because you are using a threadpool thread, when it hits an await the thread will return to the threadpool. The thing to be careful with here is that the normal behaviour is when the awaited operation completes it will try to get back onto the thread that it was started on (now probably being used by another Task) so you may observe latency problems in getting the results back as threadpool threads are now tied up starting other tasks. Over time the threadpool will try to adjust the number of available threads to meet demand but you might find this doesn't happen quickly enough if you work comes in bursts. The result will be apparently poor performance as you may only have a small number of threads available.

Can .NET Task instances go out of scope during run?

If I have the following block of code in a method (using .NET 4 and the Task Parallel Library):
var task = new Task(() => DoSomethingLongRunning());
task.Start();
and the method returns, will that task go out of scope and be garbage collected, or will it run to completion? I haven't noticed any issues with GCing, but want to make sure I'm not setting myself up for a race condition with the GC.

Update:
After I answered this question (a long time ago!) I found out that it's not true that Tasks will always run to completion - there's a small, let's say "corner" case, where tasks may not finish.
The reason for that is this: As I have answered previously, Tasks are essentially threads; but they are background threads. Background threads are automatically aborted when all foreground threads finish. So, if you don't do anything with the task and the program ends, there's a chance the task won't complete.
You should always await on tasks. More information can be found on the excellent answer Jon gave me.
Original:
Task are scheduled to the ThreadPool, meaning that they are essentially threads¹ (actually, they encapsulate threads).
From the Thread documentation:
It is not necessary to retain a
reference to a Thread object once you
have started the thread. The thread
continues to execute until the thread
procedure is complete.
So, no, there is no need to retain a reference to it.
Also, the documentation states that the preferred way to create a Task is to use it's factory:
You can also use the StartNew method
to create and start a task in one
operation. This is the preferred way
to create and start tasks if creation
and scheduling do not have to be
separated (...)
Hope it helps.
¹ Accordingly to the documentation:
A task represents an asynchronous
operation, and in some ways it
resembles the creation of a new thread
or ThreadPool work item, but at a
higher level of abstraction.

The task will run to completion. Even if there aren't any other references to it (not being rooted I believe is the term), the thread pool will still hold a reference to it, and prevent it from being Garbage Collected at least (I say at least, because even after it completes, there is no guarantee that it will be Garbage Collected) until completion.

How do I abort CCR threads\tasks?

I want to implement a timeout on the execution of tasks in a project that uses the CCR. Basically when I post an item to a Port or enqueue a Task to a DispatcherQueue I want to be able to abort the task or the thread that its running on if it takes longer than some configured time. How can I do this?

Can you confirm what you are asking? Are you running a long-lived task in the Dispatcher? Killing the thread would break the CCR model, so you need to be able to signal to the thread to finish its work and yield. Assuming it's a loop that is not finishing quick enough, you might choose to enqueue a timer:
var resultTimeoutPort = new Port<DateTime>();
dispatcherQueue.EnqueueTimer(TimeSpan.FromSeconds(RESULT_TIMEOUT),
resultTimeoutPort);
and ensure the blocking thread has available a reference to resultTimeoutPort. In the blocking loop, one of the exit conditions might be:
do
{
//foomungus amount of work
}while(resultTimeoutPort.Test()==null&&
someOtherCondition)
Please post more info if I'm barking up the wrong tree.

You could register the thread (Thread.CurrentThread) at the beginning of your CCR "Receive" handler (or in a method that calls your method via a delegate). Then you can do your periodic check and abort if necessary basically the same way you would have done it if you created the thread manually. The catch is that if you use your own Microsoft.Ccr.Core.Dispatcher with a fixed number of threads, I don't think there is a way to get those threads back once you abort them (based on my testing). So, if your dispatcher has 5 threads, you'll only be able to abort 5 times before posting will no longer work regardless of what tasks have been registered. However, if you construct a DispatcherQueue using the CLR thread pool, any CCR threads you abort will be replaced automatically and you won't have that problem. From what I've seen, although the CCR dispatcher is recommended, I think using the CLR thread pool is the way to go in this situation.