We Keep Coding

Why do I seem to have so few threads

I am trying to understand some code (for performance reasons) that is processing tasks from a queue. The code is C# .NET Framework 4.8 (And I didn't write this stuff)
I have this code creating a timer that from what I can tell should use a new thread every 10 seconds
_myTimer = new Timer(new TimerCallback(OnTimerGo), null, 0, 10000 );
Inside the onTimerGo it calls DoTask() inside of DoTask() it grabs a task off a queue and then does this
System.Threading.Tasks.Task.Factory.StartNew(ProcessTask, task).ContinueWith(c => DoTask());
My reading of this is that a new thread should start running OnTimerGo every 10 seconds, and that thread should in parralel run ProcessTask on tasks as fast as it can get them from the queue.
I inserted some code to call ThreadPool.GetMaxThreads and ThreadPool.GetAvailableThreads to figure out how many threads were in use. Then I queued up 10,000 things for it to do and let it loose.
I never see more then 4 threads in use at a time. This is running on a c4.4xlarge ec2 instance... so 16 vCPU 30 gb mem. The get max and available return over 2k. So I would expect more threads. By looking at the logging I can see that a total of 50ish different threads (by thread id) end up doing the work over the course of 20 minutes. Since the timer is set to every 10 seconds, I would expect 100 threads to be doing the work (or for it to finish sooner).
Looking at the code, the only time a running thread should stop is if it asks for a task from the queue and doesn't get one. Some other logging shows that there are never more than 2 tasks running in a thread. This is probably because they work is pretty fast. So the threads shouldn't be exiting, and I can even see from the logs that many of them end up doing as many as 500 tasks over the 20 minutes.
so... what am I missing here. Are the ThreadPool.GetMaxThreads and ThreadPool.GetAvailableThreads not accurate if run from inside a thread? Is something shutting down some of the threads while letting others keep going?
EDIT: adding more code
public static void StartScheduler()
{
lock (TimerLock)
{
if (_timerShutdown == false)
{
_myTimer = new Timer(new TimerCallback(OnTimerGo), null, 0, 10 );
const int numberOfSecondsPerMinute = 60;
const int margin = 1;
var pollEventsPerMinute = (numberOfSecondsPerMinute/SystemPreferences.TaskPollingIntervalSeconds);
_numberOfTimerCallsForHeartbeat = pollEventsPerMinute - margin;
}
}
}
private static void OnTimerGo(object state)
{
try
{
_lastTimer = DateTime.UtcNow;
var currentTickCount = Interlocked.Increment(ref _timerCallCount);
if (currentTickCount == _numberOfTimerCallsForHeartbeat)
{
Interlocked.Exchange(ref _timerCallCount, 0);
MonitoringTools.SendHeartbeatMetric(Heartbeat);
}
CheckForTasks();
}
catch (Exception e)
{
Log.Warn("Scheduler: OnTimerGo exception", e);
}
}
public static void CheckForTasks()
{
try
{
if (DoTask())
_lastStart = DateTime.UtcNow;
_lastStartOrCheck = DateTime.UtcNow;
}
catch (Exception e)
{
Log.Error("Unexpected exception checking for tasks", e);
}
}
private static bool DoTask()
{
Func<DataContext, bool> a = db =>
{
var mtid = Thread.CurrentThread.ManagedThreadId;
int totalThreads = Process.GetCurrentProcess().Threads.Count;
int maxWorkerThreads;
int maxPortThreads;
ThreadPool.GetMaxThreads(out maxWorkerThreads, out maxPortThreads);
int AvailableWorkerThreads;
int AvailablePortThreads;
ThreadPool.GetAvailableThreads(out AvailableWorkerThreads, out AvailablePortThreads);
int usedWorkerThreads = maxWorkerThreads - AvailableWorkerThreads;
string usedThreadMessage = $"Thread {mtid}: Threads in Use count: {usedWorkerThreads}";
Log.Info(usedThreadMessage);
var taskTypeAndTasks = GetTaskListTypeAndTasks();
var task = GetNextTask(db, taskTypeAndTasks.Key, taskTypeAndTasks.Value);
if (_timerShutdown)
{
Log.Debug("Task processing stopped.");
return false;
}
if (task == null)
{
Log.DebugFormat("DoTask: Idle in thread {0} ({1} tasks running)", mtid, _processingTaskLock);
return false;
}
Log.DebugFormat("DoTask: starting task {2}:{0} on thread {1}", task.Id, mtid, task.Class);
System.Threading.Tasks.Task.Factory.StartNew(ProcessTask, task).ContinueWith(c => DoTask());
Log.DebugFormat("DoTask: done ({0})", mtid);
return true;
};
return DbExtensions.WithDbWrite(ctx => a(ctx));
}

The Task.Factory.StartNew by default doesn't create a new thread. It borrows a thread from the ThreadPool instead.
The ThreadPool is intended as a small pool of reusable threads, to help amortize the cost of running frequent and lightweight operations like callbacks, continuations, event handers etc. Depleting the ThreadPool from available workers by scheduling too much work on it, results in a situation that is called saturation or starvation. And as you've already figured out, it's not a happy situation to be.
You can prevent the saturation of the ThreadPool by running your long-running work on dedicated threads instead of ThreadPool threads. This can be done by passing the TaskCreationOptions.LongRunning as argument to the Task.Factory.StartNew:
_ = Task.Factory.StartNew(ProcessTask, task, CancellationToken.None,
TaskCreationOptions.LongRunning,
TaskScheduler.Default).ContinueWith(t => DoTask(), CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
The above code schedules the ProcessTask(task) on a new thread, and after the invocation is completed either successfully or unsuccessfully, the DoTask will be invoked on the same thread. Finally the thread will be terminated. The discard _ signifies that the continuation Task (the task returned by the ContinueWith) is fire-and-forget. Which, to put it mildly, is architecturally suspicious. 😃
In case you are wondering why I pass the TaskScheduler.Default explicitly as argument to StartNew and ContinueWith, check out this link.

My reading of this is that a new thread should start running OnTimerGo every 10 seconds, and that thread should in parralel run ProcessTask on tasks as fast as it can get them from the queue.
Well, that is definitely not what's happening. It's a lot of uncertainty about your code, but it's clear that another DoTask is starting AFTER ProcessTask completes. And that is not parallel execution. Your line of code is this
System.Threading.Tasks.Task.Factory.StartNew(ProcessTask, task).ContinueWith(c => DoTask());
I suggest you to start another DoTask right there like this:
System.Threading.Tasks.Task.Factory.StartNew(ProcessTask, task);
DoTask();
Make sure your code is ready for parallel execution, though.

Multiple parallel Tasks in C# do not improve calculation time

I have a complicated math problem to solve and I decided to do some independent calculations in parallel to improve calculation time. In many CAE programs, like ANSYS or SolidWorks, it is possible to set multiple cores for that purpose.
I created a simple Windows Form example to illustrate my problem. Here the function CalculateStuff() raises A from Sample class in power 1.2 max times. For 2 tasks it's max / 2 times and for 4 tasks it's max / 4 times.
I calculated the resulting time of operation both for only one CalculateStuff() function or four duplicates (CalculateStuff1(), ...2(), ...3(), ...4() - one for each task) with the same code. I'm not sure, if it matters to use the same function for each task (anyway, Math.Pow is the same). I also tried to enable or disable the ProgressBar.
The table represents time of operation (sec) for all 12 cases. I expected it to be like 2 and 4 times faster for 2 and 4 tasks, but in some cases 4 tasks are even worse than 1. My computer has 2 processors, 10 cores each. According to Debug window, CPU usage increases with more tasks. What's wrong with my code here or do I misunderstand something? Why multiple tasks do not improve time of operation?
private readonly ulong max = 400000000ul;
// Sample class
private class Sample
{
public double A { get; set; } = 1.0;
}
// Clear WinForm elements
private void Clear()
{
PBar1.Value = PBar2.Value = PBar3.Value = PBar4.Value = 0;
TextBox.Text = "";
}
// Button that launches 1 task
private async void BThr1_Click(object sender, EventArgs e)
{
Clear();
DateTime start = DateTime.Now;
Sample sample = new Sample();
await Task.Delay(100);
Task t = Task.Run(() => CalculateStuff(sample, PBar1, max));
await t;
TextBox.Text = (DateTime.Now - start).ToString(#"hh\:mm\:ss");
t.Dispose();
}
// Button that launches 2 tasks
private async void BThr2_Click(object sender, EventArgs e)
{
Clear();
DateTime start = DateTime.Now;
Sample sample1 = new Sample();
Sample sample2 = new Sample();
await Task.Delay(100);
Task t1 = Task.Run(() => CalculateStuff(sample1, PBar1, max / 2));
Task t2 = Task.Run(() => CalculateStuff(sample2, PBar2, max / 2));
await t1; await t2;
TextBox.Text = (DateTime.Now - start).ToString(#"hh\:mm\:ss");
t1.Dispose(); t2.Dispose();
}
// Button that launches 4 tasks
private async void BThr4_Click(object sender, EventArgs e)
{
Clear();
DateTime start = DateTime.Now;
Sample sample1 = new Sample();
Sample sample2 = new Sample();
Sample sample3 = new Sample();
Sample sample4 = new Sample();
await Task.Delay(100);
Task t1 = Task.Run(() => CalculateStuff(sample1, PBar1, max / 4));
Task t2 = Task.Run(() => CalculateStuff(sample2, PBar2, max / 4));
Task t3 = Task.Run(() => CalculateStuff(sample3, PBar3, max / 4));
Task t4 = Task.Run(() => CalculateStuff(sample4, PBar4, max / 4));
await t1; await t2; await t3; await t4;
TextBox.Text = (DateTime.Now - start).ToString(#"hh\:mm\:ss");
t1.Dispose(); t2.Dispose(); t3.Dispose(); t4.Dispose();
}
// Calculate some math stuff
private static void CalculateStuff(Sample s, ProgressBar pb, ulong max)
{
ulong c = max / (ulong)pb.Maximum;
for (ulong i = 1; i <= max; i++)
{
s.A = Math.Pow(s.A, 1.2);
if (i % c == 0)
pb.Invoke(new Action(() => pb.Value = (int)(i / c)));
}
}

Tasks are not threads. "Asynchronous" does not mean "simultaneous".
What's wrong with my code here or do I misunderstand something?
You're misunderstanding what tasks are.
You should think of tasks as something that you can do in any order you desire. Take the example of a cooking recipe:
Cut the potatoes
Cut the vegetables
Cut the meat
If these were not tasks and it was synchronous code, you would always do these steps in the exact order they were listed.
If they were tasks, that doesn't mean these jobs will be done simultaneously. You are only one person (= one thread), and you can only do one thing at a time.
You can do the tasks in any order you like, you can possibly even halt one task to begin on another, but you still can't do more than one thing at the same time. Regardless of the order in which you complete the tasks, the total time taken to complete all three tasks remains the same, and this is not (inherently) any faster.
If they were threads, that's like hiring 3 chefs, which means these jobs can be done simultaneously.
Asynchronicity does cut down on idling time, when it is awaitable.
Do note that asynchronous code can lead to time gains in cases where your synchronous code would otherwise be idling, e.g. waiting for a network response. This is not taken into account in the above example, which is exactly why I listed "cut [x]" jobs rather than "wait for [x] to boil".
Your job (the calculation) is not asynchronous code. It never idles (in a way that it's awaitable) and therefore it runs synchronously. This means you're not getting any benefit from running this asynchronously.
Reducing your code to a simpler example:
private static void CalculateStuff(Sample s, ProgressBar pb, ulong max)
{
Thread.Sleep(5000);
}
Very simply put, this job takes 5 seconds and cannot be awaited. If you run 3 of these tasks at the same time, they will still be handled one after the other, taking 15 seconds total.
If the job inside your tasks were actually awaitable, you would see a time benefit. E.g.:
private static async void CalculateStuff(Sample s, ProgressBar pb, ulong max)
{
await Task.Delay(5000);
}
This job takes 5 seconds but is awaitable. If you run 3 of these tasks at the same time, your thread will not waste time idling (i.e. waiting for the delay) and will instead start on the following task. Since it can await (i.e. do nothing for) these tasks at the same time, this means that the total processing time takes 5 seconds total (plus some negligible overhead cost).
According to Debug window, CPU usage increases with more tasks.
The managing of tasks takes a small overhead cost, which means that the total amounts of work (which can be measured in CPU usage over time) is slightly higher compared to synchronous code. That is to be expected.
This small cost usually pales in comparison to the benefits gained from well written asynchronous code. However, your code is simply not leveraging the actual benefits from asynchronicity, so you're only seeing the overhead cost and not its benefits, which is why your monitoring is giving you the opposite result of what you were expecting.
My computer has 2 processors, 10 cores each.
CPU cores, threads and tasks are three very different beasts.
Tasks are handled by threads, but they don't necessarily have a one-to-one mapping. Take the example of a team of 4 developers which has 10 bugs to resolve. While this means it's impossible for all 10 bugs to be resolved at the same time, these developers (threads) can take on the tickets (tasks) one after the other, taking on a new ticket (task) whenever they finished their previous ticket (task).
CPU cores are like workstations. It makes little sense to have less workstations (CPU cores) than you have developers (threads), since you'll end up with idling developers.
Additionally, you might not want your developers to be able to claim all workstations. Maybe HR and accounting (= other OS processes) also need to have some guaranteed workstations so they can do their job.
The company (= computer) doesn't just grind to a halt because the developers are fixing some bugs. This is what used to happen on single core machines - if one process claims the CPU, nothing else can happen. If that one process takes long or hangs, everything freezes.
This is why we have a thread pool. There is no straightforward real-world-analogy here (unless maybe a consultancy firm that dynamically adjusts how many developers it sends to your company), but the thread pool is basically able to decide how many developers are allowed to work at the company at the same time in order to ensure that development tasks can be seen to as fast as possible while also ensuring other departments can still do their work on the workstations as well.
It's a careful balancing act, not sending too many developers as that floods the systems, while also not sending too few developers as that means the work gets done too slowly.
The exact configuration of your threadpool is not something I can troubleshoot over a simple Q&A. But the behavior you describe is consistent with having less CPUs (dedicated to your runtime) and/or threads compared to how many tasks you have.

There are a lot of possible reasons that you might not see the performance gains you're expecting, including things like what else your machine's cores are getting used for at the moment. Running this trimmed-down version of your code, I am able to see a marked improvement when running parallel:
private IEnumerable<Sample> CalculateMany(int n)
{
return Enumerable.Range(0, n)
.AsParallel() // comment this to remove parallelism
.Select(i => { var s = new Sample(); CalculateStuff(s, max / (ulong)n); return s; })
.ToList();
}
// Calculate some math stuff
private static void CalculateStuff(Sample s, ulong max)
{
for (ulong i = 1; i <= max; i++)
{
s.A = Math.Pow(s.A, 1.2);
}
}
Here's running CalculateMany with n values as 1, 2, and 4:
Here's what I get if not using parallelism:
I see similar results using Task.Run():
private IEnumerable<Sample> CalculateMany(int n)
{
var tasks =
Enumerable.Range(0, n)
.Select(i => Task.Run(() => { var s = new Sample(); CalculateStuff(s, max / (ulong)n); return s; }))
.ToArray() ;
Task.WaitAll(tasks);
return tasks
.Select(t => t.Result)
.ToList();
}

Unfortunately I can not give you a reason other than probably something with state machine magic that is happening but this significally increases performance:
private async void BThr4_Click(object sender, EventArgs e)
{
Clear();
DateTime start = DateTime.Now;
await Task.Delay(100);
Task<Sample> t1 = Task<Sample>.Run(() => CalculateStuff(PBar1, max / 4));
Task<Sample> t2 = Task<Sample>.Run(() => CalculateStuff(PBar2, max / 4));
Task<Sample> t3 = Task<Sample>.Run(() => CalculateStuff(PBar3, max / 4));
Task<Sample> t4 = Task<Sample>.Run(() => CalculateStuff(PBar4, max / 4));
Sample sample1 = await t1;
Sample sample2 = await t2;
Sample sample3 = await t3;
Sample sample4 = await t4;
TextBox.Text = (DateTime.Now - start).ToString(#"hh\:mm\:ss");
t1.Dispose(); t2.Dispose(); t3.Dispose(); t4.Dispose();
}
// Calculate some math stuff
private static Sample CalculateStuff(ProgressBar pb, ulong max)
{
Sample s = new Sample();
ulong c = max / (ulong)pb.Maximum;
for (ulong i = 1; i <= max; i++)
{
s.A = Math.Pow(s.A, 1.2);
if (i % c == 0)
pb.Invoke(new Action(() => pb.Value = (int)(i / c)));
}
return s;
}
This way you are not keeping Sample instances that the tasks have to access in the calling function but you create the instances within the task and then just return them to the caller after the task has completed.

Thread Local Storage working principle

This is an example about Thread Local Storage (TLS) from Apress parallel programming book. I know that if we have 4 cores computer 4 thread can run parallel in same time. In this example we create 10 task and we suppose that have 4 cores computer. Each Thread local storage live in on thread so when start 10 task parallel only 4 thread perform. And We have 4 TLS so 10 task try to change 4 Thread local storage object. i want to ask how Tls prevent data race problem when thread count < Task count ??
using System;
using System.Threading;
using System.Threading.Tasks;
namespace Listing_04
{
class BankAccount
{
public int Balance
{
get;
set;
}
}
class Listing_04
{
static void Main(string[] args)
{
// create the bank account instance
BankAccount account = new BankAccount();
// create an array of tasks
Task<int>[] tasks = new Task<int>[10];
// create the thread local storage
ThreadLocal<int> tls = new ThreadLocal<int>();
for (int i = 0; i < 10; i++)
{
// create a new task
tasks[i] = new Task<int>((stateObject) =>
{
// get the state object and use it
// to set the TLS data
tls.Value = (int)stateObject;
// enter a loop for 1000 balance updates
for (int j = 0; j < 1000; j++)
{
// update the TLS balance
tls.Value++;
}
// return the updated balance
return tls.Value;
}, account.Balance);
// start the new task
tasks[i].Start();
}
// get the result from each task and add it to
// the balance
for (int i = 0; i < 10; i++)
{
account.Balance += tasks[i].Result;
}
// write out the counter value
Console.WriteLine("Expected value {0}, Balance: {1}",
10000, account.Balance);
// wait for input before exiting
Console.WriteLine("Press enter to finish");
Console.ReadLine();
}
}
}

We have 4 TLS so 10 task try to change 4 Thread local storage object
In your example, you could have anywhere between 1 and 10 TLS slots. This is because a) you are not managing your threads explicitly and so the tasks are executed using the thread pool, and b) the thread pool creates and destroys threads over time according to demand.
A loop of only 1000 iterations will completely almost instantaneously. So it's likely all ten of your tasks will get through the thread pool before the thread pool decides a work item has been waiting long enough to justify adding any new threads. But there is no guarantee of this.
Some important parts of the documentation include these statements:
By default, the minimum number of threads is set to the number of processors on a system
and
When demand is low, the actual number of thread pool threads can fall below the minimum values.
In other words, on your four-core system, the default minimum number of threads is four, but the actual number of threads active in the thread pool could in fact be less than that. And if the tasks take long enough to execute, the number of active threads could rise above that.
The biggest thing to keep in mind here is that using TLS in the context of a thread pool is almost certainly the wrong thing to do.
You use TLS when you have control over the threads, and you want a thread to be able to maintain some data private or unique to that thread. That's the opposite of what happens when you are using the thread pool. Even in the simplest case, multiple tasks can use the same thread, and so would wind up sharing TLS. And in more complicated scenarios, such as when using await, a single task could wind up executed in different threads, and so that one task could wind up using different TLS values depending on what thread is assigned to that task at that moment.
how Tls prevent data race problem when thread count < Task count ??
That depends on what "data race problem" you're talking about.
The fact is, the code you posted is filled with problems that are at the very least odd, if not outright wrong. For example, you are passing account.Balance as the initial value for each task. But why? This value is evaluated when you create the task, before it could ever be modified later, so what's the point of passing it?
And if you thought you were passing whatever the current value is when the task starts, that seems like that would be wrong too. Why would it be valid to make the starting value for a given task vary according to how many tasks had already completed and been accounted for in your later loop? (To be clear: that's not what's happening…but even if it were, it'd be a strange thing to do.)
Beyond all that, it's not clear what you thought using TLS here would accomplish anyway. When each task starts, you reinitialize the TLS value to 0 (i.e. the value of account.Balance that you've passed to the Task<int> constructor). So no thread involved ever sees a value other than 0 during the context of executing any given task. A local variable would accomplish exactly the same thing, without the overhead of TLS and without confusing anyone who reads the code and tries to figure out why TLS was used when it adds no value to the code.
So, does TLS solve some sort of "data race problem"? Not in this example, it doesn't appear to. So asking how it does that is impossible to answer. It doesn't do that, so there is no "how".
For what it's worth, I modified your example slightly so that it would report the individual threads that were assigned to the tasks. I found that on my machine, the number of threads used varied between two and eight. This is consistent with my eight-core machine, with the variation due to how much the first thread in the pool can get done before the pool has initialized additional threads and assigned tasks to them. Most commonly, I would see the first thread completing between three and five of the tasks, with the remaining tasks handled by remaining individual threads.
In each case, the thread pool created eight threads as soon as the tasks were started. But most of the time, at least one of those threads wound up unused, because the other threads were able to complete the tasks before the pool was saturated. That is, there is overhead in the thread pool just managing the tasks, and in your example the tasks are so inexpensive that this overhead allows one or more thread pool threads to finish one task before the thread pool needs that thread for another.
I've copied that version below. Note that I also added a delay between trial iterations, to allow the thread pool to terminate the threads it created (on my machine, this took 20 seconds, hence the delay time hard-coded…you can see the threads being terminated in the debugger output).
static void Main(string[] args)
{
while (_PromptContinue())
{
// create the bank account instance
BankAccount account = new BankAccount();
// create an array of tasks
Task<int>[] tasks = new Task<int>[10];
// create the thread local storage
ThreadLocal<int> tlsBalance = new ThreadLocal<int>();
ThreadLocal<(int Id, int Count)> tlsIds = new ThreadLocal<(int, int)>(
() => (Thread.CurrentThread.ManagedThreadId, 0), true);
for (int i = 0; i < 10; i++)
{
int k = i;
// create a new task
tasks[i] = new Task<int>((stateObject) =>
{
// get the state object and use it
// to set the TLS data
tlsBalance.Value = (int)stateObject;
(int id, int count) = tlsIds.Value;
tlsIds.Value = (id, count + 1);
Console.WriteLine($"task {k}: thread {id}, initial value {tlsBalance.Value}");
// enter a loop for 1000 balance updates
for (int j = 0; j < 1000; j++)
{
// update the TLS balance
tlsBalance.Value++;
}
// return the updated balance
return tlsBalance.Value;
}, account.Balance);
// start the new task
tasks[i].Start();
}
// Make sure this thread isn't busy at all while the thread pool threads are working
Task.WaitAll(tasks);
// get the result from each task and add it to
// the balance
for (int i = 0; i < 10; i++)
{
account.Balance += tasks[i].Result;
}
// write out the counter value
Console.WriteLine("Expected value {0}, Balance: {1}", 10000, account.Balance);
Console.WriteLine("{0} thread ids used: {1}",
tlsIds.Values.Count,
string.Join(", ", tlsIds.Values.Select(t => $"{t.Id} ({t.Count})")));
System.Diagnostics.Debug.WriteLine("done!");
_Countdown(TimeSpan.FromSeconds(20));
}
}
private static void _Countdown(TimeSpan delay)
{
System.Diagnostics.Stopwatch sw = System.Diagnostics.Stopwatch.StartNew();
TimeSpan remaining = delay - sw.Elapsed,
sleepMax = TimeSpan.FromMilliseconds(250);
int cchMax = $"{delay.TotalSeconds,2:0}".Length;
string format = $"\r{{0,{cchMax}:0}}", previousText = null;
while (remaining > TimeSpan.Zero)
{
string nextText = string.Format(format, remaining.TotalSeconds);
if (previousText != nextText)
{
Console.Write(format, remaining.TotalSeconds);
previousText = nextText;
}
Thread.Sleep(remaining > sleepMax ? sleepMax : remaining);
remaining = delay - sw.Elapsed;
}
Console.Write(new string(' ', cchMax));
Console.Write('\r');
}
private static bool _PromptContinue()
{
Console.Write("Press Esc to exit, any other key to proceed: ");
try
{
return Console.ReadKey(true).Key != ConsoleKey.Escape;
}
finally
{
Console.WriteLine();
}
}

Load Test using C# Async Await

I am creating a console program, which can test read / write to a Cache by simulating multiple clients, and have written following code. Please help me understand:
Is it correct way to achieve the multi client simulation
What can I do more to make it a genuine load test
void Main()
{
List<Task<long>> taskList = new List<Task<long>>();
for (int i = 0; i < 500; i++)
{
taskList.Add(TestAsync());
}
Task.WaitAll(taskList.ToArray());
long averageTime = taskList.Average(t => t.Result);
}
public static async Task<long> TestAsync()
{
// Returns the total time taken using Stop Watch in the same module
return await Task.Factory.StartNew(() => // Call Cache Read / Write);
}

Adjusted your code slightly to see how many threads we have at a particular time.
static volatile int currentExecutionCount = 0;
static void Main(string[] args)
{
List<Task<long>> taskList = new List<Task<long>>();
var timer = new Timer(Print, null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1));
for (int i = 0; i < 1000; i++)
{
taskList.Add(DoMagic());
}
Task.WaitAll(taskList.ToArray());
timer.Change(Timeout.Infinite, Timeout.Infinite);
timer = null;
//to check that we have all the threads executed
Console.WriteLine("Done " + taskList.Sum(t => t.Result));
Console.ReadLine();
}
static void Print(object state)
{
Console.WriteLine(currentExecutionCount);
}
static async Task<long> DoMagic()
{
return await Task.Factory.StartNew(() =>
{
Interlocked.Increment(ref currentExecutionCount);
//place your code here
Thread.Sleep(TimeSpan.FromMilliseconds(1000));
Interlocked.Decrement(ref currentExecutionCount);
return 4;
}
//this thing should give a hint to scheduller to use new threads and not scheduled
, TaskCreationOptions.LongRunning
);
}
The result is: inside a virtual machine I have from 2 to 10 threads running simultaneously if I don't use the hint. With the hint — up to 100. And on real machine I can see 1000 threads at once. Process explorer confirms this. Some details on the hint that would be helpful.

If it is very busy, then apparently your clients have to wait a while before their requests are serviced. Your program does not measure this, because your stopwatch starts running when the service request starts.
If you also want to measure what happen with the average time before a request is finished, you should start your stopwatch when the request is made, not when the request is serviced.
Your program takes only threads from the thread pool. If you start more tasks then there are threads, some tasks will have to wait before TestAsync starts running. This wait time would be measured if you remember the time Task.Run is called.
Besides the flaw in time measurements, how many service requests do you expect simultaneously? Are there enough free threads in your thread pool to simulate this? If you expect about 50 service requests at the same time, and the size of your thread pool is only 20 threads, then you'll never run 50 service requests at the same time. Vice versa: if your thread pool is way bigger than your number of expected simultaneous service requests, then you'll measure longer times than are actual the case.
Consider changing the number of threads in your thread pool, and make sure no one else uses any threads of the pool.

Is it a good idea to use longtime Thread.Sleep?

I have a job list. Each job has its own run time. They need to run when it comes time. I think two different ways.
public class Job
{
public int JobPeriod {get;set;} // for example as hour: daily = 24, weekly = 7 * 24, monthly = 30 * 24
public DateTime RunTime {get;set}
}
First Way :
I start a new main thread. This thread checks jobs at certain time interval (5 sec, 10 sec etc.). When a job's run time has come, the main thread will start and finish the job. The main thread which continually run in this way.
while (true)
{
lock (Locker)
{
// checks job list.
var jobs = foo.GetIncomingTimeJobs();
foreach (var job in jobs)
{
ParameterizedThreadStart ts = RunJob;
var th = new Thread(ts);
th.Start(job);
}
Thread.Sleep(10000);
}
}
public void RunJob(Job job)
{
// do somethings
}
Second Way :
When application is started, I create a new thread for each job in the job list. All of these created threads will start. When Job's thread is started, job's thread checks the job's run time.
For example :
var jobs = foo.GetAllJobs();
foreach (var job in jobs)
{
ParameterizedThreadStart ts = RunJob;
var th = new Thread(ts);
th.Start(job);
}
public void RunJob(Job job)
{
while (true)
{
lock (Locker)
{
// do somethings
var period = job.JobPeriod * 60 * 1000;
Thread.Sleep(period);
}
}
}
If there are ten jobs , there will be ten threads. And These ten threads will never end. will sleep, will continue, will sleep, will continue ...
Is it normal for threads to sleep such a long time ? Which way should I use ? Or Is there another way of doing such a thing?

Both approaches are in most cases incorrect. Usual solution for this kind of problems is using System.Threading.Timer. Sample code for your case can look like that:
private void CheckJobs(object state)
{
lock (Locker)
{
// checks job list.
var jobs = foo.GetIncomingTimeJobs();
foreach (var job in jobs)
{
var thread = new Thread(foo);
thread.Start();
}
}
}
private void StartProcessing()
{
var timer = new System.Threading.Timer(CheckJobs, null, 0, 10000);
}
When you call StartProcessing() function, the timer will be initialized and jobs list will be checked every 10 seconds.
If you go with Thread.Sleep() your application will become very unresponsive.