We Keep Coding

Task.Delay vs Thread.Sleep difference

private static async Task MainFunc()
{
var watch = System.Diagnostics.Stopwatch.StartNew();
List<Task<int>> list = new List<Task<int>>();
for (int i = 1; i <= 3; i++)
{
list.Add(TaskFunc(i));
}
var taskResult = await Task.WhenAll(list);
foreach (var item in taskResult)
{
Console.Write($"i= {item}.{ Environment.NewLine }");
}
list.Clear();
watch.Stop();
var elapsedMs1 = watch.ElapsedMilliseconds;
Console.WriteLine($"Total execution time: { elapsedMs1 }");
Console.WriteLine();
watch.Restart();
for (int i = 1; i <= 3; i++)
{
list.Add(Task.Run(() => ThreadFunc(i)));
}
var threadResult = await Task.WhenAll(list);
foreach (var item in threadResult)
{
Console.Write($"i= {item}.{ Environment.NewLine }");
}
watch.Stop();
var elapsedMs2 = watch.ElapsedMilliseconds;
Console.WriteLine($"Total execution time: { elapsedMs2 }");
}
private static async Task<int> TaskFunc(int i)
{
if (i == 1)
await Task.Delay(2000);
else if (i == 2)
await Task.Delay(1000);
else if (i == 3)
await Task.Delay(5000);
return i;
}
private static int ThreadFunc(int i)
{
if (i == 1)
Thread.Sleep(2000);
else if (i == 2)
Thread.Sleep(1000);
else if (i == 3)
Thread.Sleep(5000);
return i;
}
In this example there are two functions, TaskFunc and ThreadFunc, which are called from MainFunc seperately. I am curious as to why the second method doesn't seem to have any effect, and seems to be skipped. Not even the Thread.Sleep(...) seems to get executed.
As you can see, the total execution time for ThreadFunc is very short, even though the sleep timers should be same for both methods. Also, i is always set to 4. I suppose that the main thread is doing something wrong. Can someone explain what is going on here?

The issue here is scoping. The i you are using inside the Task.Run(() => TheadFunc(i)) is not a new integer, but rather, since it's a delegate, the value of i will only be retrieved once the delegate is executed.
This leads to i being 4 in all cases, since your for-loop increases it that much. But since you do not have an if-condition for a value of 4, it won't perform any delays with Thread.Sleep(...).

Task.Delay delays too long

I've created a multi task program. This program has around 20 main tasks and each of them calls some sub tasks to operate file I/Os. I wanted each main task to repeat periodically every 500ms, so I enterd the code Task.Delay(500).
The problem is Task.Delay delays a lot more than 500ms sometimes. There is a case it delays more than 3 seconds.
How can I fix it?
The original progam is so big that I created a sample program below.
(1) If Task.Delay is on, over-delay happens.
(2) If Thead.Sleep is on, over-delay doesn't happen.
ThreadPool.SetMinThreads() doesn't seem to resolve it.
Thanks.
class Program
{
const int DELAY_TIME = 500;
const int TASKS = 100;
const int WAITS = 100;
const int WARNING_THRESHOLD = 100;
static void Main(string[] args)
{
//ThreadPool.SetMinThreads(workerThreads: 200, completionPortThreads: 200);
Console.WriteLine("*** Start...");
Test();
Console.WriteLine("*** Done!");
Console.ReadKey();
}
private static void Test()
{
List<Task> tasks = new List<Task>();
for (int taskId = 0; taskId < TASKS; taskId++)
{
tasks.Add(DelaysAsync(taskId));
}
Task.WaitAll(tasks.ToArray());
}
static async Task DelaysAsync(int taskId)
{
await Task.Yield();
Stopwatch sw = new Stopwatch();
for (int i = 0; i < WAITS; i++)
{
sw.Reset();
sw.Start();
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
sw.Stop();
Console.Write($"Task({taskId})_iter({i}) Elapsed={sw.ElapsedMilliseconds}");
if (sw.ElapsedMilliseconds > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine(" *********** Too late!! ************");
}
else
{
Console.WriteLine();
}
}
}
}

I’ve run your test, with .NET 4.6.1 and VS 2017. Here on Xeon E3-1230 v3 CPU it never printed “Too late”, the Elapsed value was within 498-527 ms.
The Thread.Sleep version performed very similarly, 500-528ms per sleep, however the total execution time was much longer because the runtime refused to create 100 OS threads, that’s way too many, so less than 100 DelaysAsync functions ran in parallel. The debugger showed me there were 27 worker threads in Thread.Sleep version and only 9 worker threads in Task.Delay version.
I think you have other apps on your PC creating too many threads and consuming too much CPU. Windows tries to load balance threads evenly so when the whole system is CPU bound, more native threads = more CPU time and therefore less jitter.
If that’s your case and you want to prioritize your app in the scheduler, instead of using Thread.Sleep and more threads, raise the priority of your process.

It seems that I could find the answer. I changed the previous sample program like below. The main difference is using StopWatch or DateTime to measure time durations.
In StopWatch version, many delays happen.
In DateTime version, no or at least very little delays happen(s).
I guess that the cause is the contention of Timer that is used by both StopWatch and Task.Delay. I concluded that I should not use StopWatch and Task.Delay together.
Thank you.
class Program
{
const int DELAY_TIME = 500;
const int TASKS = 100;
const int WAITS = 100;
const int WARNING_THRESHOLD = 500;
static void Main(string[] args)
{
using (Process p = Process.GetCurrentProcess())
{
p.PriorityClass = ProcessPriorityClass.RealTime;
//ThreadPool.SetMinThreads(workerThreads: 200, completionPortThreads: 200);
int workerThreads;
int completionPortThreads;
ThreadPool.GetAvailableThreads(out workerThreads, out completionPortThreads);
Console.WriteLine($"{workerThreads}, {completionPortThreads}");
Console.WriteLine("*** Start...");
Test();
Console.WriteLine("*** Done!");
Console.ReadKey();
}
}
private static void Test()
{
int totalCount = 0;
List<Task<int>> tasks = new List<Task<int>>();
for (int taskId = 0; taskId < TASKS; taskId++)
{
//tasks.Add(DelaysWithStopWatchAsync(taskId)); // many delays
tasks.Add(DelaysWithDateTimeAsync(taskId)); // no delays
}
Task.WaitAll(tasks.ToArray());
foreach (var task in tasks)
{
totalCount += task.Result;
}
Console.WriteLine($"Total counts of deday = {totalCount}");
}
static async Task<int> DelaysWithStopWatchAsync(int taskId)
{
await Task.Yield();
int count = 0;
Stopwatch sw = new Stopwatch();
for (int i = 0; i < WAITS; i++)
{
sw.Reset();
sw.Start();
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
sw.Stop();
Console.Write($"task({taskId})_iter({i}) elapsed={sw.ElapsedMilliseconds}");
if (sw.ElapsedMilliseconds > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine($" *********** Too late!! ************");
count++;
}
else
{
Console.WriteLine();
}
}
return count;
}
static async Task<int> DelaysWithDateTimeAsync(int taskId)
{
await Task.Yield();
int count = 0;
for (int i = 0; i < WAITS; i++)
{
DateTime start = DateTime.Now;
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
DateTime end = DateTime.Now;
int duration = (end - start).Milliseconds;
Console.Write($"Task({taskId})_iter({i}) Elapsed={duration}");
if (duration > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine($" *********** Too late!! ************");
count++;
}
else
{
Console.WriteLine();
}
}
return count;
}
}

Restart concurrent tasks as soon as they fail for x number of times

I have a console app that is making HTTP queries and adding/updating products in my database according to response. Some fail and need to be retried a few times.
The way I came up with was to use a dictionary to store the product ID and a Task. Then I can check all the task results and re-run.
This is working but it strikes me as inefficient. Tasks are not being re-created until all tasks have finished. It would be more efficient if they were immediately restarted but I can't figure out how to do this. Also every retry involves a query to the database as only the ID is stored.
I made small app that shows how I am currently retrying failed requests.
Can someone suggest a more efficient method for retrying?
class Program
{
private static void Main(string[] args)
{
HttpQuery m = new HttpQuery();
var task = Task.Run(() => m.Start());
Task.WaitAll(task);
Console.WriteLine("Finished");
Console.ReadLine();
}
}
class HttpQuery
{
public async Task Start()
{
// dictionary where key represent reference to something that needs to be processed and bool whether it has completed or not
ConcurrentDictionary<int, Task<bool>> monitor = new ConcurrentDictionary<int, Task<bool>>();
// start async tasks.
Console.WriteLine("starting first try");
for (int i = 0; i < 1000; i++)
{
Console.Write(i+",");
monitor[i] = this.Query(i);
}
// wait for completion
await Task.WhenAll(monitor.Values.ToArray());
Console.WriteLine();
// start retries
// number of retries per query
int retries = 10;
int count = 0;
// check if max retries exceeded or all completed
while (count < retries && monitor.Any(x => x.Value.Result == false))
{
// make list of numbers that failed
List<int> retryList = monitor.Where(x => x.Value.Result == false).Select(x => x.Key).ToList();
Console.WriteLine("starting try number: " + (count+1) + ", Processing: " + retryList.Count);
// create list of tasks to wait for
List<Task<bool>> toWait = new List<Task<bool>>();
foreach (var i in retryList)
{
Console.Write(i + ",");
monitor[i] = this.Query(i);
toWait.Add(monitor[i]);
}
// wait for completion
await Task.WhenAll(toWait.ToArray());
Console.WriteLine();
count++;
}
Console.WriteLine("ended");
Console.ReadLine();
}
public async Task<bool> Query(int i)
{
// simulate a http request that may or may not fail
Random r = new Random();
int delay = i * r.Next(1, 10);
await Task.Delay(delay);
if (r.Next(0,2) == 1)
{
return true;
}
else
{
return false;
}
}
}

You can create another method and wrap all these ugly retry logic. All of that ugly code goes away :)
public async Task Start()
{
const int MaxNumberOfTries = 10;
List<Task<bool>> tasks = new List<Task<bool>>();
for (int i = 0; i < 1000; i++)
{
tasks.Add(this.QueryWithRetry(i, MaxNumberOfTries));
}
await Task.WhenAll(tasks);
}
public async Task<bool> QueryWithRetry(int i, int numOfTries)
{
int tries = 0;
bool result;
do
{
result = await Query(i);
tries++;
} while (!result && tries < numOfTries);
return result;
}

Continuation Task does not execute when Task is cancelled

The idea: create a Task that prints an increasing number of asterisks. When the user presses Enter, the Task prints 10 asterisks and then stops.
The code:
namespace CancellingLongRunningTasks
{
using System;
using System.Threading;
using System.Threading.Tasks;
class Program
{
static void Main()
{
var cancellationTokenSource = new CancellationTokenSource();
var token = cancellationTokenSource.Token;
Task task = Task.Run(() =>
{
int count = 1;
while (!token.IsCancellationRequested)
{
Console.WriteLine(new string('*', count));
Thread.Sleep(1000);
count++;
}
}, token).ContinueWith(
parent =>
{
var count = 10;
while (count > 0)
{
Console.WriteLine(new string('*', count));
Thread.Sleep(1000);
count--;
}
}, TaskContinuationOptions.OnlyOnCanceled);
Console.WriteLine("Press enter to stop the task.");
if (Console.ReadLine().Contains(Environment.NewLine))
{
cancellationTokenSource.Cancel();
task.Wait();
}
}
}
}
The question: why isn't my continuation task executed?

It isn't being executed because you're not actively cancelling the Task, only checking if a cancellation was requested. Use CancellationToken.ThrowIfCancellationRequested which throws an OperationCanceledException and will transform the Task into a cancelled state or simply throw the exception with the corresponding token:
Task task = Task.Run(() =>
{
int count = 1;
while (!token.IsCancellationRequested)
{
Console.WriteLine(new string('*', count));
Thread.Sleep(1000);
count++;
}
token.ThrowIfCancellationRequested();
}, token)
Which is equivalent to:
Task task = Task.Run(() =>
{
int count = 1;
while (!token.IsCancellationRequested)
{
Console.WriteLine(new string('*', count));
Thread.Sleep(1000);
count++;
}
throw new OperationCanceledException(token);
}, token)

Task synchronization without a UI thread

In the code below I want to syncronize the reporting of the results of a list of tasks. This is working now because task.Result blocks until the task completes. However, task id = 3 takes a long time to complete and blocks all of the other finished tasks from reporting their status.
I think that I can do this by moving the reporting (Console.Write) into a .ContinueWith instruction but I don't have a UI thread so how do I get a TaskScheduler to syncronize the .ContinueWith tasks?
What I have now:
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
var tasks = new List<Task<int>>();
for (var i = 0; i < 10; i++)
{
var num = i;
var t = Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 5000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
});
tasks.Add(t);
}
foreach (var task in tasks)
{
Console.WriteLine("Completed {0} on {1}", task.Result, Thread.CurrentThread.ManagedThreadId);
}
Console.WriteLine("End of Main");
Console.ReadKey();
}
I would like to move to this or something similar but I need the Console.Write("Completed...") to all happen on the same thread:
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
for (var i = 0; i < 10; i++)
{
var num = i;
Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
}).ContinueWith(value =>
{
Console.WriteLine("Completed {0} on {1}", value.Result, Thread.CurrentThread.ManagedThreadId);
}
/* need syncronization context */);
}
Console.WriteLine("End of Main");
Console.ReadKey();
}
-- SOLUTION --
After getting some comments and reading some of the solutions this is the complete solution that does what I want. The goal here is to process severl long running tasks as fast as possible and then do something with the results of each task one at a time.
static void Main(string[] args)
{
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
var results = new BlockingCollection<int>();
Task.Factory.StartNew(() =>
{
while (!results.IsCompleted)
{
try
{
var x = results.Take();
Console.WriteLine("Completed {0} on {1}", x, Thread.CurrentThread.ManagedThreadId);
}
catch (InvalidOperationException)
{
}
}
Console.WriteLine("\r\nNo more items to take.");
});
var tasks = new List<Task>();
for (var i = 0; i < 10; i++)
{
var num = i;
var t = Task.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
results.Add(num);
});
tasks.Add(t);
}
Task.Factory.ContinueWhenAll(tasks.ToArray(), _ => results.CompleteAdding());
Console.WriteLine("End of Main");
Console.ReadKey();
}

You'll have to create a writer task of some sort, however, keep in mind even this task can be rescheduled onto another native or managed thread! Using the default scheduler in TPL you have no control over which managed thread receives the work.
public class ConcurrentConsole
{
private static BlockingCollection<string> output
= new BlockingCollection<string>();
public static Task CreateWriterTask(CancellationToken token)
{
return new Task(
() =>
{
while (!token.IsCancellationRequested)
{
string nextLine = output.Take(token);
Console.WriteLine(nextLine);
}
},
token);
}
public static void WriteLine(Func<string> writeLine)
{
output.Add(writeLine());
}
}
When I switched your code to use this I received the following output:
End of Main
Done 1 on 6
Completed 1 on 6
Done 5 on 9
Completed 5 on 9
Done 0 on 4
Completed 0 on 4
Done 2 on 5
Completed 2 on 13
Done 7 on 10
Completed 7 on 10
Done 4 on 8
Completed 4 on 5
Done 9 on 12
Completed 9 on 9
Done 6 on 6
Completed 6 on 5
Done 8 on 11
Completed 8 on 4
Done 3 on 7
Completed 3 on 7
Even with your code sending () => String.Format("Completed {0} on {1}"... to ConcurrentConsole.WriteLine, ensuring the ManagedThreadId would be picked up on the ConcurrentConsole Task, it still would alter which thread it ran on. Although with less variability than the executing tasks.

You can use OrderedTaskScheduler to ensure only one task completion is run at a time; however, they will run on a threadpool thread (not necessarily all on the same thread).
If you really need them all on the same thread (not just one at a time), then you can use ActionThread from the Nito.Async library. It provides a SynchronizationContext for its code, which can be picked up by FromCurrentSynchronizationContext.

I would suggest:
1) Creating a lock object
2) Create a list of strings to be written
3) Spawn a thread that loops, sleeping for a bit, then locking the list of strings, then if it isn't empty, writing all of them and emptying the list
4) Other threads then lock the list, add their status, unlock and continue.
object writeListLocker = new object();
List<string> linesToWrite = new List<string>();
// Main thread loop
for (; ; )
{
lock (writerListLocker)
{
foreach (string nextLine in linesToWrite)
Console.WriteLine(nextLine);
linesToWrite.Clear();
}
Thread.Sleep(500);
}
// Reporting threads
lock (writerListLocker)
{
linesToWrite.Add("Completed (etc.)");
}

I think you expect a result like the following.
Starting on 8
Done 1 on 11
Completed 1 on 9
Done 5 on 11
Completed 5 on 9
Done 0 on 10
Completed 0 on 9
Done 2 on 12
Completed 2 on 9
Done 7 on 16
Completed 7 on 9
Done 4 on 14
Completed 4 on 9
Done 9 on 18
Completed 9 on 9
Done 6 on 15
Completed 6 on 9
Done 8 on 17
Completed 8 on 9
Done 3 on 13
Completed 3 on 9
As below, I used the StaSynchronizationContext in my code from the Understanding SynchronizationContext where a synchronized call in one thread is explained well. Please, refer to it.
My code snippet is:
static void Main(string[] args)
{
StaSynchronizationContext context = new StaSynchronizationContext();
StaSynchronizationContext.SetSynchronizationContext(context);
Console.WriteLine("Starting on {0}", Thread.CurrentThread.ManagedThreadId);
for (var i = 0; i < 10; i++)
{
var num = i;
Task<int>.Factory.StartNew(() =>
{
if (num == 3)
{
Thread.Sleep(20000);
}
Thread.Sleep(new Random(num).Next(1000, 10000));
Console.WriteLine("Done {0} on {1}", num, Thread.CurrentThread.ManagedThreadId);
return num;
}).ContinueWith(
value =>
{
Console.WriteLine("Completed {0} on {1}", value.Result, Thread.CurrentThread.ManagedThreadId);
}
,TaskScheduler.FromCurrentSynchronizationContext());
}
Console.WriteLine("End of Main");
Console.ReadKey();
}