We Keep Coding

Simulating message latency accurately

I have written a simple "latency simulator" which works, but at times, messages are delayed for longer than the time specified. I need help to ensure that messages are delayed for the correct amount of time.
The main problem, I believe, is that I am using Thread.Sleep(x), which is depended on various factors but mainly on the clock interrupt rate, which causes Thread.Sleep() to have a resolution of roughly 15ms. Further, intensive tasks will demand more CPU time and will occasionally result in a delay greater than the one requested. If you are not familiar with the resolution issues of Thread.Sleep, you can read these SO posts: here, here and here.
This is my LatencySimulator:
public class LatencySimulatorResult: EventArgs
{
public int messageNumber { get; set; }
public byte[] message { get; set; }
}
public class LatencySimulator
{
private int messageNumber;
private int latency = 0;
private int processedMessageCount = 0;
public event EventHandler messageReady;
public void Delay(byte[] message, int delay)
{
latency = delay;
var result = new LatencySimulatorResult();
result.message = message;
result.messageNumber = messageNumber;
if (latency == 0)
{
if (messageReady != null)
messageReady(this, result);
}
else
{
ThreadPool.QueueUserWorkItem(ThreadPoolCallback, result);
}
Interlocked.Increment(ref messageNumber);
}
private void ThreadPoolCallback(object threadContext)
{
Thread.Sleep(latency);
var next = (LatencySimulatorResult)threadContext;
var ready = next.messageNumber == processedMessageCount + 1;
while (ready == false)
{
ready = next.messageNumber == processedMessageCount + 1;
}
if (messageReady != null)
messageReady(this, next);
Interlocked.Increment(ref processedMessageCount);
}
}
To use it, you create a new instance and bind to the event handler:
var latencySimulator = new LatencySimulator();
latencySimulator.messageReady += MessageReady;
You then call latencySimulator.Delay(someBytes, someDelay);
When a message has finished being delayed, the event is fired and you can then process the delayed message.
It is important that the order in which messages are added is maintained. I cannot have them coming out the other end of the latency simulator in some random order.
Here is a test program to use the latency simulator and to see how long messages have been delayed for:
private static LatencySimulator latencySimulator;
private static ConcurrentDictionary<int, PendingMessage> pendingMessages;
private static List<long> measurements;
static void Main(string[] args)
{
var results = TestLatencySimulator();
var anomalies = results.Result.Where(x=>x > 32).ToList();
foreach (var result in anomalies)
{
Console.WriteLine(result);
}
Console.ReadLine();
}
static async Task<List<long>> TestLatencySimulator()
{
latencySimulator = new LatencySimulator();
latencySimulator.messageReady += MessageReady;
var numberOfMeasurementsMax = 1000;
pendingMessages = new ConcurrentDictionary<int, PendingMessage>();
measurements = new List<long>();
var sendTask = Task.Factory.StartNew(() =>
{
for (var i = 0; i < numberOfMeasurementsMax; i++)
{
var message = new Message { Id = i };
pendingMessages.TryAdd(i, new PendingMessage() { Id = i });
latencySimulator.Delay(Serialize(message), 30);
Thread.Sleep(50);
}
});
//Spin some tasks up to simulate high CPU usage
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
Task.Factory.StartNew(() => { FindPrimeNumber(100000); });
sendTask.Wait();
return measurements;
}
static long FindPrimeNumber(int n)
{
int count = 0;
long a = 2;
while (count < n)
{
long b = 2;
int prime = 1;// to check if found a prime
while (b * b <= a)
{
if (a % b == 0)
{
prime = 0;
break;
}
b++;
}
if (prime > 0)
{
count++;
}
a++;
}
return (--a);
}
private static void MessageReady(object sender, EventArgs e)
{
LatencySimulatorResult result = (LatencySimulatorResult)e;
var message = (Message)Deserialize(result.message);
if (pendingMessages.ContainsKey(message.Id) != true) return;
pendingMessages[message.Id].stopwatch.Stop();
measurements.Add(pendingMessages[message.Id].stopwatch.ElapsedMilliseconds);
}
static object Deserialize(byte[] arrBytes)
{
using (var memStream = new MemoryStream())
{
var binForm = new BinaryFormatter();
memStream.Write(arrBytes, 0, arrBytes.Length);
memStream.Seek(0, SeekOrigin.Begin);
var obj = binForm.Deserialize(memStream);
return obj;
}
}
static byte[] Serialize<T>(T obj)
{
BinaryFormatter bf = new BinaryFormatter();
using (var ms = new MemoryStream())
{
bf.Serialize(ms, obj);
return ms.ToArray();
}
}
If you run this code, you will see that about 5% of the messages are delayed for more than the expected 30ms. In fact, some are as high as 60ms. Without any background tasks or high CPU usage, the simulator behaves as expected.
I need them all to be 30ms (or as close to as possible) - I do not want some arbitrary 50-60ms delays.
Can anyone suggest how I can refactor this code so that I can achieve the desired result, but without the use of Thread.Sleep() and with as little CPU overhead as possible?

Task.Delay delays too long

I've created a multi task program. This program has around 20 main tasks and each of them calls some sub tasks to operate file I/Os. I wanted each main task to repeat periodically every 500ms, so I enterd the code Task.Delay(500).
The problem is Task.Delay delays a lot more than 500ms sometimes. There is a case it delays more than 3 seconds.
How can I fix it?
The original progam is so big that I created a sample program below.
(1) If Task.Delay is on, over-delay happens.
(2) If Thead.Sleep is on, over-delay doesn't happen.
ThreadPool.SetMinThreads() doesn't seem to resolve it.
Thanks.
class Program
{
const int DELAY_TIME = 500;
const int TASKS = 100;
const int WAITS = 100;
const int WARNING_THRESHOLD = 100;
static void Main(string[] args)
{
//ThreadPool.SetMinThreads(workerThreads: 200, completionPortThreads: 200);
Console.WriteLine("*** Start...");
Test();
Console.WriteLine("*** Done!");
Console.ReadKey();
}
private static void Test()
{
List<Task> tasks = new List<Task>();
for (int taskId = 0; taskId < TASKS; taskId++)
{
tasks.Add(DelaysAsync(taskId));
}
Task.WaitAll(tasks.ToArray());
}
static async Task DelaysAsync(int taskId)
{
await Task.Yield();
Stopwatch sw = new Stopwatch();
for (int i = 0; i < WAITS; i++)
{
sw.Reset();
sw.Start();
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
sw.Stop();
Console.Write($"Task({taskId})_iter({i}) Elapsed={sw.ElapsedMilliseconds}");
if (sw.ElapsedMilliseconds > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine(" *********** Too late!! ************");
}
else
{
Console.WriteLine();
}
}
}
}

I’ve run your test, with .NET 4.6.1 and VS 2017. Here on Xeon E3-1230 v3 CPU it never printed “Too late”, the Elapsed value was within 498-527 ms.
The Thread.Sleep version performed very similarly, 500-528ms per sleep, however the total execution time was much longer because the runtime refused to create 100 OS threads, that’s way too many, so less than 100 DelaysAsync functions ran in parallel. The debugger showed me there were 27 worker threads in Thread.Sleep version and only 9 worker threads in Task.Delay version.
I think you have other apps on your PC creating too many threads and consuming too much CPU. Windows tries to load balance threads evenly so when the whole system is CPU bound, more native threads = more CPU time and therefore less jitter.
If that’s your case and you want to prioritize your app in the scheduler, instead of using Thread.Sleep and more threads, raise the priority of your process.

It seems that I could find the answer. I changed the previous sample program like below. The main difference is using StopWatch or DateTime to measure time durations.
In StopWatch version, many delays happen.
In DateTime version, no or at least very little delays happen(s).
I guess that the cause is the contention of Timer that is used by both StopWatch and Task.Delay. I concluded that I should not use StopWatch and Task.Delay together.
Thank you.
class Program
{
const int DELAY_TIME = 500;
const int TASKS = 100;
const int WAITS = 100;
const int WARNING_THRESHOLD = 500;
static void Main(string[] args)
{
using (Process p = Process.GetCurrentProcess())
{
p.PriorityClass = ProcessPriorityClass.RealTime;
//ThreadPool.SetMinThreads(workerThreads: 200, completionPortThreads: 200);
int workerThreads;
int completionPortThreads;
ThreadPool.GetAvailableThreads(out workerThreads, out completionPortThreads);
Console.WriteLine($"{workerThreads}, {completionPortThreads}");
Console.WriteLine("*** Start...");
Test();
Console.WriteLine("*** Done!");
Console.ReadKey();
}
}
private static void Test()
{
int totalCount = 0;
List<Task<int>> tasks = new List<Task<int>>();
for (int taskId = 0; taskId < TASKS; taskId++)
{
//tasks.Add(DelaysWithStopWatchAsync(taskId)); // many delays
tasks.Add(DelaysWithDateTimeAsync(taskId)); // no delays
}
Task.WaitAll(tasks.ToArray());
foreach (var task in tasks)
{
totalCount += task.Result;
}
Console.WriteLine($"Total counts of deday = {totalCount}");
}
static async Task<int> DelaysWithStopWatchAsync(int taskId)
{
await Task.Yield();
int count = 0;
Stopwatch sw = new Stopwatch();
for (int i = 0; i < WAITS; i++)
{
sw.Reset();
sw.Start();
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
sw.Stop();
Console.Write($"task({taskId})_iter({i}) elapsed={sw.ElapsedMilliseconds}");
if (sw.ElapsedMilliseconds > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine($" *********** Too late!! ************");
count++;
}
else
{
Console.WriteLine();
}
}
return count;
}
static async Task<int> DelaysWithDateTimeAsync(int taskId)
{
await Task.Yield();
int count = 0;
for (int i = 0; i < WAITS; i++)
{
DateTime start = DateTime.Now;
await Task.Delay(DELAY_TIME).ConfigureAwait(false); // (1)
//Thread.Sleep(DELAY_TIME); // (2)
DateTime end = DateTime.Now;
int duration = (end - start).Milliseconds;
Console.Write($"Task({taskId})_iter({i}) Elapsed={duration}");
if (duration > DELAY_TIME + WARNING_THRESHOLD)
{
Console.WriteLine($" *********** Too late!! ************");
count++;
}
else
{
Console.WriteLine();
}
}
return count;
}
}

Restart concurrent tasks as soon as they fail for x number of times

I have a console app that is making HTTP queries and adding/updating products in my database according to response. Some fail and need to be retried a few times.
The way I came up with was to use a dictionary to store the product ID and a Task. Then I can check all the task results and re-run.
This is working but it strikes me as inefficient. Tasks are not being re-created until all tasks have finished. It would be more efficient if they were immediately restarted but I can't figure out how to do this. Also every retry involves a query to the database as only the ID is stored.
I made small app that shows how I am currently retrying failed requests.
Can someone suggest a more efficient method for retrying?
class Program
{
private static void Main(string[] args)
{
HttpQuery m = new HttpQuery();
var task = Task.Run(() => m.Start());
Task.WaitAll(task);
Console.WriteLine("Finished");
Console.ReadLine();
}
}
class HttpQuery
{
public async Task Start()
{
// dictionary where key represent reference to something that needs to be processed and bool whether it has completed or not
ConcurrentDictionary<int, Task<bool>> monitor = new ConcurrentDictionary<int, Task<bool>>();
// start async tasks.
Console.WriteLine("starting first try");
for (int i = 0; i < 1000; i++)
{
Console.Write(i+",");
monitor[i] = this.Query(i);
}
// wait for completion
await Task.WhenAll(monitor.Values.ToArray());
Console.WriteLine();
// start retries
// number of retries per query
int retries = 10;
int count = 0;
// check if max retries exceeded or all completed
while (count < retries && monitor.Any(x => x.Value.Result == false))
{
// make list of numbers that failed
List<int> retryList = monitor.Where(x => x.Value.Result == false).Select(x => x.Key).ToList();
Console.WriteLine("starting try number: " + (count+1) + ", Processing: " + retryList.Count);
// create list of tasks to wait for
List<Task<bool>> toWait = new List<Task<bool>>();
foreach (var i in retryList)
{
Console.Write(i + ",");
monitor[i] = this.Query(i);
toWait.Add(monitor[i]);
}
// wait for completion
await Task.WhenAll(toWait.ToArray());
Console.WriteLine();
count++;
}
Console.WriteLine("ended");
Console.ReadLine();
}
public async Task<bool> Query(int i)
{
// simulate a http request that may or may not fail
Random r = new Random();
int delay = i * r.Next(1, 10);
await Task.Delay(delay);
if (r.Next(0,2) == 1)
{
return true;
}
else
{
return false;
}
}
}

You can create another method and wrap all these ugly retry logic. All of that ugly code goes away :)
public async Task Start()
{
const int MaxNumberOfTries = 10;
List<Task<bool>> tasks = new List<Task<bool>>();
for (int i = 0; i < 1000; i++)
{
tasks.Add(this.QueryWithRetry(i, MaxNumberOfTries));
}
await Task.WhenAll(tasks);
}
public async Task<bool> QueryWithRetry(int i, int numOfTries)
{
int tries = 0;
bool result;
do
{
result = await Query(i);
tries++;
} while (!result && tries < numOfTries);
return result;
}

StackExchange.Redis Async calls slower than sync calls

I understand that the point of asynchronous methods are not to improve performance but I am finding that the asynchronous methods on the StackExchange.Redis is taking alot longer than the sync methods.
public static async Task<bool> GetScoresFromSetAsync(int score, string name)
{
string redisConnection = ConfigurationManager.AppSettings["RedisAccount"].ToString();
ConnectionMultiplexer connection = ConnectionMultiplexer.Connect(redisConnection);
IDatabase _cache = connection.GetDatabase();
List<string> scores = new List<string>();
var resultAsync = await _cache.SortedSetRangeByScoreAsync(name, score, score);
var result = _cache.SortedSetRangeByScore(name score, score);
return true;
}
The async call is taking about 5000 ms while the non async one is taking about 30ms on average. My redis is hosted on azure. Any thoughts?
Edit: I am talking about a single request here. The SortedSetRangeByScore api call is returning within 30 ms while the SortedSetRangeByScoreAsync api call is returning within 5000 ms.

Wondering how are you measuring the latency to compare? I tried measuring it with the following code and the time taken by SE.Redis for async vs sync came out to be pretty close. I hope this helps.
My client code is running on a Azure Iaas VM and connecting to a Azure Redis Cache in the same region.
Measuring sync vs async for sorted set length 10000, iterations 10000
10000 sync calls completed in average 1.41190622 ms
10000 async calls completed in average 1.43989741 ms
Measuring sync vs async for sorted set length 100000, iterations 1
1 sync calls completed in average 0.9513 ms
1 async calls completed in average 1.1436 ms
using StackExchange.Redis;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Threading.Tasks;
namespace RedisLatency
{
class Program
{
private const string host = "myazurecache.redis.cache.windows.net";
private const string password = "password";
private static int sortedsetlength;
private static int iterations;
private static IDatabase _cache;
static void Main(string[] args)
{
sortedsetlength = Int32.Parse(args[0]);
iterations = Int32.Parse(args[1]);
CreateMultiplexer(host,password);
PopulateTestData();
RunTestSync();
RunTestAsync();
}
private static void CreateMultiplexer(string host, string password)
{
Console.WriteLine("Measuring sync vs async for sorted set length {0}, iteration {1}", sortedsetlength,iterations);
ConfigurationOptions configoptions = new ConfigurationOptions();
configoptions.EndPoints.Add(host);
configoptions.Password = password;
configoptions.Ssl = true;
ConnectionMultiplexer connection = ConnectionMultiplexer.Connect(configoptions);
_cache = connection.GetDatabase();
}
private static void PopulateTestData()
{
for (int i = 0; i < sortedsetlength; i++)
{
_cache.SortedSetAdd("testsorted", "user" + i, i);
}
}
static void RunTestSync()
{
for (int warmup = 0; warmup < 100; warmup++)
{
MeasureSync();
}
Stopwatch sw = Stopwatch.StartNew();
for (int i = 0; i < iterations;i++ )
{
MeasureSync();
}
sw.Stop();
Console.WriteLine("{0} sync calls completed in average {1} ms", iterations, sw.Elapsed.TotalMilliseconds/iterations);
}
async static void RunTestAsync()
{
//warm up
for (int warmup = 0; warmup < 100; warmup++)
{
MeasureAsync().Wait();
}
Stopwatch sw = Stopwatch.StartNew();
for (int i = 0; i < iterations; i++)
{
MeasureAsync().Wait();
}
sw.Stop();
Console.WriteLine("{0} async calls completed in average {1} ms", iterations, sw.Elapsed.TotalMilliseconds/iterations);
}
static public void MeasureSync()
{
var result = _cache.SortedSetRangeByScore("testset", 1.0, sortedsetlength / 1.0);
}
async static public Task MeasureAsync()
{
var result = await _cache.SortedSetRangeByScoreAsync("testset", 1.0, sortedsetlength / 1.0);
}
}
}

.NET 2.0 Processing very large lists using ThreadPool

This is further to my question here
By doing some reading .... I moved away from Semaphores to ThreadPool.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
namespace ThreadPoolTest
{
class Data
{
public int Pos { get; set; }
public int Num { get; set; }
}
class Program
{
static ManualResetEvent[] resetEvents = new ManualResetEvent[20];
static void Main(string[] args)
{
int s = 0;
for (int i = 0; i < 100000; i++)
{
resetEvents[s] = new ManualResetEvent(false);
Data d = new Data();
d.Pos = s;
d.Num = i;
ThreadPool.QueueUserWorkItem(new WaitCallback(Process), (object)d);
if (s >= 19)
{
WaitHandle.WaitAll(resetEvents);
Console.WriteLine("Press Enter to Move forward");
Console.ReadLine();
s = 0;
}
else
{
s = s + 1;
}
}
}
private static void Process(object o)
{
Data d = (Data) o;
Console.WriteLine(d.Num.ToString());
Thread.Sleep(10000);
resetEvents[d.Pos].Set();
}
}
}
This code works and I am able to process in the sets of 20. But I don't like this code because of WaitAll. So let's say I start a batch of 20, and 3 threads take longer time while 17 have finished. Even then I will keep the 17 threads as waiting because of the WaitAll.
WaitAny would have been good... but it seems rather messy that I will have to build so much of control structures like Stacks, Lists, Queues etc in order to use the pool efficiently.
The other thing I don't like is that whole global variable in the class for resetEvents. because this array has to be shared between the Process method and the main loop.
The above code works... but I need your help in improving it.
Again... I am on .NET 2.0 VS 2008. I cannot use .NET 4.0 parallel/async framework.

There are several ways you can do this. Probably the easiest, based on what you've posted above, would be:
const int MaxThreads = 4;
const int ItemsToProcess = 10000;
private Semaphore _sem = new Semaphore(MaxThreads, MaxThreads);
void DoTheWork()
{
int s = 0;
for (int i = 0; i < ItemsToProcess; ++i)
{
_sem.WaitOne();
Data d = new Data();
d.Pos = s;
d.Num = i;
ThreadPool.QueueUserWorkItem(Process, d);
++s;
if (s >= 19)
s = 0;
}
// All items have been assigned threads.
// Now, acquire the semaphore "MaxThreads" times.
// When counter reaches that number, we know all threads are done.
int semCount = 0;
while (semCount < MaxThreads)
{
_sem.WaitOne();
++semCount;
}
// All items are processed
// Clear the semaphore for next time.
_sem.Release(semCount);
}
void Process(object o)
{
// do the processing ...
// release the semaphore
_sem.Release();
}
I only used four threads in my example because that's how many cores I have. It makes little sense to be using 20 threads when only four of them can be processing at any one time. But you're free to increase the MaxThreads number if you like.

So I'm pretty sure this is all .NET 2.0.
We'll start out defining Action, because I'm so used to using it. If using this solution in 3.5+, remove that definition.
Next, we create a queue of actions based on the input.
After that we define a callback; this callback is the meat of the method.
It first grabs the next item in the queue (using a lock since the queue isn't thread safe). If it ended up having an item to grab it executes that item. Next it adds a new item to the thread pool which is "itself". This is a recursive anonymous method (you don't come across uses of that all that often). This means that when the callback is called for the first time it will execute one item, then schedule a task which will execute another item, and that item will schedule a task that executes another item, and so on. Eventually the queue will run out, and they'll stop queuing more items.
We also want the method to block until we're all done, so for that we keep track of how many of these callbacks have finished through incrementing a counter. When that counter reaches the task limit we signal the event.
Finally we start N of these callbacks in the thread pool.
public delegate void Action();
public static void Execute(IEnumerable<Action> actions, int maxConcurrentItems)
{
object key = new object();
Queue<Action> queue = new Queue<Action>(actions);
int count = 0;
AutoResetEvent whenDone = new AutoResetEvent(false);
WaitCallback callback = null;
callback = delegate
{
Action action = null;
lock (key)
{
if (queue.Count > 0)
action = queue.Dequeue();
}
if (action != null)
{
action();
ThreadPool.QueueUserWorkItem(callback);
}
else
{
if (Interlocked.Increment(ref count) == maxConcurrentItems)
whenDone.Set();
}
};
for (int i = 0; i < maxConcurrentItems; i++)
{
ThreadPool.QueueUserWorkItem(callback);
}
whenDone.WaitOne();
}
Here's another option that doesn't use the thread pool, and just uses a fixed number of threads:
public static void Execute(IEnumerable<Action> actions, int maxConcurrentItems)
{
Thread[] threads = new Thread[maxConcurrentItems];
object key = new object();
Queue<Action> queue = new Queue<Action>(actions);
for (int i = 0; i < maxConcurrentItems; i++)
{
threads[i] = new Thread(new ThreadStart(delegate
{
Action action = null;
do
{
lock (key)
{
if (queue.Count > 0)
action = queue.Dequeue();
else
action = null;
}
if (action != null)
{
action();
}
} while (action != null);
}));
threads[i].Start();
}
for (int i = 0; i < maxConcurrentItems; i++)
{
threads[i].Join();
}
}