We Keep Coding

C# Immutable counter for multiple fields

I have a fairly high throughput on a message counter (tens of thousands per second), and looking for an efficient way of getting the count without putting locks everywhere or ideally not locking on each message count when i am giving an update every 10 seconds.
Use of immutable counter object
I am using an immutable counter class:
public class Counter
{
public Counter(int quotes, int trades)
{
Quotes = quotes;
Trades = trades;
}
readonly public int Quotes;
readonly public int Trades;
// and some other counter fields snipped
}
And would update this on each message process loop:
class MyProcessor
{
System.Timers.Timer timer;
Counter counter = new Counter(0,0);
public MyProcessor()
{
// update ever 10 seconds
this.timer = new System.Timers.Timer(10000);
timer.Elapsed += (sender, e) => {
var quotesPerSecond = this.counter.Quotes / 10.0;
var tradesPerSecond = this.counter.Trades / 10.0;
this.Counter = new Counter(0,0);
});
}
public void ProcessMessages(Messages messages)
{
foreach(var message in messages) { /* */ }
var oldCounter = counter;
this.counter = new Counter(oldCounter.Quotes, oldCounter.Trades);
}
}
I have lots of counters (not all shown), so would mean a lot of individual Interlocked.Increment calls on individual counter fields.
The only other way I can think of is lock every single run of ProcessMessages (which will be extensive) and heavy for something which is a utility as opposed to critical where the program would crash.
Is it possible to use an immutable counter object in this fashion without hard interlocking/thread mechanisms when we only need to update once every 10 seconds?
Flag check idea to avoid locks
Could the timer thread set a flag for the ProcessMessages to check and if it sees it set, start the count from zero again, i.e.
/* snipped the MyProcessor class, same as before */
System.Timers.Timer timer;
Counter counter = new Counter(0,0);
ManualResetEvent reset = new ManualResetEvent(false);
public MyProcessor()
{
// update ever 10 seconds
this.timer = new System.Timers.Timer(10000);
timer.Elapsed += (sender, e) => {
var quotesPerSecond = this.counter.Quotes / 10.0;
var tradesPerSecond = this.counter.Trades / 10.0;
// log
this.reset.Set();
});
}
// this should be called every second with a heartbeat message posted to queue
public void ProcessMessages(Messages messages)
{
if (reset.WaitOne(0) == true)
{
this.counter = new Counter(this.counter.Quotes, this.counter.Trades, this.counter.Aggregates);
reset.Reset();
}
else
{
this.counter = new Counter(
this.counter.Quotes + message.Quotes.Count,
this.counter.Trades + message.Trades.Count);
}
}
/* end of MyProcessor class */
This would work, however the update "stalls" when the process messages comes to a halt (although the throughput is very high, it does pause for a number of hours at night ideally should show the actual rather than last value).
One way around this would be to post a heartbeat message to the MyProcessor.ProcessMessages() every second to force an internal update of the message counters and subsequent reset when the reset ManualResetEvent is set.

Here are three new methods for your Counter class. One for reading the latest value from a specific location, one for updating safely a specific location, and one for creating easily a new Counter based on an existing one:
public static Counter Read(ref Counter counter)
{
return Interlocked.CompareExchange(ref counter, null, null);
}
public static void Update(ref Counter counter, Func<Counter, Counter> updateFactory)
{
var counter1 = counter;
while (true)
{
var newCounter = updateFactory(counter1);
var counter2 = Interlocked.CompareExchange(ref counter, newCounter, counter1);
if (counter2 == counter1) break;
counter1 = counter2;
}
}
public Counter Add(int quotesDelta, int tradesDelta)
{
return new Counter(Quotes + quotesDelta, Trades + tradesDelta);
}
Usage example:
Counter latest = Counter.Read(ref this.counter);
Counter.Update(ref this.counter, existing => existing.Add(1, 1));
Accessing the MyProcessor.counter field directly by multiple threads concurrently is not thread-safe, because it's neither volatile nor protected by a lock. The above methods are safe to use because they are accessing the field through interlocked operations.

I wanted to update everyone with what I had come up with, the counter updates were pushed within the thread itself.
Everything is driven by the DequeueThread loop, and specifically this.queue.ReceiveAsync(TimeSpan.FromSeconds(UpdateFrequencySeconds)) function.
This will either return an item from the queue, process it and update the counters, or timeout and then update the counters - there are no other threads involved everything, including updating message rate, is done within the thread.
In summary, nothing runs in parallel (in terms of dequing the packet), it is fetching the items one at a time and processing it and the counters thereafter. Then finally looping back to process the next item in the queue.
This removes the need for synchronisation:
internal class Counter
{
public Counter(Action<int,int,int,int> updateCallback, double updateEvery)
{
this.updateCallback = updateCallback;
this.UpdateEvery = updateEvery;
}
public void Poll()
{
if (nextUpdate < DateTimeOffset.UtcNow)
{
// post the stats, and reset
this.updateCallback(this.quotes, this.trades, this.aggregates, this.statuses);
this.quotes = 0;
this.trades = 0;
this.aggregates = 0;
this.statuses = 0;
nextUpdate = DateTimeOffset.UtcNow.AddSeconds(this.UpdateEvery);
}
}
public void AddQuotes(int count) => this.quotes += count;
public void AddTrades(int count) => this.trades += count;
public void AddAggregates(int count) => this.aggregates += count;
public void AddStatuses(int count) => this.statuses += count;
private int quotes;
private int trades;
private int aggregates;
private int statuses;
private readonly Action<int,int,int,int> updateCallback;
public double UpdateEvery { get; private set; }
private DateTimeOffset nextUpdate;
}
public class DeserializeWorker
{
private readonly BufferBlock<byte[]> queue = new BufferBlock<byte[]>();
private readonly IPolygonDeserializer polygonDeserializer;
private readonly ILogger<DeserializeWorker> logger;
private readonly Counter counter;
const double UpdateFrequencySeconds = 5.0;
long maxBacklog = 0;
public DeserializeWorker(IPolygonDeserializer polygonDeserializer, ILogger<DeserializeWorker> logger)
{
this.polygonDeserializer = polygonDeserializer ?? throw new ArgumentNullException(nameof(polygonDeserializer));
this.logger = logger;
this.counter = new Counter(ProcesCounterUpdateCallback, UpdateFrequencySeconds);
}
public void Add(byte[] data)
{
this.queue.Post(data);
}
public Task Run(CancellationToken stoppingToken)
{
return Task
.Factory
.StartNew(
async () => await DequeueThread(stoppingToken),
stoppingToken,
TaskCreationOptions.LongRunning,
TaskScheduler.Default)
.Unwrap();
}
private async Task DequeueThread(CancellationToken stoppingToken)
{
while (stoppingToken.IsCancellationRequested == false)
{
try
{
var item = await this.queue.ReceiveAsync(TimeSpan.FromSeconds(UpdateFrequencySeconds), stoppingToken);
await ProcessAsync(item);
}
catch (TimeoutException)
{
// this is ok, timeout expired
}
catch(TaskCanceledException)
{
break; // task cancelled, break from loop
}
catch (Exception e)
{
this.logger.LogError(e.ToString());
}
UpdateCounters();
}
await StopAsync();
}
protected async Task StopAsync()
{
this.queue.Complete();
await this.queue.Completion;
}
protected void ProcessStatuses(IEnumerable<Status> statuses)
{
Parallel.ForEach(statuses, (current) =>
{
if (current.Result != "success")
this.logger.LogInformation($"{current.Result}: {current.Message}");
});
}
protected void ProcessMessages<T>(IEnumerable<T> messages)
{
Parallel.ForEach(messages, (current) =>
{
// serialize by type T
// dispatch
});
}
async Task ProcessAsync(byte[] item)
{
try
{
var memoryStream = new MemoryStream(item);
var message = await this.polygonDeserializer.DeserializeAsync(memoryStream);
var messagesTask = Task.Run(() => ProcessStatuses(message.Statuses));
var quotesTask = Task.Run(() => ProcessMessages(message.Quotes));
var tradesTask = Task.Run(() => ProcessMessages(message.Trades));
var aggregatesTask = Task.Run(() => ProcessMessages(message.Aggregates));
this.counter.AddStatuses(message.Statuses.Count);
this.counter.AddQuotes(message.Quotes.Count);
this.counter.AddTrades(message.Trades.Count);
this.counter.AddAggregates(message.Aggregates.Count);
Task.WaitAll(messagesTask, quotesTask, aggregatesTask, tradesTask);
}
catch (Exception e)
{
this.logger.LogError(e.ToString());
}
}
void UpdateCounters()
{
var currentCount = this.queue.Count;
if (currentCount > this.maxBacklog)
this.maxBacklog = currentCount;
this.counter.Poll();
}
void ProcesCounterUpdateCallback(int quotes, int trades, int aggregates, int statuses)
{
var updateFrequency = this.counter.UpdateEvery;
logger.LogInformation(
$"Queue current {this.queue.Count} (max {this.maxBacklog }), {quotes / updateFrequency} quotes/sec, {trades / updateFrequency} trades/sec, {aggregates / updateFrequency} aggregates/sec, {statuses / updateFrequency} status/sec");
}
}

Threads monitoring a Queue<Actions>

I doing a small project to map a network (routers only) using SNMP. In order to speed things up, I´m trying to have a pool of threads responsible for doing the jobs I need, apart from the first job which is done by the main thread.
At this time I have two jobs, one takes a parameter the other doesn´t:
UpdateDeviceInfo(NetworkDevice nd)
UpdateLinks() *not defined yet
What I´m trying to achieve is to have those working threads waiting for a job to
appear on a Queue<Action> and wait while it is empty. The main thread will add the first job and then wait for all workers, which might add more jobs, to finish before starting adding the second job and wake up the sleeping threads.
My problem/questions are:
How to define the Queue<Actions> so that I can insert the methods and the parameters if any. If not possible I could make all functions accept the same parameter.
How to launch the working threads indefinitely. I not sure where should I create the for(;;).
This is my code so far:
public enum DatabaseState
{
Empty = 0,
Learning = 1,
Updating = 2,
Stable = 3,
Exiting = 4
};
public class NetworkDB
{
public Dictionary<string, NetworkDevice> database;
private Queue<Action<NetworkDevice>> jobs;
private string _community;
private string _ipaddress;
private Object _statelock = new Object();
private DatabaseState _state = DatabaseState.Empty;
private readonly int workers = 4;
private Object _threadswaitinglock = new Object();
private int _threadswaiting = 0;
public Dictionary<string, NetworkDevice> Database { get => database; set => database = value; }
public NetworkDB(string community, string ipaddress)
{
_community = community;
_ipaddress = ipaddress;
database = new Dictionary<string, NetworkDevice>();
jobs = new Queue<Action<NetworkDevice>>();
}
public void Start()
{
NetworkDevice nd = SNMP.GetDeviceInfo(new IpAddress(_ipaddress), _community);
if (nd.Status > NetworkDeviceStatus.Unknown)
{
database.Add(nd.Id, nd);
_state = DatabaseState.Learning;
nd.Update(this); // The first job is done by the main thread
for (int i = 0; i < workers; i++)
{
Thread t = new Thread(JobRemove);
t.Start();
}
lock (_statelock)
{
if (_state == DatabaseState.Learning)
{
Monitor.Wait(_statelock);
}
}
lock (_statelock)
{
if (_state == DatabaseState.Updating)
{
Monitor.Wait(_statelock);
}
}
foreach (KeyValuePair<string, NetworkDevice> n in database)
{
using (System.IO.StreamWriter file = new System.IO.StreamWriter(n.Value.Name + ".txt")
{
file.WriteLine(n);
}
}
}
}
public void JobInsert(Action<NetworkDevice> func, NetworkDevice nd)
{
lock (jobs)
{
jobs.Enqueue(item);
if (jobs.Count == 1)
{
// wake up any blocked dequeue
Monitor.Pulse(jobs);
}
}
}
public void JobRemove()
{
Action<NetworkDevice> item;
lock (jobs)
{
while (jobs.Count == 0)
{
lock (_threadswaitinglock)
{
_threadswaiting += 1;
if (_threadswaiting == workers)
Monitor.Pulse(_statelock);
}
Monitor.Wait(jobs);
}
lock (_threadswaitinglock)
{
_threadswaiting -= 1;
}
item = jobs.Dequeue();
item.Invoke();
}
}
public bool NetworkDeviceExists(NetworkDevice nd)
{
try
{
Monitor.Enter(database);
if (database.ContainsKey(nd.Id))
{
return true;
}
else
{
database.Add(nd.Id, nd);
Action<NetworkDevice> action = new Action<NetworkDevice>(UpdateDeviceInfo);
jobs.Enqueue(action);
return false;
}
}
finally
{
Monitor.Exit(database);
}
}
//Job1 - Learning -> Update device info
public void UpdateDeviceInfo(NetworkDevice nd)
{
nd.Update(this);
try
{
Monitor.Enter(database);
nd.Status = NetworkDeviceStatus.Self;
}
finally
{
Monitor.Exit(database);
}
}
//Job2 - Updating -> After Learning, create links between neighbours
private void UpdateLinks()
{
}
}

Your best bet seems like using a BlockingCollection instead of the Queue class. They behave effectively the same in terms of FIFO, but a BlockingCollection will let each of your threads block until an item can be taken by calling GetConsumingEnumerable or Take. Here is a complete example.
http://mikehadlow.blogspot.com/2012/11/using-blockingcollection-to-communicate.html?m=1
As for including the parameters, it seems like you could use closure to enclose the NetworkDevice itself and then just enqueue Action instead of Action<>

Why does the performance differ so greatly with seemingly the same code?

I have a WCF service called TapApiService which contains the following methods:
public class TapApiService : Contracts.ServiceContracts.ITapApiService
{
private static readonly IConfigurationManagerWrapper ConfigWrapper = new ConfigurationManagerWrapper();
private static UriManager uriManager = new UriManager(ConfigWrapper.AppSettings["tapScheme"], ConfigWrapper.AppSettings["tapHost"], ConfigWrapper);
public JobLongForm GetJob(string jobId)
{
var request = uriManager.GetUriFromKey("jobResource", jobId).ToString().GetTapRequest("GET", ConfigWrapper);
return JsonConvert.DeserializeObject<JobLongForm>(request.GetResponseString());
}
public IEnumerable<string> GetAllJobIds()
{
return GetAllJobs().Select(job => job.Id);
}
public IEnumerable<JobShortForm> GetAllJobs()
{
var request = uriManager.GetUriFromKey("allJobsResource").ToString().GetTapRequest("GET", ConfigWrapper);
return JsonConvert.DeserializeObject<JobShortForm[]>(request.GetResponseString());
}
public IEnumerable<JobLongForm> GetFullJobs(IEnumerable<string> jobIds)
{
return jobIds.Select(GetJob);
}
}
I have a Console Application I'm using to test. It has the following code:
var proxy = new TapApiService();
var stopwatch = new Stopwatch();
stopwatch.Start();
var ids = proxy.GetAllJobIds();
proxy.GetFullJobs(ids);
stopwatch.Stop();
Console.WriteLine(stopwatch.GetElapsedTimeString());
stopwatch.Reset();
stopwatch.Start();
proxy.GetAllJobIds().AsParallel().Select(id => proxy.GetJob(id));
stopwatch.Stop();
Console.WriteLine(stopwatch.GetElapsedTimeString());
The results are: 00:00:00.713 and 00:00:00.104 The AsParallel() implementation is clearly a lot faster. However, if I move the AsParallel() to inside of the GetFullJobs(IEnumerable<string> jobIds) method like this:
public IEnumerable<JobLongForm> GetFullJobs(IEnumerable<string> jobIds)
{
return jobIds.AsParallel().Select(GetJob);
}
I get 00:00:00.813 and 00:00:00.158. The second implementation is still faster. I thought their performance would now be the same. What am I missing here?

Consuming blocking collection with multiple tasks/consumers

I have the following code that I populate users from a source, for the sake of example it is as below. what I want to do is to consume BlockingCollection with multiple consumers.
Is below the right way to do that? Also what would be the best number of threads ? ok this would depend on hardware, memory etc. Or how can i do it in a better way?
Also would below implementation ensure that i will process everything in the collection until it is empty?
class Program
{
public static readonly BlockingCollection<User> users = new BlockingCollection<User>();
static void Main(string[] args)
{
for (int i = 0; i < 100000; i++)
{
var u = new User {Id = i, Name = "user " + i};
users.Add(u);
}
Run();
}
static void Run()
{
for (int i = 0; i < 100; i++)
{
Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning);
}
}
static void Process()
{
foreach (var user in users.GetConsumingEnumerable())
{
Console.WriteLine(user.Id);
}
}
}
public class User
{
public int Id { get; set; }
public string Name { get; set; }
}

A few small things
You never called CompleteAdding, by not doing that your consuming foreach loops will never complete and hang forever. Fix that by doing users.CompleteAdding() after the initial for loop.
You never wait for the work to finish, Run() will spin up your 100 threads (which likely WAY too much unless your real process involves a lot of waiting for uncontested resources). Because Tasks are not foreground threads they will not keep your program open when your Main exits. You need a CountdownEvent to track when everything is done.
You don't start up your consumers till after your producer has finished all of it's work, you should spin off the producer in to a separate thread or start the consumers first so they are ready to work while you populate the producer on the main thread.
here is a updated version of the code with the fixes
class Program
{
private const int MaxThreads = 100; //way to high for this example.
private static readonly CountdownEvent cde = new CountdownEvent(MaxThreads);
public static readonly BlockingCollection<User> users = new BlockingCollection<User>();
static void Main(string[] args)
{
Run();
for (int i = 0; i < 100000; i++)
{
var u = new User {Id = i, Name = "user " + i};
users.Add(u);
}
users.CompleteAdding();
cde.Wait();
}
static void Run()
{
for (int i = 0; i < MaxThreads; i++)
{
Task.Factory.StartNew(Process, TaskCreationOptions.LongRunning);
}
}
static void Process()
{
foreach (var user in users.GetConsumingEnumerable())
{
Console.WriteLine(user.Id);
}
cde.Signal();
}
}
public class User
{
public int Id { get; set; }
public string Name { get; set; }
}
For the "Best number of threads" like I said earlier, it really depends on what you are waiting on.
If what you are processing is CPU bound, the optimum number of threads is likely Enviorment.ProcessorCount.
If what you are doing is waiting on a external resource, but new requests do not affect old requests (for example asking 20 different servers for information, server the load on server n does not affect the load on server n+1) in that case I would let Parallel.ForEach just choose the number of threads for you.
If you are waiting on a resource that is contended (for example reading/writing to a hard disk) you will want to not use very many threads at all (perhaps even only use one). I just posted a answer in another question about that, when reading in from the hard disk, you should only just use one thread at a time so the hard drive is not jumping around all over trying to complete all the reads at once.

Activator.CreateInstance<T> vs Compiled Expression. Inverse performance on two different machines

A friend and I were testing using compiled expressions for object creation instead of Activator.CreateInstance<T> and ran into some interesting results. We found that when we ran the same code on each of our machines we saw completely opposite results. He got the expected result, significantly better performance out of the compiled expression, whereas I was surprised to see Activator.CreateInstance<T> out perform by 2x.
Both computers ran compiled in .NET 4.0
Computer 1 has .NET 4.5 installed. Computer 2 does not.
Computer 1 over 100000 objects:
45ms - Type<Test>.New()
19ms - System.Activator.CreateInstance<Test>();
Computer 2 over 100000 objects:
13ms - Type<Test>.New()
86ms - System.Activator.CreateInstance<Test>();
And here is the code:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Linq.Expressions;
namespace NewNew
{
class Program
{
static void Main(string[] args)
{
Stopwatch benchmark = Stopwatch.StartNew();
for (int i = 0; i < 100000; i++)
{
var result = Type<Test>.New();
}
benchmark.Stop();
Console.WriteLine(benchmark.ElapsedMilliseconds + " Type<Test>.New()");
benchmark = Stopwatch.StartNew();
for (int i = 0; i < 100000; i++)
{
System.Activator.CreateInstance<Test>();
}
benchmark.Stop();
Console.WriteLine(benchmark.ElapsedMilliseconds + " System.Activator.CreateInstance<Test>();");
Console.Read();
}
static T Create<T>(params object[] args)
{
var types = args.Select(p => p.GetType()).ToArray();
var ctor = typeof(T).GetConstructor(types);
var exnew = Expression.New(ctor);
var lambda = Expression.Lambda<T>(exnew);
var compiled = lambda.Compile();
return compiled;
}
}
public delegate object ObjectActivator(params object[] args);
public static class TypeExtensions
{
public static object New(this Type input, params object[] args)
{
if (TypeCache.Cache.ContainsKey(input))
return TypeCache.Cache[input](args);
var types = args.Select(p => p.GetType());
var constructor = input.GetConstructor(types.ToArray());
var paraminfo = constructor.GetParameters();
var paramex = Expression.Parameter(typeof(object[]), "args");
var argex = new Expression[paraminfo.Length];
for (int i = 0; i < paraminfo.Length; i++)
{
var index = Expression.Constant(i);
var paramType = paraminfo[i].ParameterType;
var accessor = Expression.ArrayIndex(paramex, index);
var cast = Expression.Convert(accessor, paramType);
argex[i] = cast;
}
var newex = Expression.New(constructor, argex);
var lambda = Expression.Lambda(typeof(ObjectActivator), newex, paramex);
var result = (ObjectActivator)lambda.Compile();
TypeCache.Cache.Add(input, result);
return result(args);
}
}
public class TypeCache
{
internal static IDictionary<Type, ObjectActivator> Cache;
static TypeCache()
{
Cache = new Dictionary<Type, ObjectActivator>();
}
}
public class Type<T>
{
public static T New(params object[] args)
{
return (T)typeof(T).New(args);
}
}
public class Test
{
public Test()
{
}
public Test(string name)
{
Name = name;
}
public string Name { get; set; }
}
}

There are at least two causes for this:
The overhead of calling Type<Test>.New() or System.Activator.CreateInstance<Test>() for the first time is relatively large. Because of this, I changed 100000 to 10000000.
Build your application in release mode, run it without a debugger.
With those two changes, the two methods take about equally long. On my system, I get between 1100 and 1200 for both methods, sometimes one is a little bit higher, sometimes the other is.
Note that Activator.CreateInstance<T>() can only call the default constructor, while your New() accepts a number of arguments. If you make your New() less powerful, and always use the default constructor there too, it is slightly faster than Activator.CreateInstance<T>() on my system.
Note also that your handling of constructors with parameters does not actually work if two different constructors of the same type should be used, depending on the passed arguments. You choose once which constructor to use for the remainder of the entire program.