Performance reading large Dataset from Multiple parallel threads - c#

I’m working on a Genetic Machine Learning project developed in .Net (as opposed to Matlab – My Norm). I’m no pro .net coder so excuse any noobish implementations.
The project itself is huge so I won’t bore you with the full details but basically a population of Artificial Neural Networks (like decision trees) are each evaluated on a problem domain that in this case uses a stream of sensory inputs. The top performers in the population are allowed to breed and produced offspring (that inherit tendencies from both parents) and the poor performers are killed off or breed-out of the population. Evolution continues until an acceptable solution is found. Once found, the final evolved ‘Network’ is extracted from the lab and placed in a light-weight real-world application. The technique can be used to develop very complex control solution that would be almost impossible or too time consuming to program normally, like automated Car driving, mechanical stability control, datacentre load balancing etc, etc.
Anyway, the project has been a huge success so far and is producing amazing results, but the only problem is the very slow performance once I move to larger datasets. I’m hoping is just my code, so would really appreciate some expert help.
In this project, convergence to a solution close to an ideal can often take around 7 days of processing! Just making a little tweak to a parameter and waiting for results is just too painful.
Basically, multiple parallel threads need to read sequential sections of a very large dataset (the data does not change once loaded). The dataset consists of around 300 to 1000 Doubles in a row and anything over 500k rows. As the dataset can exceed the .Net object limit of 2GB, it can’t be stored in normal 2d array – The simplest way round this was to use a Generic List of single arrays.
The parallel scalability seems to be a big limiting factor as running the code on a beast of a server with 32 Xeon cores that normally eats Big dataset for breakfast does not yield much of a performance gain over a Corei3 desktop!
Performance gains quickly dwindle away as the number of cores increases.
From profiling the code (with my limited knowledge) I get the impression that there is a huge amount of contention reading the dataset from multiple threads.
I’ve tried experimenting with different dataset implementations using Jagged arrays and various concurrent collections but to no avail.
I’ve knocked up a quick and dirty bit of code for benchmarking that is similar to the core implementation of the original and still exhibits the similar read performance issues and parallel scalability issues.
Any thoughts or suggestions would be much appreciated or confirmation that this is the best I’m going to get.
Many thanks
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading.Tasks;
//Benchmark script to time how long it takes to read dataset per iteration
namespace Benchmark_Simple
{
class Program
{
public static TrainingDataSet _DataSet;
public static int Features = 100; //Real test will require 300+
public static int Rows = 200000; //Real test will require 500K+
public static int _PopulationSize = 500; //Real test will require 1000+
public static int _Iterations = 10;
public static List<NeuralNetwork> _NeuralNetworkPopulation = new List<NeuralNetwork>();
static void Main()
{
Stopwatch _Stopwatch = new Stopwatch();
//Create Dataset
Console.WriteLine("Creating Training DataSet");
_DataSet = new TrainingDataSet(Features, Rows);
Console.WriteLine("Finished Creating Training DataSet");
//Create Neural Network Population
for (int i = 0; i <= _PopulationSize - 1; i++)
{
_NeuralNetworkPopulation.Add(new NeuralNetwork());
}
//Main Loop
for (int i = 0; i <= _Iterations - 1; i++)
{
_Stopwatch.Restart();
Parallel.ForEach(_NeuralNetworkPopulation, _Network => { EvaluateNetwork(_Network); });
//######## Removed for simplicity ##########
//Run Evolutionary Genetic Algorithm on population - I.E. Breed the strong, kill of the weak
//##########################################
//Repeat until acceptable solution is found
Console.WriteLine("Iteration time: {0}", _Stopwatch.ElapsedMilliseconds / 1000);
_Stopwatch.Stop();
}
Console.ReadLine();
}
private static void EvaluateNetwork(NeuralNetwork Network)
{
//Evaluate network on 10% of the Training Data at a random starting point
double Score = 0;
Random Rand = new Random();
int Count = (Rows / 100) * 10;
int RandonStart = Rand.Next(0, Rows - Count);
//The data must be read sequentially
for (int i = RandonStart; i <= RandonStart + Count; i++)
{
double[] NetworkInputArray = _DataSet.GetDataRow(i);
//####### Dummy Evaluation - just give it somthing to do for the sake of it
double[] Temp = new double[NetworkInputArray.Length + 1];
for (int j = 0; j <= NetworkInputArray.Length - 1; j++)
{
Temp[j] = Math.Log(NetworkInputArray[j] * Rand.NextDouble());
}
Score += Rand.NextDouble();
//##################
}
Network.Score = Score;
}
public class TrainingDataSet
{
//Simple demo class of fake data for benchmarking
private List<double[]> DataList = new List<double[]>();
public TrainingDataSet(int Features, int Rows)
{
Random Rand = new Random();
for (int i = 1; i <= Rows; i++)
{
double[] NewRow = new double[Features];
for (int j = 0; j <= Features - 1; j++)
{
NewRow[j] = Rand.NextDouble();
}
DataList.Add(NewRow);
}
}
public double[] GetDataRow(int Index)
{
return DataList[Index];
}
}
public class NeuralNetwork
{
//Simple Class to represent a dummy Neural Network -
private double _Score;
public NeuralNetwork()
{
}
public double Score
{
get { return _Score; }
set { _Score = value; }
}
}
}
}

The first thing is that the only way to answer any performance questions is by profiling the application. I'm using the VS 2012 builtin profiler - there are others https://stackoverflow.com/a/100490/19624
From an initial read through the code, i.e. a static analysis the only thing that jumped out at me was the continual reallocation of Temp inside the loop; this is not efficient and if possible needs moving outside of the loop.
With a profiler you can see what's happening:
I profiled first using the code you posted, (top marks to you for posting a full compilable example of the problem, if you hadn't I wouldn't be answering this now).
This shows me that the bulk is in the inside of the loop, I moved the allocation to the the Parallel.ForEach loop.
Parallel.ForEach(_NeuralNetworkPopulation, _Network =>
{
double[] Temp = new double[Features + 1];
EvaluateNetwork(_Network, Temp);
});
So what I can see from the above is that there is 4.4% wastage on the reallocation; but the probably unsurprising thing is that it is the inner loop that is taking 87.6%.
This takes me to my first rule of optimisation which is to first to review your algorithm rather than optimizing the code. A poor implementation of a good algorithm is usually faster than a highly optimized poor algorithm.
Removing the repeated allocate of Temp changes the picture slightly;
Also worth tuning a bit by specifying the parallelism; I've found that Parallel.ForEach is good enough for what I use it for, but again you may get better results from manually partitioning the work up into queues.
Parallel.ForEach(_NeuralNetworkPopulation,
new ParallelOptions { MaxDegreeOfParallelism = 32 },
_Network =>
{
double[] Temp = new double[Features + 1];
EvaluateNetwork(_Network, Temp);
});
Whilst running I'm getting what I'd expect in terms of CPU usage: although my machine was also running another lengthy process which was taking the base level (the peak in the chart below is when profiling this program).
So to summarize
Review the most frequently executed part and come up with new algorithm if possible.
Profile on the target machine
Only when you're sure about (1) above is it then worth looking at optimising the algorithm; considering the following
a) Code optimisations
b) Memory tuning / partioning of data to keep as much in cache
c) Improvements to threading usage

Related

Quick way of checking if a 2D array contains an element c#

I'm currently coding battleships as a part of a college project. The game works perfectly fine but I'd like to implement a way to check if a ship has been completely sunk. This is the method I'm currently using:
public static bool CheckShipSunk(string[,] board, string ship){
for(int i = 0; i < board.GetLength(0); i++){
for(int j = 0; j < board.GetLength(1); j++){
if(board[i,j] == ship){return false;}
}
}
return true;
}
The problem with this is that there are 5 ships, and this is very inefficient when checking hundreds of elements 5 times over, not to mention the sub-par quality of college computers. Is there an easier way of checking if a 2D array contains an element?
Use an arithmetic approach to loop-through with just 1 loop.
public static bool CheckShipSunk(string[,] board, string ship){
int rows = board.GetLength(0);
int cols = board.GetLength(1);
for (int i = 0; i < rows * cols; i++) {
int row = i / cols;
int col = i % cols;
if (board[row, col] == ship)
return false;
}
return true;
}
But I am with Nysand on just caching and storing that information in cells. The above code although might work, is not recommended as it is still not as efficient
this is very inefficient when checking hundreds of elements 5 times over
Have you done any profiling? Computers are fast even your old college computers. Checking hundreds of elements should take microseconds. From Donald Knuths famous quote
There is no doubt that the grail of efficiency leads to abuse. Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.
So if you feel your program is slow I would recommend to start with profiling. If you are in university this might be a very valuable skill to learn.
There are also better algorithms/datastructures that could be employed. I would for example expect each ship to know what locations they are at, and various other information, like if it is sunk at all. Selecting appropriate data structures are also a very important skill to learn, but a difficult one. Also, try to not get stuck in analysis-paralysis, a terrible inefficient ugly working solution is still better than the most beautiful code that does not work.
However, a very easy thing to fix is moving .GetLength out of the loop. This is a very slow call, and only doing this once should make your loop a several times faster for almost no effort. You might also consider replacing the strings with some other identifier, like an int.
Willy-Nilly you have to scan either the entire array or up to the first ship.
You can simplify the code by quering the array with help of Linq, but not increase performance which has O(length * width) time complexity.
using System.Linq;
...
// No explicit loops, juts a query (.net will loop for you)
public static bool CheckShipSunk(string[,] board, string ship) => board is null
? throw new ArgumentNullException(nameof(board))
: board.AsEnumerable().Any(item => item == ship);
If you are looking for performance (say, you have a really huge array, many ships to test etc.), I suggest changing the data structure: Dictionary<string, (int row, int col)>
instead of the string[,] array:
Dictionary<string, (int row, int col)> sunkShips =
new Dictionary<string, (int row, int col)>(StringComparer.OrdinalIgnoreCase) {
{ "Yamato", (15, 46) },
{ "Bismark", (11, 98) },
{ "Prince Of Wales", (23, 55) },
};
and then get it as easy as
public static bool CheckShipSunk(IDictionary<string, (int row, int col)> sunkShips,
string ship) =>
sunkShips?.Keys?.Contains(ship) ?? false;
Note that time complexity is O(1) which means it doesn't depend on board length and width

How to improve performance of ConcurrentDictionary.Count in C#

Recently, I needed to choose between using SortedDictionary and SortedList, and settled on SortedList.
However, now I discovered that my C# program is slowing to a crawl when performing SortedList.Count, which I check using a function/method called thousands of times.
Usually my program would call the function 10,000 times within 35 ms, but while using SortedList.Count, it slowed to 300-400 ms, basically 10x slower.
I also tried SortedList.Keys.Count, but this reduced my performance another 10 times, to over 3000 ms.
I have only ~5000 keys/objects in SortedList<DateTime, object_name>.
I can easily and instantly retrieve data from my sorted list by SortedList[date] (in 35 ms), so I haven't found any problem with the list structure or objects its holding.
Is this performance normal?
What else can I use to obtain the number of records in the list, or just to check that the list is populated?
(besides adding a separate tracking flag, which I may do for now)
CORRECTION:
Sorry, I'm actually using:
ConcurrentDictionary<string, SortedList<DateTime, string>> dict_list = new ConcurrentDictionary<string, SortedList<DateTime, string>>();
And I had various counts in various places, sometimes checking items in the list and other times in ConcurrentDicitonary. So the issue applies to ConcurrentDicitonary and I wrote quick test code to confirm this, which takes 350 ms, without using concurrency.
Here is the test with ConcurrentDicitonary, showing 350 ms:
public static void CountTest()
{
//Create test ConcurrentDictionary
ConcurrentDictionary<int, string> test_dict = new ConcurrentDictionary<int, string>();
for (int i = 0; i < 50000; i++)
{
test_dict[i] = "ABC";
}
//Access .Count property 10,000 times
int tick_count = Environment.TickCount;
for (int i = 1; i <= 10000; i++)
{
int dict_count = test_dict.Count;
}
Console.WriteLine(string.Format("Time: {0} ms", Environment.TickCount - tick_count));
Console.ReadKey();
}
this article recommends calling this instead:
dictionary.Skip(0).Count()
The count could be invalid as soon as the call from the method returns. If you want to write the count to a log for tracing purposes, for example, you can use alternative methods, such as the lock-free enumerator
Well, ConcurrentDictionary<TKey, TValue> must work properly with many threads at once, so it needs some synchronization overhead.
Source code for Count property: https://referencesource.microsoft.com/#mscorlib/system/Collections/Concurrent/ConcurrentDictionary.cs,40c23c8c36011417
public int Count
{
[SuppressMessage("Microsoft.Concurrency", "CA8001", Justification = "ConcurrencyCop just doesn't know about these locks")]
get
{
int count = 0;
int acquiredLocks = 0;
try
{
// Acquire all locks
AcquireAllLocks(ref acquiredLocks);
// Compute the count, we allow overflow
for (int i = 0; i < m_tables.m_countPerLock.Length; i++)
{
count += m_tables.m_countPerLock[i];
}
}
finally
{
// Release locks that have been acquired earlier
ReleaseLocks(0, acquiredLocks);
}
return count;
}
}
Looks like you need to refactor your existing code. Since you didn't provide any code, we can't tell you what you could optimize.
For performance sensitive code I would not recommend to use ConcurrentDictionary.Count property as it has a locking implementation. You could use Interlocked.Increment instead to do the count yourself.

How to best implement K-nearest neighbours in C# for large number of dimensions?

I'm implementing the K-nearest neighbours classification algorithm in C# for a training and testing set of about 20,000 samples each, and 25 dimensions.
There are only two classes, represented by '0' and '1' in my implementation. For now, I have the following simple implementation :
// testSamples and trainSamples consists of about 20k vectors each with 25 dimensions
// trainClasses contains 0 or 1 signifying the corresponding class for each sample in trainSamples
static int[] TestKnnCase(IList<double[]> trainSamples, IList<double[]> testSamples, IList<int[]> trainClasses, int K)
{
Console.WriteLine("Performing KNN with K = "+K);
var testResults = new int[testSamples.Count()];
var testNumber = testSamples.Count();
var trainNumber = trainSamples.Count();
// Declaring these here so that I don't have to 'new' them over and over again in the main loop,
// just to save some overhead
var distances = new double[trainNumber][];
for (var i = 0; i < trainNumber; i++)
{
distances[i] = new double[2]; // Will store both distance and index in here
}
// Performing KNN ...
for (var tst = 0; tst < testNumber; tst++)
{
// For every test sample, calculate distance from every training sample
Parallel.For(0, trainNumber, trn =>
{
var dist = GetDistance(testSamples[tst], trainSamples[trn]);
// Storing distance as well as index
distances[trn][0] = dist;
distances[trn][1] = trn;
});
// Sort distances and take top K (?What happens in case of multiple points at the same distance?)
var votingDistances = distances.AsParallel().OrderBy(t => t[0]).Take(K);
// Do a 'majority vote' to classify test sample
var yea = 0.0;
var nay = 0.0;
foreach (var voter in votingDistances)
{
if (trainClasses[(int)voter[1]] == 1)
yea++;
else
nay++;
}
if (yea > nay)
testResults[tst] = 1;
else
testResults[tst] = 0;
}
return testResults;
}
// Calculates and returns square of Euclidean distance between two vectors
static double GetDistance(IList<double> sample1, IList<double> sample2)
{
var distance = 0.0;
// assume sample1 and sample2 are valid i.e. same length
for (var i = 0; i < sample1.Count; i++)
{
var temp = sample1[i] - sample2[i];
distance += temp * temp;
}
return distance;
}
This takes quite a bit of time to execute. On my system it takes about 80 seconds to complete. How can I optimize this, while ensuring that it would also scale to larger number of data samples? As you can see, I've tried using PLINQ and parallel for loops, which did help (without these, it was taking about 120 seconds). What else can I do?
I've read about KD-trees being efficient for KNN in general, but every source I read stated that they're not efficient for higher dimensions.
I also found this stackoverflow discussion about this, but it seems like this is 3 years old, and I was hoping that someone would know about better solutions to this problem by now.
I've looked at machine learning libraries in C#, but for various reasons I don't want to call R or C code from my C# program, and some other libraries I saw were no more efficient than the code I've written. Now I'm just trying to figure out how I could write the most optimized code for this myself.
Edited to add - I cannot reduce the number of dimensions using PCA or something. For this particular model, 25 dimensions are required.
Whenever you are attempting to improve the performance of code, the first step is to analyze the current performance to see exactly where it is spending its time. A good profiler is crucial for this. In my previous job I was able to use the dotTrace profiler to good effect; Visual Studio also has a built-in profiler. A good profiler will tell you exactly where you code is spending time method-by-method or even line-by-line.
That being said, a few things come to mind in reading your implementation:
You are parallelizing some inner loops. Could you parallelize the outer loop instead? There is a small but nonzero cost associated to a delegate call (see here or here) which may be hitting you in the "Parallel.For" callback.
Similarly there is a small performance penalty for indexing through an array using its IList interface. You might consider declaring the array arguments to "GetDistance()" explicitly.
How large is K as compared to the size of the training array? You are completely sorting the "distances" array and taking the top K, but if K is much smaller than the array size it might make sense to use a partial sort / selection algorithm, for instance by using a SortedSet and replacing the smallest element when the set size exceeds K.

Create ThreadLocal random generators with known seeds

I'm struggling to find a way to have a single random number generator per thread, while at the same time making sure that when the program is re-run, the same numbers are produced.
What I do now is something like this:
class Program {
static void Main(string[] args) {
var seed = 10;
var data = new List<double>();
var dataGenerator = new Random(seed);
for (int i = 0; i < 10000; i++) {
data.Add(dataGenerator.NextDouble());
}
var results = new ConcurrentBag<double>();
Parallel.ForEach(data, (d) => {
var result = Calculate(d, new Random(d.GetHashCode());
results.Add(result);
});
}
static double Calculate(double x, Random random) {
return x * random.NextDouble();
}
}
Because the randomgenerator that creates the 'data' list is provided a seed and the randomgenerators that are used in the calculation are provided a seed based on the hashcode of the number being processed, the results are repeatable. Regardless the number of threads and the order in which they are instantiated.
I'm wondering if it's possible to instantiate just a single randomgenerator for each thread. The following following piece of code seems to accomplish that, but because the random generators are not provided with a (reproducible) seed anymore, the results are not repeatable.
class Program {
static void Main(string[] args) {
var seed = 10;
var data = new List<double>();
var dataGenerator = new Random(seed);
for (int i = 0; i < 10000; i++) {
data.Add(dataGenerator.NextDouble());
}
var results = new ConcurrentBag<double>();
var localRandom = new ThreadLocal<Random>(() => new Random());
Parallel.ForEach(data, (d) => {
var result = Calculate(d, localRandom.Value);
results.Add(result);
});
}
static double Calculate(double x, Random random) {
return x * random.NextDouble();
}
}
Can anyone think of a nice solution to this problem?
It's possible, indeed you very nearly do it correctly in your question, but the problem is that that isn't quite what you want.
If you seeded your thread-local Random with the same number each time, you would make the results deterministic within that thread, related to the number of previous operations. What you want is a pseudo-random number that is deterministic relative to the input.
Well, you could just stick with Random(). It's not that heavy.
Alternatively, you can have your own pseudo-random algorithm. Here's a simple example based on a re-hashing algorithm (intended to distribute the bits of hashcodes even better):
private static double Calculate(double x)
{
unchecked
{
uint h = (uint)x.GetHashCode();
h += (h << 15) ^ 0xffffcd7d;
h ^= (h >> 10);
h += (h << 3);
h ^= (h >> 6);
h += (h << 2) + (h << 14);
return (h ^ (h >> 16)) / (double)uint.MaxValue * x;
}
}
This isn't a particularly good pseudo-random generator, but it's pretty fast. It also does no allocation and leads to no garbage collection.
There-in lies the trade-off of this entire approach; you can simplify the above and be even faster but less "random" or you can be more "random" for more effort. I'm sure there's code out there that is both faster and more "random" than the above, which is more to demonstrate the approach than anything else, but among the rival algorithms you're looking at a trade-off of the quality of the generated number versus the performance. new Random(d).NextDouble() is at a particular point in that trade-off, other approaches are at other points.
Edit: The re-hashing algorithm I used is a Wang/Jenkins hash. I couldn't remember the name when I wrote it.
Edit: Having a better idea of your requirements from the comments, I'd now say that...
You want to create a PRNG class, it could use the algorithm above, that of System.Random (taking reflected code as a starting point), the 128bitXorShift algorithm you mention or whatever. The important difference is that it must have a Reseed method. For example, if you copied System.Random's approach, your reseed would look like most of the constructor's body (indeed, you'd probably refactor so that apart from maybe creating the array it uses, the constructor would call into reseed).
Then you'd create an instance per thread, and call .Reseed(d.GetHashCode()) at the point where you'd create a new Random in your existing code.
Note also that this gives you another advantage, which is that if you depend upon consistent results from your PRNG (which it seems you do), then the fact that you are not promised a consistent algorithm in System.Random between framework versions (perhaps even including patches and security fixes) is a bad point for you, and this approach adds consistency.
However, you are also not promised a consistent algorithm to double.GetHashCode(). I'd doubt they'd change it (unlike string.GetHashCode(), which is often changed), but just in case you could make your Reseed() take a double do something like:
private static unsafe int GetSeedInteger(double d)
{
if(d == 0.0)
return 0;
long num = *((long*)&d);
return ((int)num) ^ (int)(num >> 32);
}
Which pretty much just copies the current double.GetHashCode(), but now you'll be consistent in the face of framework changes.
It might be worth considering breaking the set of tasks into chunks yourself, creating threads for each chunk, and then just creating this object as a local in the per-chunk method.
Pros:
Accessing ThreadLocal<T> is more expensive than accessing a local T.
If the tasks are consistent in relative time to execute, you don't need a lot of Parallel.ForEach's cleverness.
Cons:
Parallel.ForEach is really good at balancing things out. What you're doing has to be very naturally balanced, or saving a lot on a pre-chunk basis, before eschewing its use gains you anything.

What's the appropriate collection for calculating a running mean?

I'm sifting through some of my old bugs and while reviewing some nasty code I realized that my averaging or smoothing algorithm was pretty bad. I did a little research which led me to the "running mean" - makes sense, pretty straightforward. I was thinking through a possible implementation and realized that I don't know which collection would provide the type of "sliding" functionality that I need. In other words, I need to push/add an item to the end of the collection and then also pop/remove the first item from the collection. I think if I knew what this was called I could find the correct collection but I don't know what to search for.
Ideally a collection where you set the max size and anything added to it that exceeds that size would pop off the first item.
To illustrate, here is what I came up with while messing around:
using System;
using System.Collections.Generic;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
LinkedList<int> samples = new LinkedList<int>();
// Simulate packing the front of the samples, this would most like be a pre-averaged
// value from the raw samples
for (int i = 0; i < 10; i++)
{
samples.AddLast(0);
}
for (int i = 0; i < 100; i++)
{
// My attempt at a "sliding collection" - not really sure what to call it but as
// an item is added the first item is removed
samples.RemoveFirst();
samples.AddLast(i);
foreach (int v in samples)
{
Console.Write("{0:000} ", v);
}
Console.WriteLine(String.Empty);
}
Console.ReadLine();
}
}
}
As you can see I am manually handling the removal of the first item. I'm just asking if there is a standard collection that is optimized for this type of use?
It appears that you're looking for a Circular Buffer. Here's a .NET implementation on CodePlex. You may also want to look at this question: How would you code an efficient Circular Buffer in Java or C#?
From the sample you've provided, it isn't clear how exactly this relates to an online-mean algorithm. If the only operation allowed on the buffer is to append; it should be trivial to cache and update the "total" inside the buffer (add the new value, subtract the removed one); making the maintaining of the mean an O(1) operation for every append. In this case, the buffer is effectively holding the Simple Moving Average (SMA) of a series.
Have you had a look at Queue Class
Does a List satisfy your needs?
List<String> myList = new List<String>();
myList.Add("Something to the end");
myList.RemoveAt(0);
#Ani - I'm creating a new Answer instead of comment because I have some code to paste. I took a swing at a dead simple object to assist with my running mean and came up with the following:
class RollingMean
{
int _pos;
int _count;
double[] _buffer;
public RollingMean(int size)
{
_buffer = new double[size];
_pos = 0;
_count = 0;
}
public RollingMean(int size, double initialValue)
: this(size)
{
// Believe it or not there doesn't seem to be a better(performance) way...
for (int i = 0; i < size; i++)
{
_buffer[i] = initialValue;
}
_count = size;
}
public double Push(double value)
{
_buffer[_pos] = value;
_pos = (++_pos > _buffer.Length - 1) ? 0 : _pos;
_count = Math.Min(++_count, _buffer.Length);
return Mean;
}
public double Mean
{
get
{
return _buffer.Sum() / _count;
}
}
}
I'm reading 16 channels of data from a data acquisition system so I will just instantiate one of these for each channel and I think it will be cleaner than managing a multi-dimensional array or separate set of buffer/postition for each channel.
Here is sample usage for anyone interested:
static void Main(string[] args)
{
RollingMean mean = new RollingMean(10, 7);
mean.Push(3);
mean.Push(4);
mean.Push(5);
mean.Push(6);
mean.Push(7.125);
Console.WriteLine( mean.Mean );
Console.ReadLine();
}
I was going to make the RollingMean object a generic rather than lock into double but I couldn't find a generic constraint to limit the tpye numerical types. I moved on, gotta get back to work. Thanks for you help.

Categories