We Keep Coding

How to perform Change Point Analysis using R.NET

How to perform Change Point Analysis using R.NET. I am using below code
REngine.SetEnvironmentVariables();
REngine engine = REngine.GetInstance();
double[] data = new double[] { 1, 2, 3, 4, 5, 6 };
NumericVector vector = engine.CreateNumericVector(data);
engine.SetSymbol("mydatapoints", vector);
engine.Evaluate("library(changepoint)");
engine.Evaluate("chpoints = cpt.mean(mydatapoints, method="BinSeg")");
DynamicVector result = engine.Evaluate("x<-cpts(chpoints)").AsVector(); ;
engine.Dispose();
I am receiving below error on engine.Evaluate("library(changepoint)");
Error in library(changepoint) : there is no package called
'changepoint'
Edit # 1
The changepoint package is supposed to be installed explicitly, it is not there by default. Installed it using RGui -> Packages -> Load package.
Now the error has been changed to
Status Error for chpoints = cpt.mean(mydatapoints, method=”BinSeg”) :
unexpected input
Edit # 2
After fixing first two errors, the following one appears on second Evaluate statement.
Error in BINSEG(sumstat, pen = pen.value, cost_func = costfunc,
minseglen = minseglen, : Q is larger than the maximum number of
segments 4
The same error appears on R as well using these commands
value.ts <- c(29.89, 29.93, 29.72, 29.98)
chpoints = cpt.mean(value.ts, method="BinSeg")

The error is not in your calling code but rather in your use of R (as you apparently now realize.) So the labeling of this as something to do with rdotnet or c-sharp seems misleading:
mydatapoints <- c(1, 2, 3, 4, 5, 6 )
library(changepoint);
chpoints = cpt.mean(mydatapoints, method="BinSeg");
#Error in BINSEG(sumstat, pen = pen.value, cost_func = costfunc, minseglen = minseglen, :
# Q is larger than the maximum number of segments 4
I'm not sure what you intended. Change-point analysis generally requires paired datapoints ... x-y and all that jazz. And giving R regression functions perfectly linear data is also unwise. It often causes non-invertible matrices.
I suggest you search with https://stackoverflow.com/search?q=%5Br%5D+changepoint to find a simple bit of code to build into your REngine calling scheme.

The data points are supposed to be converted in Time Series.
REngine.SetEnvironmentVariables();
REngine engine = REngine.GetInstance();
double[] data = new double[] { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12 };
NumericVector vector = engine.CreateNumericVector(data);
engine.Evaluate("library(changepoint)");
engine.SetSymbol("values", vector);
engine.Evaluate("values.ts = ts(values, frequency = 12, start = c(2017, 1))");
engine.Evaluate("chpoints = cpt.mean(values.ts, method=\"BinSeg\")");
var result = engine.GetSymbol("chpoints");
engine.Dispose();
Now looking for how to get the results back in C#, chpoints or result of plot(chpoints)

Syntax to create a double[,] in C#

This seems extremely simple, yet I can't seem to find applicable documentation anywhere. In C#, how do you create a 'double[,]'? Specifically, the data I'm trying to represent is something like this:
[0,0] = 0
[0,1] = 1
[1,0] = 1
[1,1] = 2
I have tried [[0,1],[1,2]] and the equivalent with {{}{}} and {[][]} and various other things, but cannot seem to figure out the syntax. It seems that a simple [0,1] alone is a 'double[,]' but I would like to find a way to represent the above data (more than just 2 numbers).
What am I missing? If anyone can point me to some simple documentation, that would be great.

See Array initializers:
For a multi-dimensional array, the array initializer must have as many levels of nesting as there are dimensions in the array. The outermost nesting level corresponds to the leftmost dimension and the innermost nesting level corresponds to the rightmost dimension. The length of each dimension of the array is determined by the number of elements at the corresponding nesting level in the array initializer. For each nested array initializer, the number of elements must be the same as the other array initializers at the same level.
In our case:
double[,] a = { { 0, 1 }, { 1, 2 } };

A multi-dimensional double array:
Double[,] newdouble = new Double[2,2];
or
Double[,] newdouble = { { 0, 0 }, { 1, 1 } };

In order to create a two dimensional array that you can assign to, you are going to need to first allocate the correct size. In this case, you have 2 rows and 2 columns, so that will be a [2,2].
double[,] twod = new double[2,2];
Next you simply assign to it like this
twod[0,0] = 0;
twod[0,1] = 1;
twod[1,0] = 1;
twod[1,1] = 2;
And then work with it however you wish.

There is three ways to initialized your array:
double[,] twoDemn = { { 0 , 1 }, { 1 , 2 } };
or:
double[,] twoDemn = new double[,] { { 0 , 1 }, { 1 , 2 } };
or:
double[,] twoDemn = new double[2,2];
twoDemn[0,0] = 0;
twoDemn[0,1] = 1;
twoDemn[1,0] = 1;
twoDemn[1,1] = 2;

C# Conversion Help & PHP

I have another problem converting an array from php to c#.
public $array = array(103 => array('', ''), 102 => array('', ''), 101 => array('', '', ''), 100 => array('', '', '', ''));
This is what I have:
public Dictionary<int, List<string>> waddlesById = new Dictionary<int, List<string>>();
problem is whenever I do this:
sizeof($this->array[$waddleId]) - 1
That equals -1, but when do it in c#:
waddlesById[waddleId].Count - 1
equals 0.
This is my construct function:
string maxSeats = String.Empty;
foreach (int waddleId in sledRacing)
{
switch (waddleId)
{
case 103:
case 102:
maxSeats = ",";
break;
case 101:
maxSeats = ",,";
break;
case 100:
maxSeats = ",,,";
break;
}
if (waddlesById.ContainsKey(waddleId))
{
waddlesById[waddleId].Add(maxSeats);
}
else
{
List<string> newInner = new List<string>();
newInner.Add(maxSeats);
waddlesById.Add(waddleId, newInner);
}
}
Any help is appreciated

Using .Count will automatically create the entry. Use waddlesById.ContainsKey(waddleId) instead: http://msdn.microsoft.com/en-us/library/htszx2dy.aspx

I think you are confusing Lists and Arrays in C#. #Simon Witehead is right, you probably need to rethink your code in C# idioms. At the same time it reminded me the LookUp type (which I believe not that commonly known) and I though I'd try to answer you question in (hopefully) somewhat useful way.
Before that, let me clear one thing first: I assume you are trying to create an array with this code:
maxSeats = ",,,";
If that's correct, then here is how you create an array with three elements:
var myArray = new string[3];
// or similar to your PHP code
var myArray = new string[] { "", "", "" };
As for the LookUp example, I think, it provides a more C# idiomatic way of doing this kind of task:
var sledRacing = new[] { 100, 102, 103, 100, 100, 102, 101 };
var lu = sledRacing.ToLookup(
k => k,
k => k == 100 ? new string[3] : (k == 101 ? new string[2] : new string[1])
);
foreach (var g in lu)
{
Console.WriteLine(g.Key);
foreach (var i in g)
{
Console.WriteLine(" - " + i.Length);
}
}
This will produce the following output:
100
- 3
- 3
- 3
102
- 1
- 1
103
- 1
101
- 2
Having said all that (and not knowing much about PHP, so I can't tell if the language lack dynamically sized array or lists and compare), you might need to rethink your code: If you want to use a List then you don't need to size it to start with. Then you might want to be more 'Object oriented' and encapsulate 'Racing Waddle' in a class and use the dictionary (or LookUp) to index them by their id and so on.
There is a similar example on MSDN using LookUp type, that might help too.

C# HMM Gesture Recognition usng Kinect

I'm working on a solution to do gesture recognition using the Kinect sensor.
Now I'm using Accord .NET to train the HMM.
I have a dataset with saved gestures. This dataset has 11 gestures and each one has 32 frames with 18 points saved.
So I have a (double [12] [32,18]) input dataset and a (int[12]) output dataset, but when i do:
double error = teacher.Run(inputSequences, output), it gives me this : "Specified argument was out of the range of valid values."
Does anyone knows how to solve this? Should treat the dataset before using it o on the hmm teacher or the dataset is ok like this?

I have used Accord.NET in the past and its really one of the best implementations for a HMM engine. However, when I trained my HMM, I passed the HMM parameters (namely PI, A and B) to the Baum Welch Teacher with the input data set supplied using an organized excel sheet. (similar to what Accord's author himself has used in his project). I somehow feel that since you are storing your data set as a multi-dimensional array and directly supplying it to the teacher, its unable to process it properly. Maybe you could supply one gesture record at a time or change the storage structure of your data set altogether. I advice going through the entire example of Accord if you haven't already because it worked just fine for me.

The problem might have been that the teaching algorithm expects the training sequences to be in the form double[12][32][18], rather than double[12][32,18]. The training data should be a collection of sequences of multivariate points. It should also be necessary to note that, if you have 11 possible classes of gestures, the integer labels given in the int[12] array should be comprised of values between 0 and 10 only.
Thus if you have 12 gesture samples, each containing 32 frames, and each frame is a vector of 18 points, you should be feeding the teacher with a double[12][32][18] array containing the observations and a int[12] array containing the expected class labels.
The example below, extracted from the HiddenMarkovClassifierLearning documentation page should help to give an idea how the vectors should be organized!
// Create a Continuous density Hidden Markov Model Sequence Classifier
// to detect a multivariate sequence and the same sequence backwards.
double[][][] sequences = new double[][][]
{
new double[][]
{
// This is the first sequence with label = 0
new double[] { 0, 1 },
new double[] { 1, 2 },
new double[] { 2, 3 },
new double[] { 3, 4 },
new double[] { 4, 5 },
},
new double[][]
{
// This is the second sequence with label = 1
new double[] { 4, 3 },
new double[] { 3, 2 },
new double[] { 2, 1 },
new double[] { 1, 0 },
new double[] { 0, -1 },
}
};
// Labels for the sequences
int[] labels = { 0, 1 };
In the above code, we have set the problem for 2 sequences of observations, where each sequence containing 5 observations, and in which each observations is comprised of 2 values. As you can see, this is a double[2][5][2] array. The array of class labels is given by a int[2], containing only values ranging from 0 to 1.
Now, to make the example more complete, we can continue creating and training the model using the following code:
var initialDensity = new MultivariateNormalDistribution(2);
// Creates a sequence classifier containing 2 hidden Markov Models with 2 states
// and an underlying multivariate mixture of Normal distributions as density.
var classifier = new HiddenMarkovClassifier<MultivariateNormalDistribution>(
classes: 2, topology: new Forward(2), initial: initialDensity);
// Configure the learning algorithms to train the sequence classifier
var teacher = new HiddenMarkovClassifierLearning<MultivariateNormalDistribution>(
classifier,
// Train each model until the log-likelihood changes less than 0.0001
modelIndex => new BaumWelchLearning<MultivariateNormalDistribution>(
classifier.Models[modelIndex])
{
Tolerance = 0.0001,
Iterations = 0,
FittingOptions = new NormalOptions()
{
Diagonal = true, // only diagonal covariance matrices
Regularization = 1e-5 // avoid non-positive definite errors
}
}
);
// Train the sequence classifier using the algorithm
double logLikelihood = teacher.Run(sequences, labels);
And now we can test the model, asserting that the output class label indeed matches what we are expecting:
// Calculate the probability that the given
// sequences originated from the model
double likelihood, likelihood2;
// Try to classify the 1st sequence (output should be 0)
int c1 = classifier.Compute(sequences[0], out likelihood);
// Try to classify the 2nd sequence (output should be 1)
int c2 = classifier.Compute(sequences[1], out likelihood2);

What is the use of the ArraySegment<T> class?

I just came across the ArraySegment<byte> type while subclassing the MessageEncoder class.
I now understand that it's a segment of a given array, takes an offset, is not enumerable, and does not have an indexer, but I still fail to understand its usage. Can someone please explain with an example?

ArraySegment<T> has become a lot more useful in .NET 4.5+ and .NET Core as it now implements:
IList<T>
ICollection<T>
IEnumerable<T>
IEnumerable
IReadOnlyList<T>
IReadOnlyCollection<T>
as opposed to the .NET 4 version which implemented no interfaces whatsoever.
The class is now able to take part in the wonderful world of LINQ so we can do the usual LINQ things like query the contents, reverse the contents without affecting the original array, get the first item, and so on:
var array = new byte[] { 5, 8, 9, 20, 70, 44, 2, 4 };
array.Dump();
var segment = new ArraySegment<byte>(array, 2, 3);
segment.Dump(); // output: 9, 20, 70
segment.Reverse().Dump(); // output 70, 20, 9
segment.Any(s => s == 99).Dump(); // output false
segment.First().Dump(); // output 9
array.Dump(); // no change

It is a puny little soldier struct that does nothing but keep a reference to an array and stores an index range. A little dangerous, beware that it does not make a copy of the array data and does not in any way make the array immutable or express the need for immutability. The more typical programming pattern is to just keep or pass the array and a length variable or parameter, like it is done in the .NET BeginRead() methods, String.SubString(), Encoding.GetString(), etc, etc.
It does not get much use inside the .NET Framework, except for what seems like one particular Microsoft programmer that worked on web sockets and WCF liking it. Which is probably the proper guidance, if you like it then use it. It did do a peek-a-boo in .NET 4.6, the added MemoryStream.TryGetBuffer() method uses it. Preferred over having two out arguments I assume.
In general, the more universal notion of slices is high on the wishlist of principal .NET engineers like Mads Torgersen and Stephen Toub. The latter kicked off the array[:] syntax proposal a while ago, you can see what they've been thinking about in this Roslyn page. I'd assume that getting CLR support is what this ultimately hinges on. This is actively being thought about for C# version 7 afaik, keep your eye on System.Slices.
Update: dead link, this shipped in version 7.2 as Span.
Update2: more support in C# version 8.0 with Range and Index types and a Slice() method.

Buffer partioning for IO classes - Use the same buffer for simultaneous
read and write operations and have a
single structure you can pass around
the describes your entire operation.
Set Functions - Mathematically speaking you can represent any
contiguous subsets using this new
structure. That basically means you
can create partitions of the array,
but you can't represent say all odds
and all evens. Note that the phone
teaser proposed by The1 could have
been elegantly solved using
ArraySegment partitioning and a tree
structure. The final numbers could
have been written out by traversing
the tree depth first. This would have
been an ideal scenario in terms of
memory and speed I believe.
Multithreading - You can now spawn multiple threads to operate over the
same data source while using segmented
arrays as the control gate. Loops
that use discrete calculations can now
be farmed out quite easily, something
that the latest C++ compilers are
starting to do as a code optimization
step.
UI Segmentation - Constrain your UI displays using segmented
structures. You can now store
structures representing pages of data
that can quickly be applied to the
display functions. Single contiguous
arrays can be used in order to display
discrete views, or even hierarchical
structures such as the nodes in a
TreeView by segmenting a linear data
store into node collection segments.
In this example, we look at how you can use the original array, the Offset and Count properties, and also how you can loop through the elements specified in the ArraySegment.
using System;
class Program
{
static void Main()
{
// Create an ArraySegment from this array.
int[] array = { 10, 20, 30 };
ArraySegment<int> segment = new ArraySegment<int>(array, 1, 2);
// Write the array.
Console.WriteLine("-- Array --");
int[] original = segment.Array;
foreach (int value in original)
{
Console.WriteLine(value);
}
// Write the offset.
Console.WriteLine("-- Offset --");
Console.WriteLine(segment.Offset);
// Write the count.
Console.WriteLine("-- Count --");
Console.WriteLine(segment.Count);
// Write the elements in the range specified in the ArraySegment.
Console.WriteLine("-- Range --");
for (int i = segment.Offset; i < segment.Count+segment.Offset; i++)
{
Console.WriteLine(segment.Array[i]);
}
}
}
ArraySegment Structure - what were they thinking?

What's about a wrapper class? Just to avoid copy data to temporal buffers.
public class SubArray<T> {
private ArraySegment<T> segment;
public SubArray(T[] array, int offset, int count) {
segment = new ArraySegment<T>(array, offset, count);
}
public int Count {
get { return segment.Count; }
}
public T this[int index] {
get {
return segment.Array[segment.Offset + index];
}
}
public T[] ToArray() {
T[] temp = new T[segment.Count];
Array.Copy(segment.Array, segment.Offset, temp, 0, segment.Count);
return temp;
}
public IEnumerator<T> GetEnumerator() {
for (int i = segment.Offset; i < segment.Offset + segment.Count; i++) {
yield return segment.Array[i];
}
}
} //end of the class
Example:
byte[] pp = new byte[] { 1, 2, 3, 4 };
SubArray<byte> sa = new SubArray<byte>(pp, 2, 2);
Console.WriteLine(sa[0]);
Console.WriteLine(sa[1]);
//Console.WriteLine(b[2]); exception
Console.WriteLine();
foreach (byte b in sa) {
Console.WriteLine(b);
}
Ouput:
3
4
3
4

The ArraySegment is MUCH more useful than you might think. Try running the following unit test and prepare to be amazed!
[TestMethod]
public void ArraySegmentMagic()
{
var arr = new[] {0, 1, 2, 3, 4, 5, 6, 7, 8, 9};
var arrSegs = new ArraySegment<int>[3];
arrSegs[0] = new ArraySegment<int>(arr, 0, 3);
arrSegs[1] = new ArraySegment<int>(arr, 3, 3);
arrSegs[2] = new ArraySegment<int>(arr, 6, 3);
for (var i = 0; i < 3; i++)
{
var seg = arrSegs[i] as IList<int>;
Console.Write(seg.GetType().Name.Substring(0, 12) + i);
Console.Write(" {");
for (var j = 0; j < seg.Count; j++)
{
Console.Write("{0},", seg[j]);
}
Console.WriteLine("}");
}
}
You see, all you have to do is cast an ArraySegment to IList and it will do all of the things you probably expected it to do in the first place. Notice that the type is still ArraySegment, even though it is behaving like a normal list.
OUTPUT:
ArraySegment0 {0,1,2,}
ArraySegment1 {3,4,5,}
ArraySegment2 {6,7,8,}

In simple words: it keeps reference to an array, allowing you to have multiple references to a single array variable, each one with a different range.
In fact it helps you to use and pass sections of an array in a more structured way, instead of having multiple variables, for holding start index and length. Also it provides collection interfaces to work more easily with array sections.
For example the following two code examples do the same thing, one with ArraySegment and one without:
byte[] arr1 = new byte[] { 1, 2, 3, 4, 5, 6 };
ArraySegment<byte> seg1 = new ArraySegment<byte>(arr1, 2, 2);
MessageBox.Show((seg1 as IList<byte>)[0].ToString());
and,
byte[] arr1 = new byte[] { 1, 2, 3, 4, 5, 6 };
int offset = 2;
int length = 2;
byte[] arr2 = arr1;
MessageBox.Show(arr2[offset + 0].ToString());
Obviously first code snippet is more preferred, specially when you want to pass array segments to a function.