We Keep Coding

Accord.Net Multiclass SVM DynamicTimeWarping Exception

I want to combine dynamic time warping and svm to use as a classifier. I use Accord .net, but there is something wrong with my code,here is my code:
double[][] inputs = new double[100][];
for(int i = 0; i < linesX.Length; i++)
{
inputs[i] = Array.ConvertAll(linesX[i].Split(','), Double.Parse);
}
int[] outputs = Array.ConvertAll(linesY, s => int.Parse(s));
// Create the Sequential Minimal Optimization learning algorithm
var smo = new MulticlassSupportVectorLearning<DynamicTimeWarping>()
{
// Set the parameters of the kernel
Kernel = new DynamicTimeWarping(alpha: 1, degree: 1)
};
// And use it to learn a machine!
var svm = smo.Learn(inputs, outputs);
// Now we can compute predicted values
int[] predicted = svm.Decide(inputs);
// And check how far we are from the expected values
double error = new ZeroOneLoss(outputs).Loss(predicted);
My inputs are (100,800), outputs are (100,1), there will be an exception at this line:var svm = smo.Learn(inputs, outputs);The exception is “System.AggregateException” happens in Accord.MachineLearning.dllWhat's wrong with my code

Please refer to the correct setup HERE. You're not assigning the Learner property.
Here's your modified code with some random input data:
static void Main(string[] args)
{
Random r = new Random();
double[][] inputs = new double[10][];
int[] outputs = new int[10];
for (int i = 0; i < 10; i++)
{
inputs[i] = new double[8];
for (int j = 0; j < 8; j++)
{
inputs[i][j] = r.Next(1, 100);
}
outputs[i] = r.Next(1, 6);
}
var smo = new MulticlassSupportVectorLearning<DynamicTimeWarping>()
{
Learner = (param) => new SequentialMinimalOptimization<DynamicTimeWarping>()
{
Kernel = new DynamicTimeWarping(alpha: 1, degree: 1),
}
};
var svm = smo.Learn(inputs, outputs);
int[] predicted = svm.Decide(inputs);
double error = new ZeroOneLoss(outputs).Loss(predicted);
Console.WriteLine();
Console.WriteLine("output = \n{0}", Matrix.ToString(outputs));
Console.WriteLine();
Console.WriteLine("predicted = \n{0}", Matrix.ToString(predicted));
Console.WriteLine();
Console.WriteLine("error = {0}", error);
Console.ReadLine();
}
Which will produce something like this:
output =
2 3 1 2 1 2 2 3 5 1
predicted =
2 1 1 2 1 2 2 2 2 1
error = 0.3

Write 3 arrays to 1

I have 3 arrays: x, y and z. How can I write all three to one in one-after-one mode?
For example: I have X=1,2 Y=3,4 Z=5,6
I need an array W = X[1],Y[1],Z[1],X[2],Y[2],Z[2] ... X[n],Y[n],Z[n]
float[] posX = new float[rmv2.lod[0].VerticesCount[0]];
float[] posY = new float[rmv2.lod[0].VerticesCount[0]];
float[] posZ = new float[rmv2.lod[0].VerticesCount[0]];
for (int i = 0; i < 1; i += 3)
{
posX[i] = rmv2.lod[0].Mesh[0].Vertex[i].X;
posY[i + 1] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
posZ[i + 2] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
}

I don't see the w array you're trying to write the output to. Are you looking for something like this?
int n = rmv2.lod[0].VerticesCount[0];
float[] posX = new float[n];
float[] posY = new float[n];
float[] posZ = new float[n];
float[] w = new float[n * 3];
for (int i = 0; i < n; i++)
{
posX[i] = rmv2.lod[0].Mesh[0].Vertex[i].X;
posY[i] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
posZ[i] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
w[i * 3 + 0] = rmv2.lod[0].Mesh[0].Vertex[i].X;
w[i * 3 + 1] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
w[i * 3 + 2] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
}

Perhaps you should look at jagged arrays...
https://msdn.microsoft.com/en-us/library/2s05feca.aspx

Try this:
float[] x = new float[] { 1, 2, 9 };
float[] y = new float[] { 3, 4 };
float[] z = new float[] { 5, 6 };
int max = Math.Max(x.Length, Math.Max(y.Length, z.Length));
List<float> combined = new List<float>();
for (int i = 0; i < max; i++)
{
if (x.Length > i) combined.Add(x[i]);
if (y.Length > i) combined.Add(y[i]);
if (z.Length > i) combined.Add(z[i]);
}
Produces a list with the combined arrays, regardless of their original lengths.
1, 3, 5, 2, 4, 6, 9

If I right understand you, try this solution. With this method arrays can have different length
static T[] ConcatOneAfterOne<T>(params T[][] arrays)
{
int outputArrayLength = 0;
for (int i = 0; i < arrays.Length; i++)
{
outputArrayLength += arrays[i].Length;
}
T[] output = new T[outputArrayLength];
int outputIndex = 0;
int sourceIndex = 0;
while (outputIndex != outputArrayLength)
{
for (int arrayIndex = 0; arrayIndex < arrays.Length; arrayIndex++)
{
if (sourceIndex < arrays[arrayIndex].Length)
{
output[outputIndex++] = arrays[arrayIndex][sourceIndex];
}
}
sourceIndex++;
}
return output;
}
[Test]
static void ConcatOneAfterOneTest()
{
int[] result = ConcatOneAfterOne(new[] { 1, 2 }, new[] { 3, 4 }, new[] { 5, 6 });
CollectionAssert.AreEqual(new int[] { 1, 3, 5, 2, 4, 6 }, result);
}

How does ACCORDs Deep neural learning work?

I get an error when I try to run DeepNeuralLearning from ACCORDs toolbox. It works when I run it as:
var inputs = new double[3][];
inputs[0] = new double[] {1, 2, 3};
inputs[1] = new double[] {1, 2, 3};
inputs[2] = new double[] {1, 2, 3};
var classes = new[] {0, 1, 2};
var outputs = Tools.Expand(classes, -1, 1);
var network = new DeepBeliefNetwork(3, 4, 3, 2);
var teacher = new DeepNeuralNetworkLearning(network)
{
Algorithm = (ann, i) => new ParallelResilientBackpropagationLearning(ann),
LayerIndex = network.Layers.Length - 1
};
var error = teacher.RunEpoch(inputs, outputs);
However, when I tweak the code to accept a 21 x 30,000 matrix for inputs and a 5 x 30000 matrix for outputs, it gives me an index was out of bounds exception. This is very strange as alll I did was change the matrix size. I tried changing the numbers for hidden layers but to no resolve. Anyone have any ideas what I am doing wrong?
double[][] inputs;
int[] classes;
var outputsList = Outputs(out inputs, out classes);
GetPerformanceOfStock(outputsList, inputs, classes);
//reclassify as 0 -4
for (var i = 0; i < classes.Length; i++)
{
classes[i] = classes[i] - 1;
}
//mean subtract
for (var i = 0; i < inputs.Length; i++)
{
var avg = inputs[i].Average();
for (var j = 0; j < inputs[i].Length; j++)
{
inputs[i][j] = inputs[i][j] - avg;
}
}
var outputs = Tools.Expand(classes, -1, 1);
//SPLIT INTO TEST AND TRAINIG DATA
var trainingIndex = (int) Math.Floor(.1*inputs.Length);
var fullsize = inputs.Length;
var trainingInputs = new double[trainingIndex][];
var trainingOutputs = new double[trainingIndex][];
for (var i = 0; i < trainingIndex; i++)
{
trainingInputs[i] = inputs[i];
trainingOutputs[i] = outputs[i];
}
var testingInputs = new double[fullsize - trainingIndex][];
var testingOutputs = new double[fullsize - trainingIndex][];
var counter = 0;
for (var i = fullsize - 1; i >= trainingIndex; i--)
{
testingInputs[counter] = inputs[i];
testingOutputs[counter] = outputs[i];
counter++;
}
//Inmitialize network
var network = new DeepBeliefNetwork(inputs.Length, 400, 3, 2);
//var network = new DeepBeliefNetwork(new BernoulliFunction(), trainingInputs.Length, 50, 25, 10);
var teacher = new DeepNeuralNetworkLearning(network)
{
Algorithm = (ann, i) => new ParallelResilientBackpropagationLearning(ann),
LayerIndex = network.Layers.Length - 1
};
teacher.RunEpoch(inputs, outputs);

How can I emulate Microsoft Excel's Solver functionality (GRG Nonlinear) in C#?

I have a non-linear optimization problem with constraints. It can be solved in Microsoft Excel with the Solver add-in, but I am having trouble replicating that in C#.
My problem is shown in the following spreadsheet. I am solving the classic A x = b problem but with the caveat that all components of x must be non-negative. So instead of using standard linear algebra I use Solver with the non-negative constraint, minimizing the sum of the squared differences, and get a reasonable solution. I have tried to replicate this in C# using either Microsoft Solver Foundation or Solver SDK. However I can't seem to get anywhere with them because with MSF I can't figure out how to define the goal and with Solver SDK I always get back status "optimal" and a solution of all 0s which is definitely not even a local minimum.
Here is my code for Solver SDK:
static double[][] A = new double[][] { new double[] { 1, 0, 0, 0, 0 }, new double[] { 0.760652602, 1, 0, 0, 0 }, new double[] { 0.373419404, 0.760537565, 1, 0, 0 }, new double[] { 0.136996731, 0.373331934, 0.760422587, 1, 0 }, new double[] { 0.040625222, 0.136953801, 0.373244464, 0.76030755, 1 } };
static double[][] b = new double[][] { new double[] { 2017159 }, new double[] { 1609660 }, new double[] { 837732.8125 }, new double[] { 330977.3125 }, new double[] { 87528.38281 } };
static void Main(string[] args)
{
using(Problem problem = new Problem(Solver_Type.Minimize, 5, 0))
{
problem.VarDecision.LowerBound.Array = new double[] { 0.0, 0.0, 0.0, 0.0, 0.0 };
problem.VarDecision.UpperBound.Array = new double[] { Constants.PINF, Constants.PINF, Constants.PINF, Constants.PINF, Constants.PINF };
problem.Evaluators[Eval_Type.Function].OnEvaluate += new EvaluateEventHandler(SumOfSquaredErrors);
problem.ProblemType = Problem_Type.OptNLP;
problem.Solver.Optimize();
Optimize_Status status = problem.Solver.OptimizeStatus;
Console.WriteLine(status.ToString());
foreach(double x in problem.VarDecision.FinalValue.Array)
{
Console.WriteLine(x);
}
}
}
static Engine_Action SumOfSquaredErrors(Evaluator evaluator)
{
double[][] x = new double[evaluator.Problem.Variables[0].Value.Array.Length][];
for(int i = 0; i < x.Length; i++)
{
x[i] = new double[1] { evaluator.Problem.Variables[0].Value.Array[i] };
}
double[][] b_calculated = MatrixMultiply(A, x);
double sum_sq = 0.0;
for(int i = 0; i < b_calculated.Length; i++)
{
sum_sq += Math.Pow(b_calculated[i][0] - b[i][0], 2);
}
evaluator.Problem.FcnObjective.Value[0] = sum_sq;
return Engine_Action.Continue;
}
static double[][] MatrixMultiply(double[][] left, double[][] right)
{
if(left[0].Length != right.Length)
{
throw new ArgumentException();
}
double[][] sum = new double[left.Length][];
for(int i = sum.GetLowerBound(0); i <= sum.GetUpperBound(0); i++)
{
sum[i] = new double[right[i].Length];
}
for(int i = 0; i < sum.Length; i++)
{
for(int j = 0; j < sum[0].Length; j++)
{
for(int k = 0; k < right.Length; k++)
{
sum[i][j] += left[i][k] * right[k][j];
}
}
}
return sum;
}
I don't have any code for Microsoft Solver Foundation because I don't think the goal function can be written in a single line and it doesn't allow for delegates like Solver SDK does.

One alternative would be to formulate this as an LP problem:
minimize sum of the elements in x
subject to Ax >= b
This should be fairly simple to formulate using Solver Foundation, based on one of the LP samples.
UPDATE JULY 5
The above approach also looks overly complex, but maybe this is due to the Frontline Solver API. Using Microsoft Solver Foundation, and minimizing the sum of squared differences, the following program:
private static void Main(string[] args)
{
var solver = SolverContext.GetContext();
var model = solver.CreateModel();
var A = new[,]
{
{ 1, 0, 0, 0, 0 },
{ 0.760652602, 1, 0, 0, 0 },
{ 0.373419404, 0.760537565, 1, 0, 0 },
{ 0.136996731, 0.373331934, 0.760422587, 1, 0 },
{ 0.040625222, 0.136953801, 0.373244464, 0.76030755, 1 }
};
var b = new[] { 2017159, 1609660, 837732.8125, 330977.3125, 87528.38281 };
var n = A.GetLength(1);
var x = new Decision[n];
for (var i = 0; i < n; ++i)
model.AddDecision(x[i] = new Decision(Domain.RealNonnegative, null));
// START NLP SECTION
var m = A.GetLength(0);
Term goal = 0.0;
for (var j = 0; j < m; ++j)
{
Term Ax = 0.0;
for (var i = 0; i < n; ++i) Ax += A[j, i] * x[i];
goal += Model.Power(Ax - b[j], 2.0);
}
model.AddGoal(null, GoalKind.Minimize, goal);
// END NLP SECTION
var solution = solver.Solve();
Console.WriteLine("f = {0}", solution.Goals.First().ToDouble());
for (var i = 0; i < n; ++i) Console.WriteLine("x[{0}] = {1}", i, x[i].GetDouble());
}
generates the following solution, which should be in line with the solution from the linked Excel sheet:
f = 254184688.179922
x[0] = 2017027.31820845
x[1] = 76226.6063397686
x[2] = 26007.3375581303
x[3] = 1.00650383558278E-07
x[4] = 4.18546775823669E-09
If I am not mistaken, unlike GRG, Solver Foundation cannot support general non-linear constraints out-of-the-box, I believe you will need additional plug-ins to handle these. For your problem, this is of course not an issue.
For completeness, to formulate the LP problem instead, exchange the code between START NLP SECTION and END NLP SECTION with the following code:
var m = A.GetLength(0);
var constraints = new Term[m];
for (var j = 0; j < m; ++j)
{
Term Ax = 0.0;
for (var i = 0; i < n; ++i) Ax += A[j, i] * x[i];
model.AddConstraint(null, constraints[j] = Model.GreaterEqual(Ax, b[j]));
}
model.AddGoal(null, GoalKind.Minimize, Model.Sum(x));
which will yield the following output (note that objective functions are different in the two cases, hence the large differences in f):
f = 2125502.27815564
x[0] = 2017159
x[1] = 75302.7580022821
x[2] = 27215.9247379241
x[3] = 5824.5954154355
x[4] = 0

matrix to array c#

Which would be the most efficient way to convert a squared matrix like
1 2 3
4 5 6
7 8 9
into
[1 2 3 4 5 6 7 8 9]
in c#
I was doing
int[,] array2D = new int[,] { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } };
int[] array1D = new int[9];
int ci=0;
for (int i = 0; i < 3; i++)
{
for (int j = 0; j < 3; j++)
{
array1D[ci++] = array2D[i, j]);
}
}

LINQ makes this trivial.
int[,] array2d = ...;
var array1d = array2d.Cast<int>().ToArray();
Otherwise, your way is adequate but could be generalized:
int[,] array2d = ...;
var rows = array2d.GetLength(0);
var cols = array2d.GetLength(1);
var array1d = new int[rows * cols];
var current = 0;
for (int i = 0; i < rows; i++)
{
for (int j = 0; j < cols; j++)
{
array1d[current++] = array2d[i, j];
}
}
Or even:
int[,] array2d = ...;
var array1d = new int[array2d.GetLength(0) * array2d.GetLength(1)];
var current = 0;
foreach (var value in array2d)
{
array1d[current++] = value;
}

As Jeff said, LINQ makes this trivial. OfType<>() should generally be a little faster than Cast<> though:
array1D = array2D.OfType<int>().ToArray();
The implementation of OfType<> however will still suffer from boxing/unboxing penalties, as #phoog mentioned.
Just for the fun of it, if you want a fast LINQ-based solution (avoiding the cost of boxing) you could use this small extension method:
static class LinqEx
{
public static IEnumerable<T> Flatten<T>(this T[,] matrix)
{
foreach (var item in matrix) yield return item;
}
}
Or this, based on Jeff's 2nd solution:
public static IEnumerable<T> Flatten<T>(this T[,] matrix)
{
var rows = matrix.GetLength(0);
var cols = matrix.GetLength(1);
for (var i = 0; i < rows;i++ )
{
for (var j = 0; j < cols; j++ )
yield return matrix[i, j];
}
}
usage:
int[,] array2D = new int[,] { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } };
int[] array1D = array2D.Flatten().ToArray();
I didn't fully profile this but I expect this will get you much better performance than the built-in options based on LINQ/IEnumerable. Jeff's second solution will however always be the fasted, it seems.

Alternate solution using Buffer.BlockCopy:
int[,] array2D = new int[,] { { 1, 2, 3 }, { 4, 5, 6 }, { 7, 8, 9 } };
int[] array1D = new int[ array2D.Length ];
Buffer.BlockCopy(array2D, 0, array1D, 0, array1D.Length * sizeof(int));

You're always better off allocating the complete result array in one hit, then copying the data in.
You should find the total size like this;
var size = arrays.Sum(a=> a.Length);
var result = new int[size];
And then copy the arrays using Array.CopyTo, instead of looping yourself;
var cursor = 0;
foreach(var a in arrays) {
a.CopyTo(result, cursor);
cursor += a.Length;
}
Array.CopyTo will be faster than your own loop; at least, not slower. It will probably use C's memcpy function internally to do a low-level block copy. This is as efficient as you can be.