We Keep Coding

How can I make CNTK solve the XOR problem?

I'm trying to make an implementation of a XOR in CNTK in C#. I followed the example here: https://github.com/microsoft/CNTK/blob/release/latest/Examples/TrainingCSharp/Common/LogisticRegression.cs
, but it doesn't work. After training, the neural network gives the wrong result:
For comparison, I run the same architecture using AForge and after training it gives the correct result:
CNTK result
AForge result
int iterations = 10000;
// initialize input and output values
double[][] input = new double[4][] {
new double[] {0, 0}, new double[] {0, 1},
new double[] {1, 0}, new double[] {1, 1}
};
double[][] output = new double[4][] {
new double[] {0}, new double[] {1},
new double[] {1}, new double[] {0}
};
int inputDim = 2;
int hiddenDim = 2;
int numOutputClasses = 1;
DeviceDescriptor device = DeviceDescriptor.CPUDevice;
Variable inputVariable1 = Variable.InputVariable(new int[] { inputDim }, DataType.Float);
Variable outputVariable1 = Variable.InputVariable(new int[] { hiddenDim }, DataType.Float);
Variable inputVariable2 = Variable.InputVariable(new int[] { hiddenDim }, DataType.Float);
Variable outputVariable2 = Variable.InputVariable(new int[] { numOutputClasses }, DataType.Float);
var weightParam1 = new Parameter(new int[] { hiddenDim, inputDim }, DataType.Float, 1, device, "w");
var biasParam1 = new Parameter(new int[] { hiddenDim }, DataType.Float, 0, device, "b");
var classifierOutput0 = CNTKLib.Sigmoid(CNTKLib.Times(weightParam1, inputVariable1) + biasParam1);
var weightParam2 = new Parameter(new int[] { numOutputClasses, hiddenDim }, DataType.Float, 1, device, "ww");
var biasParam2 = new Parameter(new int[] { numOutputClasses }, DataType.Float, 0, device, "bb");
var classifierOutput1 = CNTKLib.Sigmoid(CNTKLib.Times(weightParam2, classifierOutput0) + biasParam2);
//var loss = CNTKLib.CrossEntropyWithSoftmax(classifierOutput1, outputVariable2);
var loss = CNTKLib.BinaryCrossEntropy(classifierOutput1, outputVariable2);
var evalError = CNTKLib.ClassificationError(classifierOutput1, outputVariable2);
// prepare for training
CNTK.TrainingParameterScheduleDouble learningRatePerSample = new CNTK.TrainingParameterScheduleDouble(0.01, 1);
IList<Learner> parameterLearners = new List<Learner>() { Learner.SGDLearner(classifierOutput1.Parameters(), learningRatePerSample) };
var trainer = Trainer.CreateTrainer(classifierOutput1, loss, evalError, parameterLearners);
float[] inputValuesArr = new float[input.Length * input[0].Length];
for (int i = 0; i < input.Length; i++)
{
for(int k = 0; k < input[i].Length; k++)
{
inputValuesArr[i * input[i].Length + k] = (float)input[i][k];
}
}
float[] outputValuesArr = new float[output.Length * output[0].Length];
for (int i = 0; i < output.Length; i++)
{
for (int k = 0; k < output[i].Length; k++)
{
outputValuesArr[i * output[i].Length + k] = (float)output[i][k];
}
}
Value inputValues= Value.CreateBatch<float>(new int[] { inputDim }, inputValuesArr, device);
Value outputValues = Value.CreateBatch<float>(new int[] { numOutputClasses }, outputValuesArr, device);
// train the model
for (int minibatchCount = 0; minibatchCount < iterations; minibatchCount++)
{
//TODO: sweepEnd should be set properly instead of false.
#pragma warning disable 618
trainer.TrainMinibatch(new Dictionary<Variable, Value>() { { inputVariable1, inputValues }, { outputVariable2, outputValues } }, device);
#pragma warning restore 618
//TestHelper.PrintTrainingProgress(trainer, minibatchCount, updatePerMinibatches);
}
var inputDataMap = new Dictionary<Variable, Value>() { { inputVariable1, inputValues } };
var outputDataMap = new Dictionary<Variable, Value>() { { classifierOutput1.Output, null } };
classifierOutput1.Evaluate(inputDataMap, outputDataMap, device);
var outputValue = outputDataMap[classifierOutput1.Output];
IList<IList<float>> actualLabelSoftMax = outputValue.GetDenseData<float>(classifierOutput1.Output);
I wrote an XOR implementation, the trained network gave the result: {0.1, 0.6, 0.6, 0.7} instead of {0, 1, 1, 0}
I hope someone can help.
Update: I found an example with CNTK for c#, it works, and its structure has such layers: (2-8-1), if you change the structure to (2-2-1), and change the activation functions to sigmoidal, it starts to produce the same result as my code. This is strange, because a neural network with the same structure (2-2-1) of the AForge library does the right result.sample CNTK C# XOR

How to process data in passed to function in a channel?

I wrote a function that will split the array passed to it into other arrays, depending on the number 1 of the array. For example 3, 2, 1, 1, 8, 2, 2, 9, 9 will be turned into [2, 1, 1], [8], [9, 9].
static async Task Main(string[] args) {
List<int> list = new List<int>() { 1, 2, 2, 5, 6, 3, 9, 9, 9 };
var source1 = Try(list);
foreach (var x in source1)
{
x.ForEach(i => Console.Write("{0} ", i));
Console.WriteLine();
}
}
static List<List<int>> Try(List<int> data, int d = 0)
{
var myChannel = Channel.CreateUnbounded<int>();
List<List<int>> outList = new List<List<int>>();
for (int p = 0; p < data.Count;)
{
d = data[p++]; // Get the size of the array
d = (d <= 0) ? 0 : d; // If the size is not correct, then we correct
List<int> tempList = new List<int>(new int[d]); //creating a temp list
for (var i = 0; i < d && p < data.Count; i++, p++)
tempList[i] = data[p]; // Filling the array with data
outList.Add(tempList); // Result, Combining split arrays into one single sheet
}
return outList;
}
I want to rewrite the code above to use Сhannel. That is, the input of the function receives a Сhannel with a sequence of numbers. The function processes them and returns the separated arrays in turn.
But how can I do it using Channel?
And I wanted to try to read the size of the transmitted Сhannel, but it's not possible, as well as iterate over the elements of the transmitted Сhannel. I wanted to read them, then assign them to the List, process them and return the separated arrays one by one.
List<int> list = new List<int>() { 1, 2, 2, 5, 6, 3, 9, 9, 9 };
var myChannel = Channel.CreateUnbounded<int>() ;
_ = Task.Factory.StartNew(async () =>
{
foreach (var j in list)
{
await myChannel.Writer.WriteAsync(j);
await Task.Delay(500);
}
});
await foreach (var item in FetchItems2(myChannel))
{
Console.WriteLine($"{item}");
}
static async IAsyncEnumerable<int[]> FetchItems2(Channel<int> data, int d = 0)
{
List<int> innerList = new List<int>();
/*broken code taken from a past solution without using channels
var item = await data.Reader.ReadAsync();
for (int p = 0; p < data.Count;)
{
d = data[p++]; // Get the size of the array
d = (d <= 0) ? 0 : d; // If the size is not correct, then we correct
int[] arr = new int[d]; // Create an array
for (var i = 0; i < d && p < data.Count; i++, p++)
arr[i] = data[p]; // Filling the array with data
*/
yield return arr; // returning split arrays
}
}

Find a solution:
1 function was finalized and await was added to it when passing through all elements
static async Task Main(string[] args)
{
List<int> list = new List<int>() { 1, 2, 2, 5, 6, 3, 9, 9, 9 };
var myChannel = Channel.CreateUnbounded<int>() ;
_ = Task.Factory.StartNew(async () =>
{
foreach (var j in list)
{
await myChannel.Writer.WriteAsync(j);
await Task.Delay(100); // just to see
}
}); // data in channel
Console.WriteLine($"Start");
await foreach (var item in FetchItems2(myChannel, list.Count))
{
//Console.WriteLine($"{item}");
foreach(var x in item)
{
Console.Write($"{x} ");
}
Console.WriteLine();
}
Console.WriteLine($"End");
}
static async IAsyncEnumerable<int[]> FetchItems2(Channel<int> data, int size, int d = 0)
{
List<int> innerList = new List<int>();
for(int i = 0; i < size; i++)
{
innerList.Add(await data.Reader.ReadAsync());
}
for (int p = 0; p < innerList.Count;)
{
d = innerList[p++]; // Get the size of the array
d = (d <= 0) ? 0 : d; // If the size is not correct, then we correct
int[] arr = new int[d]; // Create an array
// create new list
for (var i = 0; i < d && p < innerList.Count; i++, p++)
arr[i] = innerList[p]; // Filling the array with data
await Task.Delay(750);
yield return arr; // Result
}
}
The code is almost the same. In another way, data is written from the channel to the List.
The array is read correctly, then outputs the data line by line via yield

calculate combination in C# without recursive function

i would like to calculate all the combination possible, but i don't want to use recursive function just some simple code in C# ?
Here is example :
col1 col2 col3
valueinf= 5 6 7
valuesup= 10 20 30
i want this for result:
res1 = 5+6+7
res2 = 5+6+30
res3 = 5+20+7
res4 = 5+20+30
res5 = 10+6+7
res6 = 10+6+30
res7 = 10+20+7
res8 = 10+20+30
the number of colonne is variable but i wil allways have 2 lines one inferior and one superior.
I don't know how to begin the algorithm. Some help will be gratefull just to begin

Well, will-nilly you have to loop:
using System.Linq;
...
private static IEnumerable<int> Sums(IEnumerable<int[]> columns) {
if (columns is null)
throw new ArgumentNullException(nameof(columns));
int[][] data = columns.ToArray();
int[] pos = new int[data.Length];
do {
yield return Enumerable.Range(0, data.Length).Sum(i => data[i][pos[i]]);
for (int i = pos.Length - 1; i >= 0; --i)
if (pos[i] == data[i].Length - 1)
pos[i] = 0;
else {
pos[i] += 1;
break;
}
}
while (!pos.All(p => p == 0));
}
The very same idea if you want to print formulae:
private static IEnumerable<string> Formulae(IEnumerable<int[]> columns) {
if (columns is null)
throw new ArgumentNullException(nameof(columns));
int[][] data = columns.ToArray();
int[] pos = new int[data.Length];
do {
yield return string.Join("+", Enumerable
.Range(0, data.Length).Select(i => data[i][pos[i]]));
for (int i = pos.Length - 1; i >= 0; --i)
if (pos[i] == data[i].Length - 1)
pos[i] = 0;
else {
pos[i] += 1;
break;
}
}
while (!pos.All(p => p == 0));
}
Demo:
List<int[]> columns = new List<int[]>() {
new int[] {5, 10},
new int[] {6, 20},
new int[] {7, 30},
};
var sums = Sums(columns);
var formulae = Formulae(columns);
Console.WriteLine(string.Join(Environment.NewLine, sums
.Select((v, i) => $"res{i+1} = {v}")));
Console.WriteLine();
Console.WriteLine(string.Join(Environment.NewLine, formulae
.Select((v, i) => $"res{i+1} = {v}")));
Outcome:
res1 = 18
res2 = 41
res3 = 32
res4 = 55
res5 = 23
res6 = 46
res7 = 37
res8 = 60
res1 = 5+6+7
res2 = 5+6+30
res3 = 5+20+7
res4 = 5+20+30
res5 = 10+6+7
res6 = 10+6+30
res7 = 10+20+7
res8 = 10+20+30
Edit: To find the best fit you can exploit the very same principle:
private static (int[] pos, int diff, string formula) BestFit(int target, IEnumerable<int[]> columns) {
if (columns is null)
throw new ArgumentNullException(nameof(columns));
int[][] data = columns.ToArray();
int[] pos = new int[data.Length];
int bestDiff = -1;
int[] bestPos = null;
do {
int diff = Math.Abs(target - Enumerable.Range(0, data.Length).Sum(i => data[i][pos[i]]));
if (diff < bestDiff || bestPos == null) {
bestPos = pos.ToArray();
bestDiff = diff;
}
for (int i = pos.Length - 1; i >= 0; --i)
if (pos[i] == data[i].Length - 1)
pos[i] = 0;
else {
pos[i] += 1;
break;
}
}
while (!pos.All(p => p == 0));
return (
bestPos,
bestDiff,
string.Join("+", Enumerable.Range(0, data.Length).Select(i => data[i][bestPos[i]]))
);
}
Demo:
int[][] data = new int[][] {
new int[] {5, 10},
new int[] {6, 20},
new int[] {7, 30},
};
var fit = BestFit(20, data);
Console.Write($"delta = {fit.diff}; formula = {fit.formula}");
Outcome:
delta = 2; formula = 5+6+7

Accord.Net Multiclass SVM DynamicTimeWarping Exception

I want to combine dynamic time warping and svm to use as a classifier. I use Accord .net, but there is something wrong with my code,here is my code:
double[][] inputs = new double[100][];
for(int i = 0; i < linesX.Length; i++)
{
inputs[i] = Array.ConvertAll(linesX[i].Split(','), Double.Parse);
}
int[] outputs = Array.ConvertAll(linesY, s => int.Parse(s));
// Create the Sequential Minimal Optimization learning algorithm
var smo = new MulticlassSupportVectorLearning<DynamicTimeWarping>()
{
// Set the parameters of the kernel
Kernel = new DynamicTimeWarping(alpha: 1, degree: 1)
};
// And use it to learn a machine!
var svm = smo.Learn(inputs, outputs);
// Now we can compute predicted values
int[] predicted = svm.Decide(inputs);
// And check how far we are from the expected values
double error = new ZeroOneLoss(outputs).Loss(predicted);
My inputs are (100,800), outputs are (100,1), there will be an exception at this line:var svm = smo.Learn(inputs, outputs);The exception is “System.AggregateException” happens in Accord.MachineLearning.dllWhat's wrong with my code

Please refer to the correct setup HERE. You're not assigning the Learner property.
Here's your modified code with some random input data:
static void Main(string[] args)
{
Random r = new Random();
double[][] inputs = new double[10][];
int[] outputs = new int[10];
for (int i = 0; i < 10; i++)
{
inputs[i] = new double[8];
for (int j = 0; j < 8; j++)
{
inputs[i][j] = r.Next(1, 100);
}
outputs[i] = r.Next(1, 6);
}
var smo = new MulticlassSupportVectorLearning<DynamicTimeWarping>()
{
Learner = (param) => new SequentialMinimalOptimization<DynamicTimeWarping>()
{
Kernel = new DynamicTimeWarping(alpha: 1, degree: 1),
}
};
var svm = smo.Learn(inputs, outputs);
int[] predicted = svm.Decide(inputs);
double error = new ZeroOneLoss(outputs).Loss(predicted);
Console.WriteLine();
Console.WriteLine("output = \n{0}", Matrix.ToString(outputs));
Console.WriteLine();
Console.WriteLine("predicted = \n{0}", Matrix.ToString(predicted));
Console.WriteLine();
Console.WriteLine("error = {0}", error);
Console.ReadLine();
}
Which will produce something like this:
output =
2 3 1 2 1 2 2 3 5 1
predicted =
2 1 1 2 1 2 2 2 2 1
error = 0.3

Write 3 arrays to 1

I have 3 arrays: x, y and z. How can I write all three to one in one-after-one mode?
For example: I have X=1,2 Y=3,4 Z=5,6
I need an array W = X[1],Y[1],Z[1],X[2],Y[2],Z[2] ... X[n],Y[n],Z[n]
float[] posX = new float[rmv2.lod[0].VerticesCount[0]];
float[] posY = new float[rmv2.lod[0].VerticesCount[0]];
float[] posZ = new float[rmv2.lod[0].VerticesCount[0]];
for (int i = 0; i < 1; i += 3)
{
posX[i] = rmv2.lod[0].Mesh[0].Vertex[i].X;
posY[i + 1] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
posZ[i + 2] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
}

I don't see the w array you're trying to write the output to. Are you looking for something like this?
int n = rmv2.lod[0].VerticesCount[0];
float[] posX = new float[n];
float[] posY = new float[n];
float[] posZ = new float[n];
float[] w = new float[n * 3];
for (int i = 0; i < n; i++)
{
posX[i] = rmv2.lod[0].Mesh[0].Vertex[i].X;
posY[i] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
posZ[i] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
w[i * 3 + 0] = rmv2.lod[0].Mesh[0].Vertex[i].X;
w[i * 3 + 1] = rmv2.lod[0].Mesh[0].Vertex[i].Y;
w[i * 3 + 2] = rmv2.lod[0].Mesh[0].Vertex[i].Z;
}

Perhaps you should look at jagged arrays...
https://msdn.microsoft.com/en-us/library/2s05feca.aspx

Try this:
float[] x = new float[] { 1, 2, 9 };
float[] y = new float[] { 3, 4 };
float[] z = new float[] { 5, 6 };
int max = Math.Max(x.Length, Math.Max(y.Length, z.Length));
List<float> combined = new List<float>();
for (int i = 0; i < max; i++)
{
if (x.Length > i) combined.Add(x[i]);
if (y.Length > i) combined.Add(y[i]);
if (z.Length > i) combined.Add(z[i]);
}
Produces a list with the combined arrays, regardless of their original lengths.
1, 3, 5, 2, 4, 6, 9

If I right understand you, try this solution. With this method arrays can have different length
static T[] ConcatOneAfterOne<T>(params T[][] arrays)
{
int outputArrayLength = 0;
for (int i = 0; i < arrays.Length; i++)
{
outputArrayLength += arrays[i].Length;
}
T[] output = new T[outputArrayLength];
int outputIndex = 0;
int sourceIndex = 0;
while (outputIndex != outputArrayLength)
{
for (int arrayIndex = 0; arrayIndex < arrays.Length; arrayIndex++)
{
if (sourceIndex < arrays[arrayIndex].Length)
{
output[outputIndex++] = arrays[arrayIndex][sourceIndex];
}
}
sourceIndex++;
}
return output;
}
[Test]
static void ConcatOneAfterOneTest()
{
int[] result = ConcatOneAfterOne(new[] { 1, 2 }, new[] { 3, 4 }, new[] { 5, 6 });
CollectionAssert.AreEqual(new int[] { 1, 3, 5, 2, 4, 6 }, result);
}