Nonlinear (weighted) least-squares estimates of the parameters of a nonlinear model - c#

I am trying to fit exponential data using a sigmoidal function (4PL) with the following formula:
y = a + (k-a) /(1 + exp((v-x)/c)
While with R I have good results, using C# and the framework Accord.Net I get very poor fit.
Here is my code:
var nls = new NonlinearLeastSquares()
{
NumberOfParameters = 4,
StartValues = new[] {0d, 40000d, 35d,1d }
Function = (parameters, input) =>
{
return parameters[0] + ((parameters[1] - parameters[0]) / ( 1 +
Math.Exp((parameters[2] - input[0] )/parameters[3]) ) );
},
Gradient = (parameters, input, result) =>
{
result[0] = 1 - ( 1 / (1 + Math.Exp((parameters[2] - input[0]) /
parameters[3]))); // d/da
result[1] = 1 / (1 + Math.Exp((parameters[2] - input[0]) /
parameters[3])); // d/dk
result[2] = -((parameters[1] - parameters[0])*
Math.Exp((parameters[2] - input[0]) / parameters[3])) /
parameters[3]*Math.Pow(1 + Math.Exp((parameters[2] -
input[0]) / parameters[3]),2); // d/dv
result[3] = ((parameters[1] - parameters[0]) * (parameters[2] -
input[0]) * Math.Exp((parameters[2] - input[0]) /
parameters[3])) / Math.Pow(parameters[3], 2) *
Math.Pow(1 + Math.Exp((parameters[2] - input[0]) /
parameters[3]), 2); // d/dc
},
Algorithm = new LevenbergMarquardt()
{
MaxIterations = 100,
Tolerance = 0
}
};
var regression = nls.Learn(inputs, outputs);
// The solution will be at:
double a = regression.Coefficients[0];
double k = regression.Coefficients[1];
double v = regression.Coefficients[2];
double c = regression.Coefficients[3];
I am stuck with this problem, any help would be really appreciate,

For whom interested I found out the solution.
The problem were the gradients of result[2] (d/dv) and result[3] (d/dc).
Parentheses were missing in the denominator.
result[2] = -((parameters[1] - parameters[0])*
Math.Exp((parameters[2] - input[0]) / parameters[3])) /
(parameters[3]*Math.Pow(1 + Math.Exp((parameters[2] -
input[0]) / parameters[3]),2)); // d/dv
result[3] = ((parameters[1] - parameters[0]) * (parameters[2] -
input[0]) * Math.Exp((parameters[2] - input[0]) /
parameters[3])) / (Math.Pow(parameters[3], 2) *
Math.Pow(1 + Math.Exp((parameters[2] - input[0]) /
parameters[3]), 2)); // d/dc

Related

Problem with calculating cubic equations in C#

Recently I was working on a discord bot to compute the roots of a cubic equation.
I have already made another program beforehand in C# that basically does the same thing, and is proven to work correctly, so I straight up copied the codes into the one which I am working with now.
(Suppose that I am calculating the roots of 7x3 + 2x2 + 12x + 9 = 0)
The code below is from the program which was created beforehand:
(Once again, the program and the results are proven to be correct, results are listed at the bottom of the code)*
// parts 1-5
double p1 = -(b / (3 * a));
double p2 = (b * c / (6 * Math.Pow(a, 2))) - (Math.Pow(b, 3) / (27 * Math.Pow(a, 3))) - (d / (2 * a));
double p3 = (c / (3 * a)) - (Math.Pow(b, 2) / (9 * Math.Pow(a, 2)));
Complex p4 = new Complex(-1.0 / 2, Math.Sqrt(3) / 2);
Complex p5 = new Complex(-1.0 / 2, -(Math.Sqrt(3) / 2));
// solve for roots
Complex root1 = p1 + Math.Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))) +
Math.Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)));
Complex root2 = p1 + (p4 * Math.Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)))) +
(p5 * Math.Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))));
Complex root3 = p1 + (p5 * Math.Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)))) +
(p4 * Math.Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))));
/*
results :
root1 = (-0.6566823203779361, 0)
root2 = (0.18548401733182518, 1.3868992549529795)
root3 = (0.18548401733182518, -1.3868992549529795)
*/
I proceed to copy the whole thing into the new program I am working on. Then I realized in my new program the Math.Cbrt function doesn't exist (I don't know why). Therefore I made a custom program to tackle this problem, here is the code in my new program :
(This is the problematic program, the results are different from the first one's.)
// parts 1-5
double p1 = -(b / (3 * a));
double p2 = (b * c / (6 * Math.Pow(a, 2))) - (Math.Pow(b, 3) / (27 * Math.Pow(a, 3))) - (d / (2 * a));
double p3 = (c / (3 * a)) - (Math.Pow(b, 2) / (9 * Math.Pow(a, 2)));
Complex p4 = new Complex(-1.0 / 2, Math.Sqrt(3.0) / 2);
Complex p5 = new Complex(-1.0 / 2, -(Math.Sqrt(3.0) / 2));
// solve for roots
Complex root1 = p1 + Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))) +
Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)));
Complex root2 = p1 + (p4 * Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)))) +
(p5 * Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))));
Complex root3 = p1 + (p5 * Cbrt(p2 + Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3)))) +
(p4 * Cbrt(p2 - Math.Sqrt(Math.Pow(p2, 2) + Math.Pow(p3, 3))));
public static Complex Cbrt(Complex value)
{
Complex result = Complex.Pow(value, 1D / 3);
return result;
}
/*
results :
root1 = (0.965490835755936, 0.936562108366076)
root2 = (0.185484017331825, -0.486224961779172)
root3 = (-1.43668913880205, -0.450337146586904)
*/
I have tried to extract different sections of the codes and run it separately but I still wasn't able to find what the cause is. If the codes are the same, why do they yield different results? Or is the Cbrt method I have created the real problem behind this?
Refering #mikuszefski 's comment,
The real cause behind this is that the Cbrt() function I have created calculates negative numbers wrongly.
For example, Cbrt(-1) should yield the result -1.
However, the answer I have gotten instead is (NaN, 0).
I have found the solution in another thread.

Having trouble in finding roots of a cubic equation with C#

Recently my friends and I decided to make a multi-functional calculator to automatically find
the roots of quadratic equations.
It wasn't that challenging after all, so we decided to go onto the next level, to make a calculator for a cubic equation.
(ax^3 + bx^2 + cx + d)
However, we stumbled across some trivial problems, and no matter how hard we tried, the results are still the same.
we are beginners in terms of coding, so we're not sure if we are actually making some stupid mistakes here
but at least we want to learn something from others.
Basically, we have tried a lot of combinations of different cubic equations, or even re-coding the whole thing. The problem is that the results we yield are always wrong, but only for the real parts of the second root and third root.
For a better understanding, we have tried 9x^3 + 8x^2 + 7x + 6, as an example.
The correct answer, according to a cubic equation calculator website, is
(The website)
First root = -0.87285
Second root = -0.00802 + 0.87391 i
Third root = -0.00802 - 0.87391 i
However, our result to this equation is :
First root = -0.87285
Second root = -0.2963 + 0.87391 i
Third root = -0.2963 + -0.87391 i
It is apparently noticeable that only parts of the second and third roots are wrong.
We have tried finding similar threads to help, but those are a little bit too difficult for us to understand or those problems are not the same as ours.
We look forward to finding the solution and the causes to this problem.
We have separated the formulas for finding the roots of a cubic equation into 5 parts.
(rootp1-rootp5)
The formulas are coded according to the formulas that can be found in the Chinese version of Wikipedia's page of Cubic equation.
(The formulas)
We have also rounded the real parts of the roots to 5 digits.
(Some code lines might be redundant, but as I have mentioned, we are new to coding)
Code (C#) :
using System;
using System.Numerics;
namespace ComplexNumbers
{
public class ComplexNo
{
public static void Main()
{
Console.WriteLine("Cubic Equation Calculator :");
Console.Write("Insert first coefficient : ");
double a = Convert.ToDouble(Console.ReadLine());
Console.Write("Insert second coefficient : ");
double b = Convert.ToDouble(Console.ReadLine());
Console.Write("Insert third coefficient : ");
double c = Convert.ToDouble(Console.ReadLine());
Console.Write("Insert constant : ");
double d = Convert.ToDouble(Console.ReadLine());
Console.WriteLine(" ");
Console.WriteLine("Solve for " + a + "x" + "^3 " + b + "x^2 " + c + "x " + d + " :");
double rootp1 = -(b / (3 * a));
double rootp2 = (b * c / (6 * Math.Pow(a, 2))) - (Math.Pow(b, 3) / (27 * Math.Pow(a, 3))) - (d / (2 * a));
double rootp3 = (c / (3 * a)) - (Math.Pow(b, 2) / (9 * Math.Pow(a, 2)));
Complex root1 = rootp1 + Math.Cbrt(rootp2 + Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3))) +
Math.Cbrt(rootp2 - Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3)));
Complex rootp4 = new Complex(-1 / 2, Math.Sqrt(3) / 2);
Complex rootp5 = new Complex(-1 / 2, -(Math.Sqrt(3) / 2));
Complex root2 = rootp1 + (rootp4 * Math.Cbrt(rootp2 + Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3)))) +
(rootp5 * Math.Cbrt(rootp2 - Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3))));
Complex root3 = rootp1 + (rootp5 * Math.Cbrt(rootp2 + Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3)))) +
(rootp4 * Math.Cbrt(rootp2 - Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3))));
Console.WriteLine(" ");
Console.WriteLine("Results :");
Console.WriteLine("First Root :");
string root1rp = Convert.ToString(Math.Round(root1.Real, 5));
string root1ip = Convert.ToString(Math.Round(root1.Imaginary, 5));
Console.WriteLine(root1rp + " + " + root1ip + "i");
Console.WriteLine("Second Root :");
string root2rp = Convert.ToString(Math.Round(root2.Real, 5));
string root2ip = Convert.ToString(Math.Round(root2.Imaginary, 5));
Console.WriteLine(root2rp + " + " + root2ip + "i");
Console.WriteLine("Third Root :");
string root3rp = Convert.ToString(Math.Round(root3.Real, 5));
string root3ip = Convert.ToString(Math.Round(root3.Imaginary, 5));
Console.WriteLine(root3rp + " + " + root3ip + "i");
Console.ReadLine();
}
}
}
(Sorry for making this thread so long and my bad grammar)
The problem is with this line Complex rootp4 = new Complex(-1 / 2, Math.Sqrt(3) / 2);. -1/2 uses integer division and evaluates to 0.
This code will work.
Console.WriteLine(" ");
Console.WriteLine("Solve for " + a + "x" + "^3 " + b + "x^2 " + c + "x " + d + " :");
double minusBover3a = -(b / (3 * a));
double rootp2 = (b * c / (6 * Math.Pow(a, 2))) - (Math.Pow(b, 3) / (27 * Math.Pow(a, 3))) - (d / (2 * a));
double rootp3 = (c / (3 * a)) - (Math.Pow(b, 2) / (9 * Math.Pow(a, 2)));
double bigCubeRootPlus = Math.Cbrt(rootp2 + Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3)));
double bigCubeRootMinus = Math.Cbrt(rootp2 - Math.Sqrt(Math.Pow(rootp2, 2) + Math.Pow(rootp3, 3)));
// ** THIS IS THE PROBLEM. "-1/2" uses integer division, so this complex has 0 for real part
Complex complexPlus = new Complex(-1.0 / 2, Math.Sqrt(3) / 2);
Complex complexMinus = new Complex(-1.0 / 2, -(Math.Sqrt(3) / 2));
Complex root1 = minusBover3a + bigCubeRootPlus + bigCubeRootMinus;
Complex root2 = minusBover3a + complexPlus * bigCubeRootPlus + complexMinus * bigCubeRootMinus;
Complex root3 = minusBover3a + complexMinus * bigCubeRootPlus + complexPlus * bigCubeRootMinus;
I admit I haven't tried your code, but you may have a casting issue in the math in your original calculations. Try this:
double rootp2 = ((double)b * (double)c / (6D * Math.Pow(a, 2D))) - (Math.Pow(b, 3D) / (27D * Math.Pow(a, 3D))) - ((double)d / (2D * (double)a));
If that makes a difference, you'd have to propagate a similar change (cast all variables as (double) and inline constants as double with D) through the other calculations.
The following is an alternative to your code:
// from https://www.daniweb.com/programming/software-development/
// code/454493/solving-the-cubic-equation-using-the-complex-struct
// algorithm described in
// https://en.wikipedia.org/wiki/Cubic_equation#General_cubic_formula
const int NRoots = 3;
double SquareRootof3 = Math.Sqrt(3);
// the 3 cubic roots of 1
var CubicUnity = new List<Complex>(NRoots)
{ new Complex(1, 0),
new Complex(-0.5, -SquareRootof3 / 2.0),
new Complex(-0.5, SquareRootof3 / 2.0) };
// intermediate calculations
double DELTA = 18 * a * b * c * d
- 4 * b * b * b * d
+ b * b * c * c
- 4 * a * c * c * c
- 27 * a * a * d * d;
double DELTA0 = b * b - 3 * a * c;
double DELTA1 = 2 * b * b * b
- 9 * a * b * c
+ 27 * a * a * d;
Complex DELTA2 = -27 * a * a * DELTA;
Complex C = Complex.Pow((DELTA1 + Complex.Pow(DELTA2, 0.5)) / 2, 1 / 3.0);
for (int i = 0; i < NRoots; i++)
{
Complex M = CubicUnity[i] * C;
Complex Root = -1.0 / (3 * a) * (b + M + DELTA0 / M);
Console.WriteLine();
Console.WriteLine($"Root {i+1}:");
Console.WriteLine($"Real {Root.Real:0.#####}");
Console.WriteLine($"Imaginary {Root.Imaginary:0.#####}i");
}

Swiss projection coordinates to ellipsoidal coordinates

I am trying to convert CH1903+ coordinates into WGS84 coordinates based on the following example provided by the swiss "Bundesamt für Statistik" using C#.
Example:
As far i can calculate all values like shown in the example. But in the end when i try to calculate the variable "S" based on the values in the example i am getting wrong results.
double E = 2679520.05;
double alpha = 1.00072913843038;
double phi = 0.820535226;
double b = 0.820535226;
double K = 0.0030667323772751
I tried both implementations:
double S = Math.Log(Math.Tan(Math.PI / 4.0 + phi / 2.0)); --> result: 0.931969600979248
or
double S = 1/alpha * (Math.Log(Math.PI/4 + b/2) - K) + E * Math.Log(Math.Tan((Math.PI/4) + (Math.Asin(E * Math.Sin(phi))/2.0))); --> result: NaN
Can somebody tell what's wrong in my implementation that i am getting this wrong results? If i understand the example correctly both calculations should return 0.933114264192610 as result for the given values.
This is my implementation for converting swiss CH1903+ coordinates -> decimal coordinates (WGS84) which works good so far
Example CH1903+ Coordinates N = 1249251.544, E = 2645320.487;
// swiss CH1903+ coordinates
double N = 1249251.544,
E = 2645320.487;
const double K = 0.0030667323772751; // Konstante der Breitenformel
const double BESSEL_GH = 6377397.155; // grosse Halbachse des Bessel-Ellipsoids
const double BESSEL_SH = 6356078.963; // kleine Halbachse des Bessel-Ellipsoids
double GRS_GH = 6.378137 * Math.Pow(10, 6); // grosse Halbachse des GRS84-Ellipsoids
double GRS_SH = 6.356752314 * Math.Pow(10, 6); ; // kleine Halbachse des GRS84-Ellipsoids
double E2_BESSEL = (Math.Pow(BESSEL_GH, 2) - Math.Pow(BESSEL_SH, 2)) / Math.Pow(BESSEL_GH, 2), // 1.numerische Exzentrizität (im Quadrat) des Bessel-Ellipsoids
E_BESSEL = Math.Sqrt(E2_BESSEL), // 1.numerische Exzentrizität des Bessel-Ellipsoids
E2_GRS = (Math.Pow(GRS_GH, 2) - Math.Pow(GRS_SH, 2)) / Math.Pow(GRS_GH, 2); // 1.numerische Exzentrizität (im Quadrat) des GRS84-Ellipsoids
const double TOLERANCE = 0.0000000000000001;
///// swiss coordinates -> ellipsoid coordinates /////
double
Y = E - 2600000.0, // swiss CH1903 coordinates
X = N - 1200000.0, // swiss CH1903 coordinates
PHI_NULL = (46 + 57 / 60.0 + 8.66 / 3600.0) * Math.PI / 180, // geogr. Breite des Nullpunkts in Bern
LAMBDA_NULL = (7 + 26 / 60.0 + 22.5 / 3600.0) * Math.PI / 180, // geogr. Länge des Nullpunkts in Bern
R = (BESSEL_GH * Math.Sqrt(1 - E2_BESSEL)) / (1 - E2_BESSEL * Math.Pow(Math.Sin(PHI_NULL), 2)), // Radius der Projektionskugel
ALPHA = Math.Sqrt(1 + E2_BESSEL / (1 - E2_BESSEL) * Math.Pow(Math.Cos(PHI_NULL), 4)), // Verhältnis Kugellänge zu Ellipsoidlänge
B_GLOBAL_NULL = Math.Asin(Math.Sin(PHI_NULL) / ALPHA), // Breite des Nullpunkts auf der Kugel
L_PSEUDO = Y / R, // Kugelkoordinaten bezüglich Pseudoäquatorsystem in Bern
B_PSEUDO = 2 * (Math.Atan(Math.Pow(Math.E, (X / R))) - (Math.PI / 4)), // Kugelkoordinaten bezüglich Pseudoäquatorsystem in Bern
L = Math.Atan(Math.Sin(L_PSEUDO) / (Math.Cos(B_GLOBAL_NULL) * Math.Cos(L_PSEUDO) - Math.Sin(B_GLOBAL_NULL) * Math.Tan(B_PSEUDO))), // Kugelkoordinaten bezüglich Nullpunkt Bern
B = Math.Asin(Math.Cos(B_GLOBAL_NULL) * Math.Sin(B_PSEUDO) + Math.Sin(B_GLOBAL_NULL) * Math.Cos(B_PSEUDO) * Math.Cos(L_PSEUDO)), // Kugelkoordinaten bezüglich Nullpunkt Bern
LAMBDA = LAMBDA_NULL + L / ALPHA,
PHI = B,
S = (Math.Log(Math.Tan(Math.PI / 4.0 + B / 2.0)) - K) / ALPHA + E_BESSEL * Math.Log(Math.Tan(Math.PI / 4.0 + Math.Asin(E_BESSEL * Math.Sin(PHI)) / 2.0));
bool cont = true;
// iterate to tolerance
while (cont)
{
double PHI_TMP = 2 * Math.Atan(Math.Pow(Math.E, S)) - Math.PI / 2;
double S_TMP = (Math.Log(Math.Tan(Math.PI / 4 + B / 2)) - K) / ALPHA + E_BESSEL * Math.Log(Math.Tan(Math.PI / 4 + Math.Asin(E_BESSEL * Math.Sin(PHI_TMP)) / 2));
if (Math.Abs(PHI - PHI_TMP) < TOLERANCE)
{
cont = false;
}
else
{
PHI = PHI_TMP;
S = S_TMP;
}
}
///// ellipsoid coordinates (CH1903) -> karth. coordinates (CH1903) /////
double RN_CH1903 = BESSEL_GH / (Math.Sqrt(1 - E2_BESSEL * Math.Pow(Math.Sin(PHI), 2))), // Normalkrümmungsradius
X_KARTH = RN_CH1903 * Math.Cos(PHI) * Math.Cos(LAMBDA),
Y_KARTH = RN_CH1903 * Math.Cos(PHI) * Math.Sin(LAMBDA),
Z_KARTH = (RN_CH1903 * (1 - E2_BESSEL)) * Math.Sin(PHI);
///// karth. coordinates (CH1903) -> karth. coordinates (ETRS89/CHTRS95) /////
double X_CH1930P = X_KARTH + 674.374,
Y_CH1930P = Y_KARTH + 15.056,
Z_CH1930P = Z_KARTH + 405.346;
///// karth. coordinates (ETRS89/CHTRS95) -> ellipsoid coordinates (ETRS89/CHTRS95) /////
double PHI_ETRS = Math.Atan(Z_CH1930P / Math.Sqrt(Math.Pow(X_CH1930P, 2) + Math.Pow(Y_CH1930P, 2))),
RN_ETRS = BESSEL_GH / (Math.Sqrt(1 - E2_GRS * Math.Pow(Math.Sin(PHI_ETRS), 2)));
cont = true;
// iterate to tolerance
while (cont)
{
double PHI_ETRS_TMP = Math.Atan((Z_CH1930P / (Math.Sqrt(Math.Pow(X_CH1930P, 2) + Math.Pow(Y_CH1930P, 2)))) / (1 - (RN_ETRS * E2_GRS) / (RN_ETRS + 0))),
RN_ETRS_TMP = BESSEL_GH / (Math.Sqrt(1 - E2_GRS * Math.Pow(Math.Sin(PHI_ETRS_TMP), 2)));
if (Math.Abs(PHI_ETRS - PHI_ETRS_TMP) < TOLERANCE)
{
cont = false;
}
else
{
PHI_ETRS = PHI_ETRS_TMP;
RN_ETRS = RN_ETRS_TMP;
}
}
double LAMBDA_ETRS = Math.Atan(Y_CH1930P / X_CH1930P);
///// ellipsoid coordinates (ETRS89/CHTRS95) -> decimal coordinates (WGS84) /////
double lat = PHI_ETRS * 180 / Math.PI;
double lon = LAMBDA_ETRS * 180 / Math.PI;

Is it true equation written on C #?

The equation is:
Is it true I have written?
Double x = 14.26
Double y = -1.22
Double z = 3.5 * Math.Pow(10.0, -2)
Double t;
t = ( 2 * Math.Cos( x - Math.PI / 6 ) ) / ( 0.5 + Math.Pow( Math.Sin( y ), 2 ) ) * ( (1 + Math.Pow( z, 2 ) ) / ( 3 - Math.Pow( z, 2 ) / 5 ) );
Because the result does not coincide with the above desired result -- t = 0,188451240697501, and I need to t deduced 0.564849.
Double x = 14.26, y = -1.22, z = 3.5 * Math.Pow(10.0, -2), t;
t = (2*Math.Cos(x-Math.PI/6))/(0.5+Math.Pow(Math.Sin(y), 2)) * ( 1 + (Math.Pow(z, 2)) / (3-Math.Pow(z, 2)/5));
Console.WriteLine(t); // 0.5648...
You have parenthesis issue, change ((1 + Math.Pow(z, 2)) to (1 + (Math.Pow(z, 2)), the addition of 1 should be calculated after the division.
Also, consider computing the numerators and denominators separately.
I think that one error it is here:
((1 + Math.Pow(z, 2)) / (3-Math.Pow(z, 2)/5))
you should do:
(1 + (Math.Pow(z, 2) / (3-Math.Pow(z, 2)/5)))

Running (one pass) calculation of covariance

I got a set of 3d vectors (x,y,z), and I want to calculate the covariance matrix without storing the vectors.
I will do it in C#, but eventually I will implement it in C on a microcontroller, so I need the algorithm in itself, and not a library.
Pseudocode would be great also.
The formula is simple if you have Matrix and Vector classes at hand:
Vector mean;
Matrix covariance;
for (int i = 0; i < points.size(); ++i) {
Vector diff = points[i] - mean;
mean += diff / (i + 1);
covariance += diff * diff.transpose() * i / (i + 1);
}
covariance *= 1 / points.size()
I personally always prefer this style rather than the two-pass calculation. The code is short and the results are flawless.
Matrix and Vector can have fixed dimension and can be easily coded for this purpose. You can even rewrite the code into discrete floating-point calculations and avoid computing the symmetric part of the covariance matrix.
Note that there is a vector outer product on the second last row of code. Not all vector libraries interpret it correctly.
I think I have found the solution. It is based on this article about how to calculate covariance manually and this one about calculating running variance. And then I adapted the algorithm in the latter to calculate covariance instead of variance, given my understanding of it from the first article.
public class CovarianceMatrix
{
private int _n;
private Vector _oldMean, _newMean,
_oldVarianceSum, _newVarianceSum,
_oldCovarianceSum, _newCovarianceSum;
public void Push(Vector x)
{
_n++;
if (_n == 1)
{
_oldMean = _newMean = x;
_oldVarianceSum = new Vector(0, 0, 0);
_oldCovarianceSum = new Vector(0, 0, 0);
}
else
{
//_newM = _oldM + (x - _oldM) / _n;
_newMean = new Vector(
_oldMean.X + (x.X - _oldMean.X) / _n,
_oldMean.Y + (x.Y - _oldMean.Y) / _n,
_oldMean.Z + (x.Z - _oldMean.Z) / _n);
//_newS = _oldS + (x - _oldM) * (x - _newM);
_newVarianceSum = new Vector(
_oldVarianceSum.X + (x.X - _oldMean.X) * (x.X - _newMean.X),
_oldVarianceSum.Y + (x.Y - _oldMean.Y) * (x.Y - _newMean.Y),
_oldVarianceSum.Z + (x.Z - _oldMean.Z) * (x.Z - _newMean.Z));
/* .X is X vs Y
* .Y is Y vs Z
* .Z is Z vs X
*/
_newCovarianceSum = new Vector(
_oldCovarianceSum.X + (x.X - _oldMean.X) * (x.Y - _newMean.Y),
_oldCovarianceSum.Y + (x.Y - _oldMean.Y) * (x.Z - _newMean.Z),
_oldCovarianceSum.Z + (x.Z - _oldMean.Z) * (x.X - _newMean.X));
// set up for next iteration
_oldMean = _newMean;
_oldVarianceSum = _newVarianceSum;
}
}
public int NumDataValues()
{
return _n;
}
public Vector Mean()
{
return (_n > 0) ? _newMean : new Vector(0, 0, 0);
}
public Vector Variance()
{
return _n <= 1 ? new Vector(0, 0, 0) : _newVarianceSum.DivideBy(_n - 1);
}
}
The code from emu is elegant, but requires an additional step to be correct:
Vector mean;
Matrix covariance;
for (int i = 0; i < points.size(); ++i) {
Vector diff = points[i] - mean;
mean += diff / (i + 1);
covariance += diff * diff.transpose() * i / (i + 1);
}
covariance = covariance/(points.size()-1);
Note the final step of normalizing the covariance.
Here is a simple example in R to demonstrate the principle:
a <- matrix(rnorm(22), ncol = 2)
a1 <- a[1:10, ]
a2 <- a[2:11, ]
cov(a1)
cov(a2)
m <- 10
# initial step
m1.1 <- mean(a1[, 1])
m1.2 <- mean(a1[, 2])
c1.11 <- cov(a1)[1, 1]
c1.22 <- cov(a1)[2, 2]
c1.12 <- cov(a1)[1, 2]
#step 1->2
m2.1 <- m1.1 + (a[11, 1] - a[1, 1])/m
m2.2 <- m1.2 + (a[11, 2] - a[1, 2])/m
c2.11 <- c1.11 + (a[11, 1]^2 - a[1, 1]^2)/(m - 1) + (m1.1^2 - m2.1^2) * m/(m - 1)
c2.22 <- c1.22 + (a[11, 2]^2 - a[1, 2]^2)/(m - 1) + (m1.2^2 - m2.2^2) * m/(m - 1)
c2.12 <- c1.12 + (a[11, 1] * a[11, 2] - a[1, 1]*a[1, 2])/(m - 1) +
(m1.1 * m1.2 - m2.1 * m2.2) * m/(m - 1)
cov(a2) - matrix(c(c2.11, c2.12, c2.12, c2.22), ncol=2)

Categories