Coding a matrix multiplication in my program, I get precision errors (inaccurate results for large matrices).
Here's my code. The current object has data stored in a flattened array, row after row. Other matrix B has data stored in a flattened array, column after column (so I can use pointer arithmetic).
protected double[,] multiply (IMatrix B)
{
int columns = B.columns;
int rows = Rows;
int size = Columns;
double[,] result = new double[rows,columns];
for (int row = 0; row < rows; row++)
{
for (int col = 0; col < columns; col++)
{
unsafe
{
fixed (float* ptrThis = data)
fixed (float* ptrB = B.Data)
{
float* mePtr = ptrThis + row*rows;
float* bPtr = ptrB + col*columns;
double value = 0.0;
for (int i = 0; i < size; i++)
{
value += *(mePtr++) * *(bPtr++);
}
result[row, col] = value;
}
}
}
}
}
Actually, the code is a bit more complicated : I do the multiply thing for several chunks (so instead of having i from 0 to size, I go from localStart to localStop), then sum up the resulting matrices.
My problem : for a big matrix I get precision error :
NUnit.Framework.AssertionException: Error at (0,1)
expected: <6.4209571409444209E+18>
but was: <6.4207619776304906E+18>
Any idea ?
Perhaps all you have to do is use Kahan summation. But you can never expect to get exactly a specific result with floating-point math.
Turns out it was just ... a bug. Ended up that instead of having :
float* mePtr = ptrThis + row*rows;
float* bPtr = ptrB + col*columns;
The correct indexers for my rows were :
float* mePtr = ptrThis + row * size;
float* bPtr = ptrB + col * size;
Sorry for that, not really fancy answer here. But thanks for the help !
I originally stated that you should convert the floats to doubles. However, as you point out that will break your algorithm.
You could try:
value += (double)*(mePtr++) * (double)*(bPtr++);
A problem with your code as it now stands is that the multiplication is being done in float precision then added to a double. Casting to double first will help to some extent.
It might be clearer to use intermediate double variables - but that's up to you.
If this doesn't give you the desire accuracy then you'll need to consider using decimal instead of double. However, this may result in a performance hit so do some benchmarks first.
Hem, it doesn't really solve your problem but in NUnit, you can allow to have a precision error and choose the value of this epsilon
As a starting point, use double everywhere instead of float.
At the very least, you should be using doubles throughout. Floats are very imprecise.
This is a phenomenon called "Matrix Creep" which happens gradually during matrix manipulations if you don't consistently normalize your matrices.
Related
I am doing project euler and i am at problem 15 now, here is a link:
https://projecteuler.net/problem=15 . I am trying to solve this with binomial coefficient. Here is a site that explains it: http://www.mathblog.dk/project-euler-15/ . You can find it at the bottom.
My question is, why is the following code wrong? Since this follows the mathematical algorithm I think: n-k+i/i
int grid = 20;
long paths = 1;
for (int i = 0; i < grid; i++)
{
paths *= (grid * 2) - (grid + i)
paths /= (i + 1);
}
Console.WriteLine(paths);
Console.ReadKey();
And why is this code wrong? This is exactly as the mathblog site but in 1 line.
int grid = 20;
long paths = 1;
for (int i = 0; i < grid; i++)
{
paths *= ((grid * 2) - i) / (i + 1);
}
Console.WriteLine(paths);
Console.ReadKey();
But why is this code right then? Isnt it the same as the previous code? And it doesn't exactly follow the mathematical algorithm does it? Because it's n-k+i/i, and this code does n-i/i
int grid = 20;
long paths = 1;
for (int i = 0; i < grid; i++)
{
paths *= ((grid * 2) - i);
paths /= (i + 1);
}
Console.WriteLine(paths);
Console.ReadKey();
Thnx guys!
If you want to combain the calculation it should be like this
paths = (path *((grid * 2) - i))/(i + 1);
By convention, in many programming languages, int/int gives an int,* not a floating point number. Your method implies that 'paths' should take values that are not int. In fact none of the three methods should work but by a happy coincidence, the last one worked: basically because all intermediate values of 'paths' happen to be binomial coefficients too.
Advice for debugging: ask your program to output the intermediate values. This helps a lot.
*: As a mathematician, I almost never need that feature. In fact the other convention (int/int -> double) would have made my life as a programmer easier on average.
I had a look at the blog you mention. This makes your message much more understandable.
The blog mentions a formula : the product for i from 1 to k of (n-k+1)/i.
So to mimick it you would need to write
for (int i = 1; i <= grid; i++) // bounds!
{
paths *= (grid * 2) - (grid - i) // minus sign!
paths /= (i + 1);
}
About the fact that this works with ints: this is an accident due to the fact that in the intermediate values of the product (at the end of each loop) are binomial coefficients all along the computation. If you would compute the products and divisions in another order, you may very well get non-integers so the computation would fail with an integer variable type for path because of the convention int/int -> int. The blog is not very helpful in not mentionning that.
I am working on program that requires me to iterate through all single precision floating point (23 fractions bits) numbers in the range of [1,2). I am not quite sure how to go about this. I am writing this program in C#.
If someone could give me some help with this, that would be awesome. Thank you!
You could use the BitConverter static class to convert float value to int and back. Thus you can access its bits.
int one = BitConverter.ToInt32(BitConverter.GetBytes(1f), 0);
int two = BitConverter.ToInt32(BitConverter.GetBytes(2f), 0);
for (int i = one; i < two; i++)
{
float f = BitConverter.ToSingle(BitConverter.GetBytes(i), 0);
// Your stuff
}
I am working on a program and it has a pretty long execution time. I'am trying to improve performance where I can, however my knowledge is limited in this area. Can anyone recommend a way to speed up the method below?
public static double DistanceBetween2Points(double[,] p1, double[,] p2, int patchSize)
{
double sum = 0;
for (int i = 0; i < patchSize; i++)
{
for (int j = 0; j < patchSize; j++)
{
sum += Math.Sqrt(Math.Pow(p1[i, j] - p2[i, j], 2));
}
}
return sum;
}
The method calculates the distance between two images by calculating the sum of all the distances between two points on the two images.
Think about your algorithm. Probably a pixel-distance isn't the best thing to get an acurate image-distance.
replace sqrt(x^2) by abs(x) or even faster:
if(x < 0) x = -x;
Rename your routine to OverallImageDistance or similar(will not improve performance) ;)
Use unsafe pointers, and calculate your distance in a single loop using these pointers:
unsafe
{
sum = 0.0;
int numPixels = patchsize*patchsize;
fixed(int *pointer1 = &p1[0])
{
fixed(int* pointer2 = &p2[0])
{
while(numPixels-- > 0)
{
double dist = *pointer1++ - *pointer2++;
if(dist < 0) dist = -dist;
sum += dist;
}
...
This should be several times faster than your original.
Well, this method is really weird and does not look like distance between pixels at all. But certainly you would want to use linear algebra instead of straightforward array calculations.
Image recognition, natural language processing and machine learning algorithms all use matrices, because matrix libraries are highly optimized for these kind of situations, when you need batch processing.
There is a plethora of matrix libraries in the wild, look here Recommendation for C# Matrix Library
EDIT: Ok, thanks for feedback, trying to improve the answer...
You can use Math.Net Numerics open source library (install MathNet.Numerics nuget package) and rewrite your method like this:
using MathNet.Numerics.LinearAlgebra;
public static double DistanceBetween2Points(double[,] p1, double[,] p2, int patchSize)
{
var A = Matrix<double>.Build.DenseOfArray(p1).SubMatrix(0, patchSize, 0, patchSize);
var B = Matrix<double>.Build.DenseOfArray(p2).SubMatrix(0, patchSize, 0, patchSize);
return (A - B).RowAbsoluteSums().Sum();
}
Essentially, loops slow down your code. When doing batch processing ideally you should avoid loops at all.
I want to create an array containing values from 0 to 1 with interval of 0.1. I can use:
float[] myArray = new float[10];
float increment = 0.1;
for(i = 0; i < 10; i++)
{
myArray[i] = increment;
increment += 0.1;
}
I was wondering whether there is a function like Enumerable.Range that permits to specify also the increment interval.
An interesting fact is that every answer posted so far has fixed the bug in your proposed code, but only one has called out that they've done so.
Binary floating point numbers have representation error when dealing with any quantity that is not a fraction of an exact power of two. ("3.0/4.0" is a representable fraction because the bottom is a power of two; "1.0/10.0" is not.)
Therefore, when you say:
for(i = 0; i < 10; i++)
{
myArray[i] = increment;
increment += 0.1;
}
You are not actually incrementing "increment" by 1.0/10.0. You are incrementing it by the closest representable fraction that has an exact power of two on the bottom. So in fact this is equivalent to:
for(i = 0; i < 10; i++)
{
myArray[i] = increment;
increment += (exactly_one_tenth + small_representation_error);
}
So, what is the value of the tenth increment? Clearly it is 10 * (exactly_one_tenth + small_representation_error) which is obviously equal to exactly_one + 10 * small_representation_error. You have multiplied the size of the representation error by ten.
Any time you repeatedly add together two floating point numbers, each subsequent addition increases the total representation error of the sum slightly and that adds up, literally, to a potentially large error. In some cases where you are summing thousands or millions of small numbers the error can become far larger than the actual total.
The far better solution is to do what everyone else has done. Recompute the fraction from integers every time. That way each result gets its own small representation error; it does not accumulate the representation errors of previously computed results.
Ugly, but...
Enumerable.Range(0,10).Select(i => i/10.0).ToArray();
No, there's no enumerable range that allows you to do that, but you could always divide by 10:
foreach (int i in Enumerable.Range(0, 10))
array[i] = (i + 1) / 10.0f
Note that this avoids the error that will accumulate if you repeatedly sum 0.1f. For example, if you sum the 10 elements in the myArray in your sample code, you get a value that's closer to 5.50000048 than 5.5.
Here is one way:
Enumerable.Range(1,10).Select(i => i /10.0)
Well you could use this:
Enumerable.Range(1,10).Select(x => x / 10.0).ToArray()
Not sure if that's better though.
I have problem in dynamic memory allocation getting data from matrix
Image image_gray = new Image("im1.jpg");
Matrix circles = new Matrix(100, 1, 3);
Question 1: How can I locate dynamic memory because I don't know the number of circles?
Emgu.CV.CvInvoke.cvHoughCircles(image_gray, circles, HOUGH_TYPE.CV_HOUGH_GRADIENT,
2, 100, 200, 100, 10, 500);
Question 2: Now circle is matrix with [100 ,3], How can I get
point center= Round (circle[i][1], circle[i][1])
How can I get
int radius= circle[i][2];
what should my for loop look like to get data from matrix and casting should be point and int.
I tried already (NOT WORKING / ERROR)
for (int i=0; i < circles.Rows; i++)
{ Matrix entry = circles.GetRow(i);
float x = entry[0];
float y = entry[1];
float r = entry[2];} // NOT WORKING
because instead of copying (required) only (i) row it copies whole matrix( circles) and float x=......gives errors
No overload for method 'this' takes '1' arguments
please help me in this regard
regards
sorry to answer my question. please give some hint to dynamic memory allocation and matrix data
The documentation suggests that it will grow the matrix to fit, but it also contradicts that, so honestly, I would try a 1 row 1 col matrix on an image with more circles and see what it gives you. I see that you're using Matrix type - I don't know how you convert to the IntPtr needed by cvHoughCircles, but it should be straight forward to iterate over each row and pull out the results, which are 3 floats: (x, y, r) -
for (int i=0; i < circles.Cols; i++) {
Matrix<float> entry = circles.GetRow(i);
float x = entry[0]; // guessing here - the doc'n is truly awful
float y = entry[1];
float r = entry[2];
}