We Keep Coding

Why the intermediate values in matrix inverse calculations are slightly different between C# and CUDA C++? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I wrote two programs to calculate the inverse of a matrix using Gaussian elimination, the first program was in C# and the second in CUDA C++. The two programs follow exactly the same procedure and gives the same final results. However, when I checked the values within the intermediate steps, I found slightly different values, less than 1e-5 relative error.
Here is a part of each code of both programs.
C#
int i, j, i1, n, y, z;
double[,] M = new double[n, n];
double[,] inv = new double[n, n];
for (i = 0; i < n; i++)
inv[i, i] = 1;
for (i = 0; i < n; i++)
{
for (j = i + 1; j < n; j++)
M[i, j] /= M[i, i];
for (j = 0; j < n; j++)
inv[i, j] /= M[i, i];
if (i != n - 1)
{
for (i1 = i + 1; i1 < n; i1++)
if (Math.Abs(M[i1, i]) >= 1e-9)
{
for (j = i + 1; j < n; j++)
M[i1, j] -= M[i1, i] * M[i, j];
for (j = 0; j < n; j++)
inv[i1, j] -= M[i1, i] * inv[i, j];
}
f = new StreamWriter("CPU.csv");
for (y = 0; y < n; y++)
{
for (z = 0; z < n; z++)
f.Write(M[y, z].ToString() + ",");
for (z = 0; z < n; z++)
f.Write(ans[y, z].ToString() + ",");
f.WriteLine();
}
f.Close();
}
}
for (i = n - 1; i > 0; i--)
{
for (i1 = 0; i1 < i; i1++)
if (Math.Abs(M[i1, i]) >= 1e-9)
for (j = 0; j < n; j++)
inv[i1, j] -= M[i1, i] * inv[i, j];
}
CUDA C++
int i, j;
double v;
double* d_A, * d_B, * d_v, * Z;
size = n * n * sizeof(double);
cudaMalloc(&d_A, size);
cudaMemcpy(d_A, A, size, cudaMemcpyHostToDevice);
cudaMalloc(&d_B, size);
cudaMalloc(&d_v, sizeof(double));
Z = new double[n * n];
Unity <<<1, n>>> (d_B, n);
cudaDeviceSynchronize();
for (i = 0; i < n; i++)
{
GetVal <<<1, 1>>> (d_A, i * (n + 1), d_v);
cudaMemcpy(&v, d_v, sizeof(double), cudaMemcpyDeviceToHost);
if (i != n - 1)
DivideRow <<<1, n - i - 1>>> (d_A, i * (n + 1) + 1, n - i - 1, v);
DivideRow <<<1, n>>> (d_B, i * n, n, v);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
if (i != n - 1)
{
dim3 GridA(1, 1);
dim3 BlockA(n - i - 1, n - i - 1);
dim3 GridB(1, 1);
dim3 BlockB(n - i - 1, n);
ModifyRow <<<GridA, BlockA>>> (d_A, i, i, i + 1, n - i - 1, n - i - 1);
ModifyRow <<<GridB, BlockB>>> (d_A, n, i, i, d_B, i + 1, 0, n - i - 1, n);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
myfile.open("GPU.csv");
for (x = 0; x < n; x++)
{
for (y = 0; y < n; y++)
myfile << Z[x * n + y] << ",";
for (y = 0; y < n; y++)
myfile << B[x * n + y] << ",";
myfile << "\n";
}
myfile.close();
}
}
cudaFree(d_v);
for (i = n - 1; i > 0; i--)
{
dim3 GridB(1, 1);
dim3 BlockB(i, n);
ModifyRow <<<GridB, BlockB>>> (d_A, n, i, i, d_B, 0, 0, i, n);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
}
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
cudaFree(d_A);
cudaFree(d_B);
I compared the values in CPU.csv and GPU.csv files, and found these differences.
What could be the reason for this? Do the calculations in CUDA C++ have less precision than C#?

From the NVIDIA documentation (about 2/3 of the way down):
The consequence [of rounding] is that different math libraries cannot be expected to compute exactly the same result for a given input. This applies to GPU programming as well. Functions compiled for the GPU will use the NVIDIA CUDA math library implementation while functions compiled for the CPU will use the host compiler math library implementation (e.g., glibc on Linux). Because these implementations are independent and neither is guaranteed to be correctly rounded, the results will often differ slightly.
Tells you all you need to know, really.

diagonals in a rectangle of asterisks

I'm trying to print a rectangle of asterisks with its diagonals.
I have the code for it, but I'm wondering if there's any way to make it more symmetrical?
int height = int.Parse(Console.ReadLine());
int width = int.Parse(Console.ReadLine());
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
if (i == 0 || j == 0 || i == height - 1 || j == width - 1 || i == j || i + j == width- 1) {
Console.Write("*");
}
else {
Console.Write(" ");
}
}
Console.WriteLine();
}
With an imput of (12, 16) it comes out:
****************
** **
* * * *
* * * *
* * * *
* * * *
* * * *
* ** *
* ** *
* * * *
* * * *
****************

To draw a diagonal in a height * width rectangle:
int height = int.Parse(Console.ReadLine());
int width = int.Parse(Console.ReadLine());
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
// Calculate the slope
int x = (int) (0.5 + (1.0 * (i) * width) / (height-0.5));
if (i == 0 || j == 0 || i == height - 1 || j == width - 1 || x == j || j == width - x - 1) {
Console.Write("*");
}
else {
Console.Write(" ");
}
}
Console.WriteLine();
}

Sobel image processing

My algorithm output is wrong. I tried many solutions, but nothing comes out.
View my result.
SourceImage
I'm sorry Lena
for (int x = 1; x < fimage.Bitmap.Width - 1; x++)
{
for (int y = 1; y < fimage.Bitmap.Height - 1; y++)
{
double sumX = 0, sumY = 0, sum = 0;
for ( int i = -1; i <= 1; i++ )
{
for ( int j = -1; j <= 1; j++ )
{
sumX += fimage[y + i, x + j].R * kernel1[i + 1, j + 1];
sumY += fimage[y + i, x + j].R * kernel2[i + 1, j + 1];
}
}
sum = Math.Sqrt(sumX * sumX + sumY * sumY);
sum = sum > 255 ? 255 : sum < 0 ? 0 : sum;
fimage[x, y] = Color.FromArgb((byte)sum, (byte)sum, (byte)sum);
}
}

Two things that are fishy:
for ( int j = -1; j <= 1; j++ )
{
sumX += fimage[y + i, x + j].R * kernel1[i + 1, j + 1];
sumY += fimage[y + i, x + j].R * kernel2[i + 1, j + 1];
}
You only respect the red component of the image here, why is that?
The other major thing is changing the input picture while iterating through it:
sum = Math.Sqrt(sumX * sumX + sumY * sumY);
sum = sum > 255 ? 255 : sum < 0 ? 0 : sum;
fimage[x, y] = Color.FromArgb((byte)sum, (byte)sum, (byte)sum);
You should save the value to a different output image (create a new Bitmap(fimage.Width, fimage.Height) with the dimensions from the source image). That could explain the weird diagonal symmetry in your picture, where there's basically a symteric copy of the other side.

Output custom shape in ConsoleApplication - C#

I have this shape and I want to output it to ConoleApplication Windows.
I have this code but it doesn't work as I need :
int i,j;
for(i=0;i<5;i++)
{
for(j=5-i;j>0;j--)
Console.WriteLine(" ");
for(j=0;j<=2*i;j++)
Console.WriteLine("*");
Console.WriteLine("\n");
}
Thanks in advance.
EDIT : I am very sorry

As everyone knows, everything Just Works™ when using LINQ... so, have you tried doing it with LINQ?
int n = 6;
var result = string.Join("\r\n", from i in Enumerable.Range(1, n)
where i != 2
let stars = Enumerable.Repeat('*', i)
let indent = new string(' ', n - i)
select indent + string.Join(" ", stars));
Console.WriteLine(result);
*
* * *
* * * *
* * * * *
* * * * * *

Something like:
for (j = 0; j <= 2 * i; j += 2)
{
printf("*");
printf(" ");
// or Console.Write("* ") if we are talking C#
}
which writes the spaces between asterisks (plus a spare one; you could remove that if it is important).

You just used WriteLine instead of Write
Here is the correct code:
int i, j;
for (i = 0; i < 5; i++)
{
for (j = 5 - i; j > 0; j--)
Console.Write(" ");
for (j = 0; j <= 2 * i; j++)
Console.Write("*");
Console.WriteLine();
}

Calculating an NxN matrix determinant in C#

How do you calculate the determinant of an NxN matrix C# ?

The OP posted another question asking specifically about 4x4 matrices, which has been closed as an exact duplicate of this question. Well, if you're not looking for a general solution but instead are constrained to 4x4 matrices alone, then you can use this ugly looking but tried-and-true code:
public double GetDeterminant() {
var m = _values;
return
m[12] * m[9] * m[6] * m[3] - m[8] * m[13] * m[6] * m[3] -
m[12] * m[5] * m[10] * m[3] + m[4] * m[13] * m[10] * m[3] +
m[8] * m[5] * m[14] * m[3] - m[4] * m[9] * m[14] * m[3] -
m[12] * m[9] * m[2] * m[7] + m[8] * m[13] * m[2] * m[7] +
m[12] * m[1] * m[10] * m[7] - m[0] * m[13] * m[10] * m[7] -
m[8] * m[1] * m[14] * m[7] + m[0] * m[9] * m[14] * m[7] +
m[12] * m[5] * m[2] * m[11] - m[4] * m[13] * m[2] * m[11] -
m[12] * m[1] * m[6] * m[11] + m[0] * m[13] * m[6] * m[11] +
m[4] * m[1] * m[14] * m[11] - m[0] * m[5] * m[14] * m[11] -
m[8] * m[5] * m[2] * m[15] + m[4] * m[9] * m[2] * m[15] +
m[8] * m[1] * m[6] * m[15] - m[0] * m[9] * m[6] * m[15] -
m[4] * m[1] * m[10] * m[15] + m[0] * m[5] * m[10] * m[15];
}
It assumes you store your vector data in a 16-element array called _values (of double in this case, but float would work too), in the following order:
0, 1, 2, 3,
4, 5, 6, 7,
8, 9, 10, 11,
12, 13, 14, 15

Reduce to upper triangular form, then make a nested loop where you multiply all the values at position i == j together. There you have it.

The standard method is LU decomposition. You may want to use a library instead of coding it yourself. I don't know about C#, but the 40-year standard is LAPACK.

This solution is achieved using row operations. I took an identity matrix of the same dimensions as of my target matrix and then converted the target matrix to the identity matrix such that, every operation which is performed on target matrix, must also be performed to the identity matrix. In last, the target matrix will become identity matrix and the identity matrix will hold the inverse of the target matrix.
private static double determinant(double[,] matrix, int size)
{
double[] diviser = new double[size];// this will be used to make 0 all the elements of a row except (i,i)th value.
double[] temp = new double[size]; // this will hold the modified ith row after divided by (i,i)th value.
Boolean flag = false; // this will limit the operation to be performed only when loop n != loop i
double determinant = 1;
if (varifyRowsAndColumns(matrix, size)) // verifies that no rows or columns are similar or multiple of another row or column
for (int i = 0; i < size; i++)
{
int count = 0;
//this will hold the values to be multiplied by temp matrix
double[] multiplier = new double[size - 1];
diviser[i] = matrix[i, i];
//if(i,i)th value is 0, determinant shall be 0
if (diviser[i] == 0)
{
determinant = 0;
break;
}
/*
* whole ith row will be divided by (i,i)th value and result will be stored in temp matrix.
* this will generate 1 at (i,i)th position in temp matrix i.e. ith row of matrix
*/
for (int j = 0; j < size; j++)
{
temp[j] = matrix[i, j] / diviser[i];
}
//setting up multiplier to be used for multiplying the ith row of temp matrix
for (int o = 0; o < size; o++)
if (o != i)
multiplier[count++] = matrix[o, i];
count = 0;
//for creating 0s at every other position than (i,i)th
for (int n = 0; n < size; n++)
{
for (int k = 0; k < size; k++)
{
if (n != i)
{
flag = true;
matrix[n, k] -= (temp[k] * multiplier[count]);
}
}
if (flag)
count++;
flag = false;
}
}
else determinant = 0;
//if determinant is not 0, (i,i)th element will be multiplied and the result will be determinant
if (determinant != 0)
for (int i = 0; i < size; i++)
{
determinant *= matrix[i, i];
}
return determinant;
}
private static Boolean varifyRowsAndColumns(double[,] matrix, int size)
{
List<double[]> rows = new List<double[]>();
List<double[]> columns = new List<double[]>();
for (int j = 0; j < size; j++)
{
double[] temp = new double[size];
for (int k = 0; k < size; k++)
{
temp[j] = matrix[j, k];
}
rows.Add(temp);
}
for (int j = 0; j < size; j++)
{
double[] temp = new double[size];
for (int k = 0; k < size; k++)
{
temp[j] = matrix[k, j];
}
columns.Add(temp);
}
if (!RowsAndColumnsComparison(rows, size))
return false;
if (!RowsAndColumnsComparison(columns, size))
return false;
return true;
}
private static Boolean RowsAndColumnsComparison(List<double[]> rows, int size)
{
int countEquals = 0;
int countMod = 0;
int countMod2 = 0;
for (int i = 0; i < rows.Count; i++)
{
for (int j = 0; j < rows.Count; j++)
{
if (i != j)
{
double min = returnMin(rows.ElementAt(i), rows.ElementAt(j));
double max = returnMax(rows.ElementAt(i), rows.ElementAt(j));
for (int l = 0; l < size; l++)
{
if (rows.ElementAt(i)[l] == rows.ElementAt(j)[l])
countEquals++;
for (int m = (int)min; m <= max; m++)
{
if (rows.ElementAt(i)[l] % m == 0 && rows.ElementAt(j)[l] % m == 0)
countMod++;
if (rows.ElementAt(j)[l] % m == 0 && rows.ElementAt(i)[l] % m == 0)
countMod2++;
}
}
if (countEquals == size)
{
return false;
// one row is equal to another row. determinant is zero
}
if (countMod == size)
{
return false;
}
if (countMod2 == size)
{
return false;
}
}
}
}
return true;
}
private static double returnMin(double[] row1, double[] row2)
{
double min1 = row1[0];
double min2 = row2[0];
for (int i = 1; i < row1.Length; i++)
if (min1 > row1[i])
min1 = row1[i];
for (int i = 1; i < row2.Length; i++)
if (min2 > row2[i])
min2 = row2[i];
if (min1 < min2)
return min1;
else return min2;
}
private static double returnMax(double[] col1, double[] col2)
{
double max1 = col1[0];
double max2 = col2[0];
for (int i = 1; i < col1.Length; i++)
if (max1 < col1[i])
max1 = col1[i];
for (int i = 1; i < col2.Length; i++)
if (max2 < col2[i])
max2 = col2[i];
if (max1 > max2)
return max1;
else return max2;
}