We Keep Coding

Why the intermediate values in matrix inverse calculations are slightly different between C# and CUDA C++? [closed]

Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I wrote two programs to calculate the inverse of a matrix using Gaussian elimination, the first program was in C# and the second in CUDA C++. The two programs follow exactly the same procedure and gives the same final results. However, when I checked the values within the intermediate steps, I found slightly different values, less than 1e-5 relative error.
Here is a part of each code of both programs.
C#
int i, j, i1, n, y, z;
double[,] M = new double[n, n];
double[,] inv = new double[n, n];
for (i = 0; i < n; i++)
inv[i, i] = 1;
for (i = 0; i < n; i++)
{
for (j = i + 1; j < n; j++)
M[i, j] /= M[i, i];
for (j = 0; j < n; j++)
inv[i, j] /= M[i, i];
if (i != n - 1)
{
for (i1 = i + 1; i1 < n; i1++)
if (Math.Abs(M[i1, i]) >= 1e-9)
{
for (j = i + 1; j < n; j++)
M[i1, j] -= M[i1, i] * M[i, j];
for (j = 0; j < n; j++)
inv[i1, j] -= M[i1, i] * inv[i, j];
}
f = new StreamWriter("CPU.csv");
for (y = 0; y < n; y++)
{
for (z = 0; z < n; z++)
f.Write(M[y, z].ToString() + ",");
for (z = 0; z < n; z++)
f.Write(ans[y, z].ToString() + ",");
f.WriteLine();
}
f.Close();
}
}
for (i = n - 1; i > 0; i--)
{
for (i1 = 0; i1 < i; i1++)
if (Math.Abs(M[i1, i]) >= 1e-9)
for (j = 0; j < n; j++)
inv[i1, j] -= M[i1, i] * inv[i, j];
}
CUDA C++
int i, j;
double v;
double* d_A, * d_B, * d_v, * Z;
size = n * n * sizeof(double);
cudaMalloc(&d_A, size);
cudaMemcpy(d_A, A, size, cudaMemcpyHostToDevice);
cudaMalloc(&d_B, size);
cudaMalloc(&d_v, sizeof(double));
Z = new double[n * n];
Unity <<<1, n>>> (d_B, n);
cudaDeviceSynchronize();
for (i = 0; i < n; i++)
{
GetVal <<<1, 1>>> (d_A, i * (n + 1), d_v);
cudaMemcpy(&v, d_v, sizeof(double), cudaMemcpyDeviceToHost);
if (i != n - 1)
DivideRow <<<1, n - i - 1>>> (d_A, i * (n + 1) + 1, n - i - 1, v);
DivideRow <<<1, n>>> (d_B, i * n, n, v);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
if (i != n - 1)
{
dim3 GridA(1, 1);
dim3 BlockA(n - i - 1, n - i - 1);
dim3 GridB(1, 1);
dim3 BlockB(n - i - 1, n);
ModifyRow <<<GridA, BlockA>>> (d_A, i, i, i + 1, n - i - 1, n - i - 1);
ModifyRow <<<GridB, BlockB>>> (d_A, n, i, i, d_B, i + 1, 0, n - i - 1, n);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
myfile.open("GPU.csv");
for (x = 0; x < n; x++)
{
for (y = 0; y < n; y++)
myfile << Z[x * n + y] << ",";
for (y = 0; y < n; y++)
myfile << B[x * n + y] << ",";
myfile << "\n";
}
myfile.close();
}
}
cudaFree(d_v);
for (i = n - 1; i > 0; i--)
{
dim3 GridB(1, 1);
dim3 BlockB(i, n);
ModifyRow <<<GridB, BlockB>>> (d_A, n, i, i, d_B, 0, 0, i, n);
cudaDeviceSynchronize();
cudaMemcpy(Z, d_A, size, cudaMemcpyDeviceToHost);
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
}
cudaMemcpy(B, d_B, size, cudaMemcpyDeviceToHost);
cudaFree(d_A);
cudaFree(d_B);
I compared the values in CPU.csv and GPU.csv files, and found these differences.
What could be the reason for this? Do the calculations in CUDA C++ have less precision than C#?

From the NVIDIA documentation (about 2/3 of the way down):
The consequence [of rounding] is that different math libraries cannot be expected to compute exactly the same result for a given input. This applies to GPU programming as well. Functions compiled for the GPU will use the NVIDIA CUDA math library implementation while functions compiled for the CPU will use the host compiler math library implementation (e.g., glibc on Linux). Because these implementations are independent and neither is guaranteed to be correctly rounded, the results will often differ slightly.
Tells you all you need to know, really.

How would I display a For and Reverse For Loop on the same Line C#

I have been trying to get both a For and Reverse For loop to display on a line in this formation Number x Number
Due to the number being an input (people % i==0) is there to find factors of the number that has been given.
for (int i = 2; i <= people - 1; i++)
{
if (people % i==0)
{
Console.Write($"{i}m x ");
//Console.WriteLine($"{i} is factors of {input}");
}
}
for (int j = people - 1; j >= 2; j-- )
{
if (people % j == 0)
{
Console.WriteLine($"{j}m");
}
}

C# for can use compound assignments/expressions (zero or more statements separated by commas)
int people = 10;
for (int i = 2, j = people - 1; i <= people - 1; i++, j--)
Console.WriteLine($"i = {i}, j = {j}");
Output
i = 2, j = 9
i = 3, j = 8
i = 4, j = 7
i = 5, j = 6
i = 6, j = 5
i = 7, j = 4
i = 8, j = 3
i = 9, j = 2
Note : this is very common in C/++ however it's less common in C#; we tend to like things declarative, neat and readable.
Or you can calculate it on the fly
int people = 10;
for (int i = 2; i <= people - 1; i++)
Console.WriteLine($"i = {i}, j = {people-i+1}");
Full Demo Here

If you're interested in a way to achieve this using Linq, this is achievable using a Zip operation:
int start = 2;
int people = 9;
Enumerable.Range(start, people - start + 1)
.Zip(Enumerable.Range(start, people - start + 1).Reverse(), (x, y) => $"{x}m x {y}m")
.ToList()
.ForEach(Console.WriteLine);
Output
2m x 9m
3m x 8m
4m x 7m
5m x 6m
6m x 5m
7m x 4m
8m x 3m
9m x 2m

Create another index j in the first for loop
for (int i = 2; i <= people - 1; i++)
{
if (people % i==0)
{
Console.Write($"{i}m x ");
//Console.WriteLine($"{i} is factors of {input}");
}
int j = people - i + 1;
if (people % j == 0)
{
Console.WriteLine($"{j}m");
}
}

recursive stackoverflow minesweeper c#

I am writing a game of minesweeper. Below is code for 3 methods in minesweeper. The first method is to check all the spaces around the button pressed and to count how many bombs are around it. The next method is to be called recursively, in order that if the user pressed a button with 0 buttons around it, it will open all of the squares that also indicate 0 squares around it. The third method is to check that it will be in bound the check. The empty space recursive call is getting me a stackoverflow error, what am I doing wrong?
Thanks!
private int GameLogicChecker(int x, int y)
{
int count = 0;
if (_grid[x, y] != -1)
{
if (x + 1 < SizeX)
{ //Right
if (_grid[x + 1, y] == -1)
count++;
}
if (x - 1 > 0)
{ //Left
if (_grid[x - 1, y] == -1)
count++;
}
if (y + 1 < SizeY)
{ //Upper
if (_grid[x, y + 1] == -1)
count++;
}
if (y - 1 > 0)
{ //Lower
if (_grid[x, y - 1] == -1)
count++;
}
if (x + 1 < SizeX && y + 1 < SizeY)
{ //Right-Upper
if (_grid[x + 1, y + 1] == -1)
count++;
}
if (x + 1 < SizeX && y - 1 > 0)
{ //Right-Lower
if (_grid[x + 1, y - 1] == -1)
count++;
}
if (x - 1 > 0 && y + 1 < SizeY)
{ //Left-Upper
if (_grid[x - 1, y + 1] == -1)
count++;
}
if (x - 1 > 0 && y - 1 > 0)
{ //Left-Lower
if (_grid[x - 1, y - 1] == -1)
count++;
}
}
return count;
}
void OpenEmptySpace(int x, int y)
{
for (var k = -1; k <= 1; k++)
{
for (var l = -1; l <= 1; l++)
{
if (CheckBounds(x + k, y + l) && GameLogicChecker(x + k, y + l) == 0)
{
_buttons[x + k, y + l].Text = "0";
OpenEmptySpace(x + k, y + l);
}
}
}
}
private bool CheckBounds(int x, int y)
{
return x >= 0 && x < SizeX && y >= 0 && y < SizeY;
}

For k = 0 and l = 0, you are calling yourself again and again and again...
Thanks to #BenVoigt for pointing out that two zeroes adjacent to each other will also lead to infinite recursion. So, in order to solve that one method is to create a boolean grid too and set a particular cell's value to true if it has been run through once. Assuming the grid is called Explored, I've added the condition for it in the code below.
If you insist on your current code, try changing the condition to:
if (CheckBounds(x + k, y + l)
&& GameLogicChecker(x + k, y + l) == 0
&& !(k == 0 && l == 0)
&& !Explored[x + k, y + l])
{
Explored[x + k, y + l] = true;
_buttons[x + k, y + l].Text = "0";
OpenEmptySpace(x + k, y + l);
}

Here is another answer for you, rewriting your methods one-by-one following better coding practices. Like in the other answer, a boolean grid called Explored[SizeX, SizeY] has been assumed.
1. GameLogicChecker()
private int GameLogicChecker(int x, int y)
{
if (_grid[x, y] == -1) return 0;
int count = 0;
if (x + 1 < SizeX && _grid[x + 1, y] == -1) //Right
{
count++;
}
if (x - 1 > 0 && _grid[x - 1, y] == -1) //Left
{
count++;
}
if (y + 1 < SizeY && _grid[x, y + 1] == -1) //Upper
{
count++;
}
if (y - 1 > 0 && _grid[x, y - 1] == -1) //Lower
{
count++;
}
if (x + 1 < SizeX && y + 1 < SizeY && _grid[x + 1, y + 1] == -1) //Right-Upper
{
count++;
}
if (x + 1 < SizeX && y - 1 > 0 && _grid[x + 1, y - 1] == -1) //Right-Lower
{
count++;
}
if (x - 1 > 0 && y + 1 < SizeY && _grid[x - 1, y + 1] == -1) //Left-Upper
{
count++;
}
if (x - 1 > 0 && y - 1 > 0 && _grid[x - 1, y - 1] == -1) //Left-Lower
{
count++;
}
return count;
}
What's better? Quicker returning from the method for special case. Reduced nesting in If(...) blocks.
2. OpenEmptySpace()
public/private void OpenEmptySpace(int x, int y)
{
for (var deltaX = -1; deltaX <= 1; deltaX += 2)
{
for (var deltaY = -1; deltaY <= 1; deltaY += 2)
{
var thisX = x + deltaX;
var thisY = y + deltaY;
if (OpeningNotNeeded(thisX, thisY))
{
continue;
}
Explored[thisX, thisY] = true;
_buttons[thisX, thisY].Text = "0";
OpenEmptySpace(thisX, thisY);
}
}
}
private bool OpeningNotNeeded(int x, int y)
{
return !CheckBounds(x, y)
|| GameLogicChecker(x, y) != 0
|| Explored[x, y];
}
What's better? Properly named indexing variables in both loops. Properly written condition (+= 2 instead of ++). Reduced nesting in If(...). Easier to read method call in the If(...) instead of three predicates. Useful temporary variables added which make it clear what x + k and y + l were in the code written earlier.
3. CheckBounds() is written fine.

Sobel image processing

My algorithm output is wrong. I tried many solutions, but nothing comes out.
View my result.
SourceImage
I'm sorry Lena
for (int x = 1; x < fimage.Bitmap.Width - 1; x++)
{
for (int y = 1; y < fimage.Bitmap.Height - 1; y++)
{
double sumX = 0, sumY = 0, sum = 0;
for ( int i = -1; i <= 1; i++ )
{
for ( int j = -1; j <= 1; j++ )
{
sumX += fimage[y + i, x + j].R * kernel1[i + 1, j + 1];
sumY += fimage[y + i, x + j].R * kernel2[i + 1, j + 1];
}
}
sum = Math.Sqrt(sumX * sumX + sumY * sumY);
sum = sum > 255 ? 255 : sum < 0 ? 0 : sum;
fimage[x, y] = Color.FromArgb((byte)sum, (byte)sum, (byte)sum);
}
}

Two things that are fishy:
for ( int j = -1; j <= 1; j++ )
{
sumX += fimage[y + i, x + j].R * kernel1[i + 1, j + 1];
sumY += fimage[y + i, x + j].R * kernel2[i + 1, j + 1];
}
You only respect the red component of the image here, why is that?
The other major thing is changing the input picture while iterating through it:
sum = Math.Sqrt(sumX * sumX + sumY * sumY);
sum = sum > 255 ? 255 : sum < 0 ? 0 : sum;
fimage[x, y] = Color.FromArgb((byte)sum, (byte)sum, (byte)sum);
You should save the value to a different output image (create a new Bitmap(fimage.Width, fimage.Height) with the dimensions from the source image). That could explain the weird diagonal symmetry in your picture, where there's basically a symteric copy of the other side.

Game of Life - Dead

I was bored and had only 30 minutes of free time so I decided to have a crack at making the game of life. I followed that rules on wikipedia and it doesn't seem to be working correctly. Could someone please tell me what I would be doing wrong?
Here are the rules:
Any live cell with fewer than two live neighbours dies, as if caused by under-population.
Any live cell with two or three live neighbours lives on to the next generation.
Any live cell with more than three live neighbours dies, as if by overcrowding.
Any dead cell with exactly three live neighbours becomes a live cell, as if by reproduction.
public void PerformLogic()
{
if (in_game)
{
time_elapsed += rate;
if (time_elapsed > frame_rate)
{
time_elapsed = 0;
for (int x = 0; x < board_width; x++)
{
for (int y = 0; y < board_height; y++)
{
if (board[x, y] == alive)
{
int surrounding_cells = 0;
for (int x2 = -1; x2 <= 1; x2++)
{
for (int y2 = -1; y2 <= 1; y2++)
{
if (!(x2 + x <= -1 || y2 + y <= -1 || x + x2 >= board_width || y + y2 >= board_height))
{
if (board[x + x2, y + y2] == alive)
{
surrounding_cells++;
}
}
}
}
if (surrounding_cells < 2)
{
board[x, y] = dead;
}
if (surrounding_cells == 2 ||
surrounding_cells == 3)
{
board[x, y] = alive;
}
if (surrounding_cells > 3)
{
board[x, y] = dead;
}
}
else if (board[x, y] == dead)
{
int surrounding_cells = 0;
for (int x2 = -1; x2 <= 1; x2++)
{
for (int y2 = -1; y2 <= ; y2++)
{
if (!(x2 + x <= -1 || y2 + y <= -1 || x + x2 >= board_width || y + y2 >= board_height))
{
if (board[x + x2, y + y2] == alive)
{
surrounding_cells++;
}
}
}
}
if (surrounding_cells == 3)
{
board[x, y] = alive;
}
}
}
}
}
}
}
Any ideas?

I believe you are updating the board too early. The game of life should update the board after it finishes scanning the whole board, rather than while scanning.
E.g.:
if (surrounding_cells > 3)
{
board[x, y] = dead;
}
After this, for the cell next to it, this cell would be treated as dead.
Marc is right, too.

for (int x2 = -1; x2 <= 1; x2++)
{
for (int y2 = -1; y2 <= 1; y2++)
{
looks to me like you're including the central cell in this loop, so 9 instead of 8.
I'm not sure the nested for is the best option, but if you are using that, add:
if(x2 == 0 && y2 == 0) continue;
at the start of the inner loop (i.e. after the last line that I've posted above)