We Keep Coding

c# Unity: Trying to make a biome generation algorithm multithreadable

I'm currently learning how to code games and have designed a biom generation algorithm.
As long as I run that algorithm below syncron, it generates the same output every time and works perfectly fine.
Now I tried to speed it up and make it multithreaded, but every time I call the method, it results in a different result.
As far as I know, I used Threadsave Collections, whenever necessary, but it still doesn't work.
Also, I tried to lock the collection, but this didn't work either.
So I'm completely clueless as to why this doesn't work.
If you see anything that I could make better or how I could fix that problem, please let me know.
This code is working:
private Biome[,] Generate(string worldSeed, Vector2Int targetChunk, List<(Biome, float)> multiplier, float centroidsPerChunk)
{
//Calculate the NeighboursToGenerate depeding on the cendroids per Chunk value
int chunkNeighboursToGenerate = (int)Math.Ceiling(Math.Sqrt(1f / centroidsPerChunk * 12.5f));
int chunkSize = 8;
//Create List that contains all centroids of the chunk
List<(Vector2Int, Biome)> centroids = new();
//Create Centdroids for every chunk of the generated region around the targetchunk
for (int chunkX = targetChunk.x - chunkNeighboursToGenerate; chunkX < targetChunk.x + chunkNeighboursToGenerate + 1; chunkX++)
{
for (int chunkZ = targetChunk.y - chunkNeighboursToGenerate; chunkZ < targetChunk.y + chunkNeighboursToGenerate + 1; chunkZ++)
{
List<(Vector2Int, Biome)> generatedCentdroids = GetCentdroidsByChunk(worldSeed, new(chunkX, chunkZ), centroidsPerChunk, chunkSize, multiplier, targetChunk, chunkNeighboursToGenerate);
foreach ((Vector2Int, Biome) generatedCentdroid in generatedCentdroids)
{
centroids.Add(generatedCentdroid);
}
}
}
Biome[,] biomeMap = new Biome[chunkSize, chunkSize];
//---Generate biomeMap of the target Chunk---
for (int tx = 0; tx < chunkSize; tx++)
{
for (int tz = 0; tz < chunkSize; tz++)
{
int x = chunkSize * chunkNeighboursToGenerate + tx;
int z = chunkSize * chunkNeighboursToGenerate + tz;
biomeMap[tz, tx] = GetClosestCentroidBiome(new(x, z), centroids.ToArray());
};
};
//Return the biome map of the target chunk
return biomeMap;
}
private static List<(Vector2Int, Biome)> GetCentdroidsByChunk(string worldSeed, Vector2Int chunkToGenerate, float centroidsPerChunk, int chunkSize, List<(Biome, float)> multiplier, Vector2Int targetChunk, int chunkNeighboursToGenerate)
{
List<(Vector2Int, Biome)> centroids = new();
//---Generate Cendroids of a single chunk---
float centroidsInThisChunk = centroidsPerChunk;
//Init randomizer
System.Random randomInstance = new(Randomizer.GetSeed(worldSeed, chunkToGenerate.x, chunkToGenerate.y));
while (centroidsInThisChunk > 0.0f)
{
//if at least one more centroid is to generate do it
//if not randomize by the given probability if another one should be generated
if (centroidsInThisChunk >= 1 || (float)randomInstance.NextDouble() * (1 - 0) + 0 <= centroidsInThisChunk)
{
//Generate random point for a new centroid
Vector2Int pos = new(randomInstance.Next(0, chunkSize + 1), randomInstance.Next(0, chunkSize + 1));
//map the point to a zerobased coordinatesystem
int mappedX = (((chunkToGenerate.x - targetChunk.x) + chunkNeighboursToGenerate) * chunkSize) + pos.x;
int mappedZ = (((chunkToGenerate.y - targetChunk.y) + chunkNeighboursToGenerate) * chunkSize) + pos.y;
Vector2Int mappedPos = new Vector2Int(mappedX, mappedZ);
//Select the biom randomized
Biome biome = Randomizer.GetRandomBiom(randomInstance, multiplier);
centroids.Add(new(mappedPos, biome));
centroidsInThisChunk -= 1.0f;
}
//if no centroid is left to generate, end the loop
else
{
break;
}
}
return centroids;
}
//Calculates the closest Centroid to the given possition
Biome GetClosestCentroidBiome(Vector2Int pixelPos, IEnumerable<(Vector2Int, Biome)> centroids)
{
//Warp the possition so the biom borders won't be straight
//Vector2 warpedPos = pixelPos + Get2DTurbulence(pixelPos);
Vector2 warpedPos = pixelPos;
float smallestDst = float.MaxValue;
Biome closestBiome = Biome.Empty;
foreach ((Vector2Int, Biome) centroid in centroids)
{
float distance = Vector2.Distance(warpedPos, centroid.Item1);
if (distance < smallestDst)
{
smallestDst = distance;
closestBiome = centroid.Item2;
}
}
return closestBiome;
}
public static class Randomizer
{
//Generates a random integerseed by combining an hashing the inputvalues
public static int GetSeed(string worldSeed, int chunkx, int chunkz)
{
var stringSeed = worldSeed + ":" + chunkx + ";" + chunkz;
MD5 md5Hasher = MD5.Create();
byte[] hashed = md5Hasher.ComputeHash(Encoding.UTF8.GetBytes(stringSeed));
return BitConverter.ToInt32(hashed, 0);
}
//Returns a random biome based on the given properbilities/multiplier
//multiplier = 2 for example means the biom is generated twice as often as usually
public static Biome GetRandomBiom(System.Random rndm, List<(Biome, float)> multiplier)
{
float multmax = 0.0f;
multiplier.ForEach(x => multmax += x.Item2);
//Generate a random value that is in the range of all multiplieres added
float biome = (float)rndm.NextDouble() * (multmax + 0.01f);
//Map the biome to the multipliers and return the biome
float multcalc = 0.0f;
for (int r = 0; r < multiplier.Count; r++)
{
multcalc += multiplier[r].Item2;
if (multcalc >= biome)
{
return multiplier[r].Item1;
}
}
//Return Biome.Empty if something did't worked correct
return Biome.Empty;
}
}
This doesn't work:
private Biome[,] Generate(string worldSeed, Vector2Int targetChunk, List<(Biome, float)> multiplier, float centroidsPerChunk)
{
//Calculate the NeighboursToGenerate depeding on the cendroids per Chunk value
int chunkNeighboursToGenerate = (int)Math.Ceiling(Math.Sqrt(1f / centroidsPerChunk * 12.5f));
int chunkSize = 8;
//Create List that contains all centroids of the chunk
ConcurrentBag<(Vector2Int, Biome)> centroids = new();
ConcurrentQueue<Task> tasks = new();
//Create Centdroids for every chunk of the generated region around the targetchunk
for (int chunkX = targetChunk.x - chunkNeighboursToGenerate; chunkX < targetChunk.x + chunkNeighboursToGenerate + 1; chunkX++)
{
for (int chunkZ = targetChunk.y - chunkNeighboursToGenerate; chunkZ < targetChunk.y + chunkNeighboursToGenerate + 1; chunkZ++)
{
tasks.Enqueue(Task.Run(() =>
{
List<(Vector2Int, Biome)> generatedCentdroids = GetCentdroidsByChunk(worldSeed, new(chunkX, chunkZ), centroidsPerChunk, chunkSize, multiplier, targetChunk, chunkNeighboursToGenerate);
foreach ((Vector2Int, Biome) generatedCentdroid in generatedCentdroids)
{
centroids.Add(generatedCentdroid);
}
}));
}
}
Biome[,] biomeMap = new Biome[chunkSize, chunkSize];
Task.WaitAll(tasks.ToArray());
//---Generate biomeMap of the target Chunk---
for (int tx = 0; tx < chunkSize; tx++)
{
for (int tz = 0; tz < chunkSize; tz++)
{
int x = chunkSize * chunkNeighboursToGenerate + tx;
int z = chunkSize * chunkNeighboursToGenerate + tz;
biomeMap[tz, tx] = GetClosestCentroidBiome(new(x, z), centroids.ToArray());
};
};
//Return the biome map of the target chunk
return biomeMap;
}

If you're starting to get into programming and you want to learn multi-threading, converting a large piece of complex code like this is not where you want to start. I highly recommend you pick up a book or tutorial on threading/async in C#/.NET before starting something like this. Unity also has its own multi-threading library with its Job System, which is built for the Unity workflow: https://docs.unity3d.com/Manual/JobSystemMultithreading.html
I don't think most people could find what's causing the problem in these two code snippets alone. But I have a couple of suggestions
Change your tasks collection to a List<T>, tasks is only ever accessed on one thread so there's no need to use ConcurrentQueue<T>
Is Biome a class? Cause if so it's technically fine but modifying data structures from multiple threads gets hairy fast. And while I can't see that you're modifying data from these snippets, without the full code I can't say for sure. Turn Biome into a struct or make a struct equivalent for threading purposes.
Also avoid calling centroids.ToArray() in your loop, as doing so will actually copy the original array over and over and over again. Call it once outside of your loop and that alone should be a pretty huge performance bump.
Just find a full-blown tutorial for threading/async/Unity's Job system (depending on which you'd rather learn for your use case) and start from there, I can tell from your use of the concurrent libraries and List<T> inside your tasks that you're new to threading. Understanding what code is ran on another thread and the repercussions from that (race conditions, and so on) is huge.

BinaryReader returning very large doubles. What type of value should I be returning?

I'm trying to read a binaryfile. I don't know the structure, but I do have some code written in R that can read it. I'm not familiar with R but have made some progress converting it to C# and struggle at the last bit.
I'm at a point where I need to list out results which I would expect to be a series of float or double.
The R code looks like this (I've removed some of the logic to keep it short):
Rcpp::NumericVector GetSwmmResult(int iType, int iIndex, int vIndex)
{
int offset;
std::vector<float> resultvec(SWMM_Nperiods);
size_t size;
// --- compute offset into output file
for ( int i=1; i<=SWMM_Nperiods; ++i)
{
offset = StartPos + (i-1)*BytesPerPeriod + 2*RECORDSIZE;
if ( iType == SUBCATCH )
{
offset += RECORDSIZE*(iIndex*SubcatchVars + vIndex);
}
else return wrap(resultvec);
// --- re-position the file and read the result
fseek(Fout, offset, SEEK_SET);
size = fread(&resultvec[i-1], RECORDSIZE, 1, Fout);
}
return wrap(resultvec);
}
In C# I expected to do something as follows, where br is my BinaryReader object:
public List<double> GetSwmmResult(int iType, int iIndex, int vIndex)
{
int offset;
List<double> resultvec = new();
int size;
// --- compute offset into output file
Debug.WriteLine("SWMM_Nperiods count = " + SWMM_Nperiods);
for (int i = 1; i <= SWMM_Nperiods; i++)
{
Debug.WriteLine("SWMM_Nperiods " + i);
offset = StartPos + (i - 1) * BytesPerPeriod + 2 * RECORDSIZE;
if (iType == SUBCATCH)
{
offset += RECORDSIZE * (iIndex * SubcatchVars + vIndex);
}
else
{
return resultvec;
}
// --- re-position the file and read the result
br.BaseStream.Position = offset;
resultvec.Add(br.ReadDouble());
Debug.WriteLine(resultvec[i - 1]);
}
return resultvec;
}
But my C# just returns a load of very large numbers like:
5.058993159887922E-15
3.10628841909217E-16
5.477524451492502E-17
I'm expecting a series of numbers, but in the 100's or 1000's, not such large numbers.
Can anybody see how I should be returning values in my C# code using the R function above as a guide? There's a variable in the R code called SEEK_SET. It's not declared anywhere, so I don't understand how it's being used, but suspect it may be what I'm missing.

Accessing processed values from FFT

I am attempting to use Lomont FFT in order to return complex numbers to build a spectrogram / spectral density chart using c#.
I am having trouble understanding how to return values from the class.
Here is the code I have put together thus far which appears to be working.
int read = 0;
Double[] data;
byte[] buffer = new byte[1024];
FileStream wave = new FileStream(args[0], FileMode.Open, FileAccess.Read);
read = wave.Read(buffer, 0, 44);
read = wave.Read(buffer, 0, 1024);
data = new Double[read];
for (int i = 0; i < read; i+=2)
{
data[i] = BitConverter.ToInt16(buffer, i) / 32768.0;
Console.WriteLine(data[i]);
}
LomontFFT LFFT = new LomontFFT();
LFFT.FFT(data, true);
What I am not clear on is, how to return/access the values from Lomont FFT implementation back into my application (console)?
Being pretty new to c# development, I'm thinking I am perhaps missing a fundamental aspect of understanding regarding how to retrieve processed values from the instance of the Lomont Class, or perhaps even calling it incorrectly.
Console.WriteLine(LFFT.A); // Returns 0
Console.WriteLine(LFFT.B); // Returns 1
I have been searching for a code snippet or explanation of how to do this, but so far have come up with nothing that I understand or explains this particular aspect of the issue I am facing. Any guidance would be greatly appreciated.
A subset of the results held in data array noted in the code above can be found below and based on my current understanding, appear to be valid:
0.00531005859375
0.0238037109375
0.041473388671875
0.0576171875
0.07183837890625
0.083465576171875
0.092193603515625
0.097625732421875
0.099639892578125
0.098114013671875
0.0931396484375
0.0848388671875
0.07354736328125
0.05963134765625
0.043609619140625
0.026031494140625
0.007476806640625
-0.011260986328125
-0.0296630859375
-0.047027587890625
-0.062713623046875
-0.076141357421875
-0.086883544921875
-0.09454345703125
-0.098785400390625
-0.0994873046875
-0.0966796875
-0.090362548828125
-0.080810546875
-0.06842041015625
-0.05352783203125
-0.036712646484375
-0.0185546875
What am I actually attempting to do? (perspective)
I am looking to load a wave file into a console application and return a spectrogram/spectral density chart/image as a jpg/png for further processing.
The wave files I am reading are mono in format
UPDATE 1
I Receive slightly different results depending on which FFT is used.
Using RealFFT
for (int i = 0; i < read; i+=2)
{
data[i] = BitConverter.ToInt16(buffer, i) / 32768.0;
//Console.WriteLine(data[i]);
}
LomontFFT LFFT = new LomontFFT();
LFFT.RealFFT(data, true);
for (int i = 0; i < buffer.Length / 2; i++)
{
System.Console.WriteLine("{0}",
Math.Sqrt(data[2 * i] * data[2 * i] + data[2 * i + 1] * data[2 * i + 1]));
}
Partial Result of RealFFT
0.314566983321381
0.625242818210924
0.30314888696868
0.118468857708093
0.0587697011760449
0.0369034115568654
0.0265842582236275
0.0207195964060356
0.0169601273233317
0.0143745438577886
0.012528799609089
0.0111831275153128
0.0102313284519146
0.00960198279358434
0.00920236001619566
Using FFT
for (int i = 0; i < read; i+=2)
{
data[i] = BitConverter.ToInt16(buffer, i) / 32768.0;
//Console.WriteLine(data[i]);
}
double[] bufferB = new double[2 * data.Length];
for (int i = 0; i < data.Length; i++)
{
bufferB[2 * i] = data[i];
bufferB[2 * i + 1] = 0;
}
LomontFFT LFFT = new LomontFFT();
LFFT.FFT(bufferB, true);
for (int i = 0; i < bufferB.Length / 2; i++)
{
System.Console.WriteLine("{0}",
Math.Sqrt(bufferB[2 * i] * bufferB[2 * i] + bufferB[2 * i + 1] * bufferB[2 * i + 1]));
}
Partial Result of FFT:
0.31456698332138
0.625242818210923
0.303148886968679
0.118468857708092
0.0587697011760447
0.0369034115568653
0.0265842582236274
0.0207195964060355
0.0169601273233317
0.0143745438577886
0.012528799609089
0.0111831275153127
0.0102313284519146
0.00960198279358439
0.00920236001619564

Looking at the LomontFFT.FFT documentation:
Compute the forward or inverse Fourier Transform of data, with
data containing complex valued data as alternating real and
imaginary parts. The length must be a power of 2. The data is
modified in place.
This tells us a few things. First the function is expecting complex-valued data whereas your data is real. A quick fix for this is to create another buffer of twice the size and setting all the imaginary parts to 0:
double[] buffer = new double[2*data.Length];
for (int i=0; i<data.Length; i++)
{
buffer[2*i] = data[i];
buffer[2*i+1] = 0;
}
The documentation also tells us that the computation is done in place. That means that after the call to FFT returns, the input array is replaced with the computed result. You could thus print the spectrum with:
LomontFFT LFFT = new LomontFFT();
LFFT.FFT(buffer, true);
for (int i = 0; i < buffer.Length/2; i++)
{
System.Console.WriteLine("{0}",
Math.Sqrt(buffer[2*i]*buffer[2*i]+buffer[2*i+1]*buffer[2*i+1]));
}
Note since your input data is real valued you could also use LomontFFT.RealFFT. In that case, given a slightly different packing rule, you would obtain the FFT results using:
LomontFFT LFFT = new LomontFFT();
LFFT.RealFFT(data, true);
System.Console.WriteLine("{0}", Math.Abs(data[0]);
for (int i = 1; i < data.Length/2; i++)
{
System.Console.WriteLine("{0}",
Math.Sqrt(data[2*i]*data[2*i]+data[2*i+1]*data[2*i+1]));
}
System.Console.WriteLine("{0}", Math.Abs(data[1]);
This would give you the non-redundant lower half of the spectrum (Unlike LomontFFT.FFT which provides the entire spectrum). Also, numerical differences on the order of double precision (around 1e-16 times the spectrum peak value) with respect to LomontFFT.FFT can be expected.

error when trying to read from an array

I am trying too have my C# console application make a beep sound. Yes I know I can use Console.Beep but I also want to lower the volume etc.
But the error I am getting is this:
Method name expected
on this line:
binaryWriter.Write(hdr(i));
This is my code:
private bool Beep(int volume, int frequency, int duration)
{
try
{
double amplitude = volume * 1.27;
double a = ((amplitude * (System.Math.Pow(2, 15))) / 1000) - 1;
double deltaFt = 2 * System.Math.PI * frequency / 8000;
double samples = 441 * (duration / 100);
int bytes = Convert.ToInt32(samples) * 4;
int[] hdr = {
0x46464952,
36 + bytes,
0x45564157,
0x20746d66,
16,
0x20001,
8000,
176400,
0x100004,
0x61746164,
bytes
};
using (System.IO.MemoryStream memoryStream = new System.IO.MemoryStream(44 + bytes))
{
using (System.IO.BinaryWriter binaryWriter = new System.IO.BinaryWriter(memoryStream))
{
for (int i = 0; i <= hdr.Length - 1; i++)
{
binaryWriter.Write(hdr(i));
}
for (int T = 0; T <= Convert.ToInt32(samples) - 1; T++)
{
short sample = Convert.ToInt16(a * System.Math.Sin(deltaFt * T));
binaryWriter.Write(sample);
binaryWriter.Write(sample);
}
binaryWriter.Flush();
memoryStream.Seek(0, System.IO.SeekOrigin.Begin);
using (System.Media.SoundPlayer sp = new System.Media.SoundPlayer(memoryStream))
{
sp.PlaySync();
}
}
}
}
catch
{
return false;
}
return true;
}

Your hdr is an array, you need to get the entry by putting the square brackets and then passing the index
binaryWriter.Write(hdr[i]);

CPU High Usage with time for serial port

Iam currently working on a project where i have to read serial port continuously. The data is coming continuously for a max of 45 min. I have to validate the data via checksum and create a packet of 79 bytes. After this i have to plot the data(Trajectory) on a real time basis. The problem with the code is that at start it uses 20% of the CPU usage(Pentium 4, 3.0 GHz, Hyper threading)(which i think is still high) but with time the CPU usage increases and at end it reaches to 60%.
The data is coming in with a baud rate of 115200 and is sent continuously at the rate 100 msec.
My code for reading, validating and plotting is as follows:
The following function receive the data and validate it...
private void serialPort1_DataReceived(object sender, SerialDataReceivedEventArgs e)
{
try
{
header1 = serialPort1.ReadByte();
if (header1 == 0)
header2 = serialPort1.ReadByte();
if ((header1 == 0) && (header2 == 1))//Store the data in an array.
{
for (int i = 0; i < 77; i++)
abudata[i] = serialPort1.ReadByte();
tail = abudata[76];
}
else
{
serialPort1.DiscardInBuffer();
}
checksum = 1;// Calculate the checksum.
for (i = 0; i < 74; i++)
checksum = checksum + (abudata[i]);
checksum1 = (abudata[75] << 8);
checksum1 = checksum1 + (abudata[74]);
if ((checksum == checksum1) && (tail == 4))
this.Invoke(new EventHandler(Display_Results));// Function to display
}
catch (Exception ode)
{
l4[4].BackColor = Color.Red;
}
}
The following function display the data on labels and draw the trajectory on a picture box
private void Display_Results(object s, EventArgs e)
{
head1[0] = header1;
head1[1] = header2;
for (k = 0; k < 77; ++k)
head1[k + 2] = (((int)abudata[k]) & 0x000000ff);
jk = 0;
for (k = 0; k < 36; ++k) //Data packing into 36 bytes
{
num_1[k] = (ulong)((head1[jk + 1]) + (head1[jk] << 8)) & 0x0000ffff;
num_1[k] = (double)num_1[k];
num_2[k] = (double)num_1[k];
jk = jk + 2;
signbit = (int)num_1[k] >> 15;
if (signbit == 1)
{
sgnval = -1;
num_1[k] = num_1[k] - 65535;
num_1[k] = num_1[k] * (-1.0);
}
else
sgnval = 1;
//Converting the data into engineering values
engval[k] = Math.Round(num_1[k] * parammaxval[k] * sgnval / 32767.0, 3);
if (k == 14)
{
try
{
curr_x = (pictureBox2.Width / 2) + (int)((engval[13] * (pictureBox2.Width)) / map_width);
curr_y = (pictureBox2.Height / 2) - (int)((engval[14] * (pictureBox2.Height)) / map_height);
PointF p1 = new Point(curr_x, curr_y);
if (_gPath != null && _gPath.PointCount > 0)
p1 = _gPath.PathPoints[_gPath.PathPoints.Length - 1];
PointF p2 = new Point(curr_x, curr_y);
_gPath.AddLine(p1, p2);
pictureBox2.Invalidate();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
}
}

I extensively work with serial port connected devices and may assure you, that regular using of SerialPort class by itself doesn't produce high CPU load.
FIRST I would suggest you to PROFILE your application. There are a bunch of profilers for .NET
Only after profiling I would suggest to decouple SerialPort reading and data processing. Use Producer Consumer pattern. Put data from SerialPort to queue and consume it from other thread.
That what I have in SerialPortDataReceived function in one of my projects
private void SerialPortDataReceived(object sender, SerialDataReceivedEventArgs e)
{
lock (SyncObject)
{
if (!_serialPort.IsOpen) return;
try
{
int toread = _serialPort.BytesToRead;
byte[] bytes = new byte[toread];
_serialPort.Read(bytes, 0, toread);
ProducerAddBytes(bytes);
}
catch (TimeOutException)
{
//logic
}
}
}
P.S. But PROFILE FIRST!

i got what was the problem with the above code..
I am using Graph path for plotting the trajectory
if (k == 14)
{
try
{
curr_x = (pictureBox2.Width / 2) + (int)((engval[13] * (pictureBox2.Width)) / map_width);
curr_y = (pictureBox2.Height / 2) - (int)((engval[14] * (pictureBox2.Height)) / map_height);
PointF p1 = new Point(curr_x, curr_y);
if (_gPath != null && _gPath.PointCount > 0)
p1 = _gPath.PathPoints[_gPath.PathPoints.Length - 1];
PointF p2 = new Point(curr_x, curr_y);
_gPath.AddLine(p1, p2);
pictureBox2.Invalidate();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
Now as the application continues to run, it gathers lots of graph points and hence plotting this huge number of point is consuming the resources.
Can anybody suggest me solution to this problem that how to plot the trajectory without slowing the system...