We Keep Coding

Looping over 3D array in parallel.for

I have a 3D array with arbitrary X, Y and Z lengths
I want to iterate over it in a parallel.for loop, which can't be nested without wasting tasks afaik
Instead, the single loop's length is ArrayLengthX * ArrayLengthY * ArrayLengthZ
Can I mathematically get the current 3D array element from the current iteration + the X, Y and Z length of the array? if so how?
edit : Example below, hope this is enough to understand what's going on
DimensionSettings dimensionSettings = dimension.Settings;
Vector3Int DimSize = dimensionSettings.GetDimensionSize();
TaskProgressMutex.WaitOne();
CurrentProgress = 0;
TotalProgress = DimSize.x * DimSize.y * DimSize.z;
TaskProgressMutex.ReleaseMutex();
int ChunkAmount = DimSize.x * DimSize.y * DimSize.z;
GD.Print("Chunks to generate : " + ChunkAmount);
ParallelLoopResult result = Parallel.For(0, ChunkAmount, (int i) =>
{
GD.Print(i);
int xcoords = ???;
int ycoords = ???;
int zcoords = ???;
Vector3Int ChunkCoords = new Vector3Int(xcoords, ycoords, zcoords);
GD.Print("Current Chunk : " + xcoords + " " + ycoords + " " + zcoords);
GenerateChunk(ChunkCoords, seed);
TaskProgressMutex.WaitOne();
CurrentProgress += 1;
//GD.Print("Chunk " + xcoords + " " + ycoords + " " + zcoords + " finished generating. Current Progress : " + CurrentProgress);
TaskProgressMutex.ReleaseMutex();
});

My suggestion is to parallelize only the outer loop (the X dimension), and do normal (non-parallel) inner loops for the Y and Z dimensions:
ParallelOptions options = new()
{
MaxDegreeOfParallelism = Environment.ProcessorCount
};
Parallel.For(0, dimSize.X, options, (int x) =>
{
for (int y = 0; y < dimSize.Y; y++)
{
for (int z = 0; z < dimSize.Z; z++)
{
Vector3Int chunkCoords = new (x, y, z);
GenerateChunk(chunkCoords, seed);
}
}
});
The idea is to increase the chunkiness of the parallel work. In general the more chunky is the workload, the less is the overall overhead of the parallelization. At the other extreme, if each unit of work is too chunky, the partitioning of the work may become imbalanced.
Also when you use the Parallel.For/Parallel.ForEach/Parallel.Invoke methods I am suggesting to specify always the MaxDegreeOfParallelism, although this suggestion deviates from the general advice offered in the official documentation.

Fast comparison of two doubles ignoring everything that comes after 6 decimal digits

I try to optimize the performance of some calculation process.
Decent amount of time is wasted on calculations like the following:
var isBigger = Math.Abs((long) (a * 1e6) / 1e6D) > ((long) ((b + c) * 1e6)) / 1e6D;
where "a","b" and "c" are doubles, "b" and "c" are positive, "a" might be negative.
isBigger should be true only if absolute value of "a" is bigger than "b+c" disregarding anything after the 6th decimal digit.
So I look at this expression, I understand what it does, but it seems hugely inefficient to me, since it multiplies and divides compared numbers by million just to get rig of anything after 6 decimal places.
Below is the program I used to try and create a better solution. So far I failed.
Can someone help me?
class Program
{
static void Main(string[] args)
{
var arrLength = 1000000;
var arr1 = GetArrayOf_A(arrLength);
var arr2 = GetArrayOf_B(arrLength);
var arr3 = GetArrayOf_C(arrLength);
var result1 = new bool[arrLength];
var result2 = new bool[arrLength];
var sw = new Stopwatch();
sw.Start();
for (var i = 0; i < arrLength; i++)
{
result1[i] = Math.Abs((long) (arr1[i] * 1e6) / 1e6D)
>
(long) ((arr2[i] + arr3[i]) * 1e6) / 1e6D;
}
sw.Stop();
var t1 = sw.Elapsed.TotalMilliseconds;
sw.Restart();
for (var i = 0; i < arrLength; i++)
{
//result2[i] = Math.Round(Math.Abs(arr1[i]) - (arr2[i] + arr3[i]),6) > 0; // Incorrect, example by index = 0
//result2[i] = Math.Abs(arr1[i]) - (arr2[i] + arr3[i]) > 0.000001; // Incorrect, example by index = 1
//result2[i] = Math.Abs(arr1[i]) - (arr2[i] + arr3[i]) > 0.0000001; // Incorrect, example by index = 2
result2[i] = Math.Abs(arr1[i]) - (arr2[i] + arr3[i]) > 0.00000001; // Incorrect, example by index = 3
}
sw.Stop();
var t2 = sw.Elapsed.TotalMilliseconds;
var areEquivalent = true;
for (var i = 0; i < arrLength; i++)
{
if (result1[i] == result2[i]) continue;
areEquivalent = false;
break;
}
Console.WriteLine($"Functions are equivalent : {areEquivalent}");
if (areEquivalent)
{
Console.WriteLine($"Current function total time: {t1}ms");
Console.WriteLine($"Equivalent function total time: {t2}ms");
}
Console.WriteLine("Press ANY key to quit . . .");
Console.ReadKey();
}
private static readonly Random _rand = new Random(DateTime.Now.Millisecond);
private const int NumberOfRepresentativeExamples = 4;
private static double[] GetArrayOf_A(int arrLength)
{
if(arrLength<=NumberOfRepresentativeExamples)
throw new ArgumentException($"{nameof(arrLength)} should be bigger than {NumberOfRepresentativeExamples}");
var arr = new double[arrLength];
// Representative numbers
arr[0] = 2.4486382579120365;
arr[1] = -1.1716818990000011;
arr[2] = 5.996414627393257;
arr[3] = 6.0740085822069;
// the rest is to build time statistics
FillTheRestOfArray(arr);
return arr;
}
private static double[] GetArrayOf_B(int arrLength)
{
if(arrLength<=NumberOfRepresentativeExamples)
throw new ArgumentException($"{nameof(arrLength)} should be bigger than {NumberOfRepresentativeExamples}");
var arr = new double[arrLength];
// Representative numbers
arr[0] = 2.057823225;
arr[1] = 0;
arr[2] = 2.057823225;
arr[3] = 2.060649901;
// the rest is to build time statistics
FillTheRestOfArray(arr);
return arr;
}
private static double[] GetArrayOf_C(int arrLength)
{
if(arrLength<=NumberOfRepresentativeExamples)
throw new ArgumentException($"{nameof(arrLength)} should be bigger than {NumberOfRepresentativeExamples}");
var arr = new double[arrLength];
// Representative numbers
arr[0] = 0.3908145999796302;
arr[1] = 1.1716809269999997;
arr[2] = 3.9385910820740282;
arr[3] = 4.0133582670728858;
// the rest is to build time statistics
FillTheRestOfArray(arr);
return arr;
}
private static void FillTheRestOfArray(double[] arr)
{
for (var i = NumberOfRepresentativeExamples; i < arr.Length; i++)
{
arr[i] = _rand.Next(0, 10) + _rand.NextDouble();
}
}
}

You don't need the division since if (x/100) < (y/100) that means that x<y.
for(var i = 0; i < arrLength; i++)
{
result2[i] = Math.Abs((long)(arr1[i] * 1e6))
> (long)((arr2[i] + arr3[i]) * 1e6);
}
with the results for me:
Arrays have 1000000 elements.
Functions are equivalent : True
Current function total time: 40.10ms 24.94 kflop
Equivalent function total time: 22.42ms 44.60 kflop
A speedup of 78.83 %
PS. Make sure you compare RELEASE versions of the binary which includes math optimizations.
PS2. The display code is
Console.WriteLine($"Arrays have {arrLength} elements.");
Console.WriteLine($"Functions are equivalent : {areEquivalent}");
Console.WriteLine($" Current function total time: {t1:F2}ms {arrLength/t1/1e3:F2} kflop");
Console.WriteLine($"Equivalent function total time: {t2:F2}ms {arrLength/t2/1e3:F2} kflop");
Console.WriteLine($"An speedup of {t1/t2-1:P2}");

Overall your question goes into the Area of Realtime Programming. Not nessesarily realtime constraint, but it goes into teh same optimisation territory. The kind where every last nanosecond has be shaved off.
.NET is not the ideal scenario for this kind of operation. Usually that thing is done in dedicated lanagauges. The next best thing is doing it in Assembler, C or native C++. .NET has additional features like the Garbage Collector and Just In Time compiler that make even getting reliable benchmark results tricky. Much less reliale runtime performance.
For the datatypes, Float should be about the fastest operation there is. For historical reasons float opeations have been optimized.
One of your comment mentions physics and you do have an array. And I see stuff like array[i] = array2[i] + array3[i]. So maybe this should be a matrix operation you run on the GPU instead? This kind of "huge paralellized array opeartions" is exactly what the GPU is good at. Exactly what drawing on the screen is at it's core.
Unless you tell us what you are actually doing here as sa operation, that is about the best answer I can give.

Is this what you're looking for?
Math.Abs(a) - (b + c) > 0.000001
or if you want to know if the difference is bigger (difference either way):
Math.Abs(Math.Abs(a) - (b + c)) > 0.000001
(I'm assuming you're not limiting to this precision because of speed but because of inherent floating point limited precision.)

In addition to asking this question on this site, I also asked a good friend of mine, and so far he provided the best answer. Here it is:
result2[i] = Math.Abs(arr1[i]) - (arr2[i] + arr3[i]) > 0.000001 ||
Math.Abs((long)(arr1[i] * 1e6)) > (long)((arr2[i] + arr3[i])*1e6);
I am happy to have such friends :)

Shortest way between two points by going through all other points from point collection

Given a point collection defined by x and y coordinates.
In this collection I get the start point, the end point and all the other n-2 points.
I have to find the shortest way between the start point and end point by going through all the other points. The shortest way is defined by its value and if possible the crossing point order.
At a first look this seems to be a graph problem, but i am not so sure about that right now, any way i am trying to find this shortest way by using only geometric relations since currently all the information that i have is only the x and y coordinates of the points, and which point is the start point and which is the end point.
My question is, can this way be found by using only geometric relations?
I am trying to implement this in C#, so if some helpful packages are available please let me know.

The simplest heuristic with reasonable performance is 2-opt. Put the points in an array, with the start point first and the end point last, and repeatedly attempt to improve the solution as follows. Choose a starting index i and an ending index j and reverse the subarray from i to j. If the total cost is less, then keep this change, otherwise undo it. Note that the total cost will be less if and only if d(p[i - 1], p[i]) + d(p[j], p[j + 1]) > d(p[i - 1], p[j]) + d(p[i], p[j + 1]), so you can avoid performing the swap unless it's an improvement.
There are a possible number of improvements to this method. 3-opt and k-opt consider more possible moves, resulting in better solution quality. Data structures for geometric search, kd-trees for example, decrease the time to find improving moves. As far as I know, the state of the art in local search algorithms for TSP is Keld Helsgaun's LKH.
Another family of algorithms is branch and bound. These return optimal solutions. Concorde (as far as I know) is the state of the art here.
Here's a Java implementation of the O(n^2 2^n) DP that Niklas described. There are many possible improvements, e.g., cache the distances between points, switch to floats (maybe), reorganize the iteration so that subsets are enumerating in increasing order of size (to allow only the most recent layer of minTable to be retained, resulting in a significant space saving).
class Point {
private final double x, y;
Point(double x, double y) {
this.x = x;
this.y = y;
}
double distanceTo(Point that) {
return Math.hypot(x - that.x, y - that.y);
}
public String toString() {
return x + " " + y;
}
}
public class TSP {
public static int[] minLengthPath(Point[] points) {
if (points.length < 2) {
throw new IllegalArgumentException();
}
int n = points.length - 2;
if ((1 << n) <= 0) {
throw new IllegalArgumentException();
}
byte[][] argMinTable = new byte[1 << n][n];
double[][] minTable = new double[1 << n][n];
for (int s = 0; s < (1 << n); s++) {
for (int i = 0; i < n; i++) {
int sMinusI = s & ~(1 << i);
if (sMinusI == s) {
continue;
}
int argMin = -1;
double min = points[0].distanceTo(points[1 + i]);
for (int j = 0; j < n; j++) {
if ((sMinusI & (1 << j)) == 0) {
continue;
}
double cost =
minTable[sMinusI][j] +
points[1 + j].distanceTo(points[1 + i]);
if (argMin < 0 || cost < min) {
argMin = j;
min = cost;
}
}
argMinTable[s][i] = (byte)argMin;
minTable[s][i] = min;
}
}
int s = (1 << n) - 1;
int argMin = -1;
double min = points[0].distanceTo(points[1 + n]);
for (int i = 0; i < n; i++) {
double cost =
minTable[s][i] +
points[1 + i].distanceTo(points[1 + n]);
if (argMin < 0 || cost < min) {
argMin = i;
min = cost;
}
}
int[] path = new int[1 + n + 1];
path[1 + n] = 1 + n;
int k = n;
while (argMin >= 0) {
path[k] = 1 + argMin;
k--;
int temp = s;
s &= ~(1 << argMin);
argMin = argMinTable[temp][argMin];
}
path[0] = 0;
return path;
}
public static void main(String[] args) {
Point[] points = new Point[20];
for (int i = 0; i < points.length; i++) {
points[i] = new Point(Math.random(), Math.random());
}
int[] path = minLengthPath(points);
for (int i = 0; i < points.length; i++) {
System.out.println(points[path[i]]);
System.err.println(points[i]);
}
}
}

The Euclidean travelling salesman problem can be reduced to this and it's NP-hard. So unless your point set is small or you have a very particular structure, you should probably look out for an approximation. Note that the Wikipedia article mentions the existence of a PTAS for the problem, which could turn out to be quite effective in practice.
UPDATE: Since your instances seem to have only few nodes, you can use a simple exponential-time dynamic programming approach. Let f(S, p) be the minimum cost to connect all the points in the set S, ending at the points p. We have f({start}, start) = 0 and we are looking for f(P, end), where P is the set of all points. To compute f(S, p), we can check all potential predecessors of p in the tour, so we have
f(S, p) = MIN(q in S \ {p}, f(S \ {p}, q) + distance(p, q))
You can represent S as a bitvector to save space (just use an single-word integer for maximum simplicity). Also use memoization to avoid recomputing subproblem results.
The runtime will be O(2^n * n^2) and the algorithm can be implemented with a rather low constant factor, so I predict it to be able to solve instance with n = 25 within seconds a reasonable amount of time.

This can be solved using an evolutionary algorithm.
Look at this: http://johnnewcombe.net/blog/post/23

You might want to look at TPL (Task Parallel Library) to speed up the application.
EDIT
I found this Link which has a Traveling Salesman algorithm:
http://msdn.microsoft.com/en-us/magazine/gg983491.aspx
The Source Code is said to be at:
http://archive.msdn.microsoft.com/mag201104BeeColony

How to populate two separate arrays from one comma-delimited list?

I have a comma delimited text file that contains 20 digits separated by commas. These numbers represent earned points and possible points for ten different assignments. We're to use these to calculate a final score for the course.
Normally, I'd iterate through the numbers, creating two sums, divide and be done with it. However, our assignment dictates that we load the list of numbers into two arrays.
so this:
10,10,20,20,30,35,40,50,45,50,45,50,50,50,20,20,45,90,85,85
becomes this:
int[10] earned = {10,20,30,40,45,50,20,45,85};
int[10] possible = {10,20,35,50,50,50,20,90,85};
Right now, I'm using
for (x=0;x<10;x++)
{
earned[x] = scores[x*2]
poss [x] = scores[(x*2)+1]
}
which gives me the results I want, but seems excessively clunky.
Is there a better way?

The following should split each alternating item the list into the other two lists.
int[20] scores = {10,10,20,20,30,35,40,50,45,50,45,50,50,50,20,20,45,90,85,85};
int[10] earned;
int[10] possible;
int a = 0;
for(int x=0; x<10; x++)
{
earned[x] = scores[a++];
possible[x] = scores[a++];
}

You can use LINQ here:
var arrays = csv.Split(',')
.Select((v, index) => new {Value = int.Parse(v), Index = index})
.GroupBy(g => g.Index % 2,
g => g.Value,
(key, values) => values.ToArray())
.ToList();
and then
var earned = arrays[0];
var possible = arrays[1];

Get rid of the "magic" multiplications and illegible array index computations.
var earned = new List<int>();
var possible = new List<int>();
for (x=0; x<scores.Length; x += 2)
{
earned.Add(scores[x + 0]);
possible.Add(scores[x + 1]);
}
This has very little that would need a text comment. This is the gold standard for self-documenting code.
I initially thought the question was a C question because of all the incomprehensible indexing. It looked like pointer magic. It was too clever.
In my codebases I usually have an AsChunked extension available that splits a list into chunks of the given size.
var earned = new List<int>();
var possible = new List<int>();
foreach (var pair in scores.AsChunked(2)) {
earned.Add(pair[0]);
possible.Add(pair[1]);
}
Now the meaning of the code is apparent. The magic is gone.
Even shorter:
var pairs = scores.AsChunked(2);
var earned = pairs.Select(x => x[0]).ToArray();
var possible = pairs.Select(x => x[1]).ToArray();

I suppose you could do it like this:
int[] earned = new int[10];
int[] possible = new int[10];
int resultIndex = 0;
for (int i = 0; i < scores.Count; i = i + 2)
{
earned[resultIndex] = scores[i];
possible[resultIndex] = scores[i + 1];
resultIndex++;
}
You would have to be sure that an equal number of values are stored in scores.

I would leave your code as is. You are technically expressing very directly what your intent is, every 2nd element goes into each array.
The only way to improve that solution is to comment why you are multiplying. But I would expect someone to quickly recognize the trick, or easily reproduce what it is doing. Here is an excessive example of how to comment it. I wouldn't recommend using this directly.
for (x=0;x<10;x++)
{
//scores contains the elements inline one after the other
earned[x] = scores[x*2] //Get the even elements into earned
poss [x] = scores[(x*2)+1] //And the odd into poss
}
However if you really don't like the multiplication, you can track the scores index separately.
int i = 0;
for (int x = 0; x < 10; x++)
{
earned[x] = scores[i++];
poss [x] = scores[i++];
}
But I would probably prefer your version since it does not depend on the order of the operations.

var res = grades.Select((x, i) => new {x,i}).ToLookup(y=>y.i%2, y=>y.x)
int[] earned = res[0].ToArray();
int[] possible = res[1].ToArray();
This will group all grades into two buckets based on index, then you can just do ToArray if you need result in array form.

here is an example of my comment so you do not need to change the code regardless of the list size:
ArrayList Test = new ArrayList { "10,10,20,20,30,35,40,50,45,50,45,50,50,50,20,20,45,90,85,85" };
int[] earned = new int[Test.Count / 2];
int[] Score = new int[Test.Count / 2];
int Counter = 1; // start at one so earned is the first array entered in to
foreach (string TestRow in Test)
{
if (Counter % 2 != 0) // is the counter even
{
int nextNumber = 0;
for (int i = 0; i < Score.Length; i++) // this gets the posistion for the next array entry
{
if (String.IsNullOrEmpty(Convert.ToString(Score[i])))
{
nextNumber = i;
break;
}
}
Score[nextNumber] = Convert.ToInt32(TestRow);
}
else
{
int nextNumber = 0;
for (int i = 0; i < earned.Length; i++) // this gets the posistion for the next array entry
{
if (String.IsNullOrEmpty(Convert.ToString(earned[i])))
{
nextNumber = i;
break;
}
}
earned[nextNumber] = Convert.ToInt32(TestRow);
}
Counter++
}

remaining values from HashSet into secondary out

We've been asked to create a program which takes 2 input (which I have parsed) for seats and passengers in a plane and randomly place passengers on seats in a plane in one output as well as placing the remaining seats in a secondary output.
I was wondering if there is a simple way to replace the remaining values in HashSet, into the listBoxLedige.
As it works now, the seats are being distributing but the values in the secondary output arent related the the first output.
if(passengers > seats)
{
MessageBox.Show("For mange passagerer");
}
else
{
HashSet<int> check = new HashSet<int>();
for(int i = 0; i <= passengers - 1; i++)
{
int resultat = rnd.Next(1, seats + 1);
while(check.Contains(resultat))
{
resultat = rnd.Next(1, seats + 1);
}
check.Add(resultat);
int[] passagerer01 = new int[passengers];
passagerer01[i] = i+1;
listBoxFulde.Items.Add("Passager #" + passagerer01[i] + "på sæde #" + resultat);
}
HashSet<int> ledige01 = new HashSet<int>();
for(int i = 0; i <= (seats - passengers - 1); i++)
{
int tilbage = rnd.Next(1, seats + 1);
while(ledige01.Contains(tilbage))
{
ledige01.Add(tilbage);
}
listBoxLedige.Items.Add("Sæde #" + tilbage);

I'm not exactly sure I understand your problem, but have you taken a look at the Except LINQ extension method? Judging by your wording ("remaining values"), it might be the right method for you.
Edit Here's how it's done:
IEnumerable<int> empty = allSeats.Except(check);
Note how empty is now a deferred enumerator (unless you do a .ToArray(), .ToList() or similar on it).

Here's what I'd do (see Range and ExceptWith):
HashSet<int> ledige01 = new HashSet<int>(
Enumerable.Range(1, seats));
ledige01.ExceptWith(taken);
Note in generating the seeds you can remove the trial and error by simply shuffling the seats and taking the first N:
var taken = HashSet<int>(Enumerable.Range(1,seats).Shuffle().Take(passengers));
For hints on how to do shuffle, see e.g. Optimal LINQ query to get a random sub collection - Shuffle
As an aside:
int[] passagerer01 = new int[passengers];
passagerer01[i] = i+1;
listBoxFulde.Items.Add("Passager #" + passagerer01[i] + "på sæde #" + resultat);
looks to be something other than you need :) But I'm assuming it's unfinished and you're likely aware of that
A 'fully' edited take:
if(passengers > seats)
{
MessageBox.Show("For mange passagerer");
}
else
{
HashSet<int> taken = new HashSet<int>();
for(int i = 0; i <= passengers - 1; i++)
{
int resultat;
do {
resultat = rnd.Next(1, seats + 1);
} while(taken.Contains(resultat));
taken.Add(resultat);
listBoxFulde.Items.Add("Passager #" + (i+1) + "på sæde #" + resultat);
}
HashSet<int> ledige01 = new HashSet<int>(
Enumerable.Range(1, seats));
ledige01.ExceptWith(taken);
if(passengers > seats)
{
MessageBox.Show("For mange passagerer");
}
else
{
HashSet<int> taken = new HashSet<int>();
for(int i = 0; i <= passengers - 1; i++)
{
int resultat;
do {
resultat = rnd.Next(1, seats + 1);
} while(taken.Contains(resultat));
taken.Add(resultat);
listBoxFulde.Items.Add("Passager #" + (i+1) + "på sæde #" + resultat);
}
HashSet<int> ledige01 = new HashSet<int>(Enumerable.Range(1, seats));
ledige01.ExceptWith(taken);
foreach(var tilbage in ledige01)
listBoxLedige.Items.Add("Sæde #" + tilbage);

Not a hundred percent sure I understand your problem or solution, but are you aware that you declare and initialize passagerer01 once for each passenger, then deterministically (not randomly) assign seat i+1 to passenger i and afterwards throw away that array? If you wanted to keep the information, you'd have to declare the array outside the for loop.
Also, it doesn't really seem like you're doing anything meaningful in that second part of your code. To determine the empty seats, it would make sense to go through numbers 1 to passengers, check if they are in the set check and, if not, add them to set ledige01. Or, of course, to do something equivalent using a library method as suggested by sehe.
As a last note, in computer science you usually start counting from zero. Thus, you would usually have seat numbers 0 to seats-1 and choose seats randomly like so: rnd.Next(0, seats). And you would usually loop like this: for(int i = 0; i < passengers; i++) instead of for(int i = 0; i <= passengers - 1; i++).