We Keep Coding

Calculating floating point error bound

I have geometrical algorithms and im struggling with floating point inaccuracies.
For example, I'm calculating wether a point lies on the left/right/on a plane (C#):
const double Epsilon = 1e-10;
internal double ComputeDistance(Vector3D v)
{
Vector3D normal = Plane.Normal;
Vertex v0 = Plane.Origin;
double a = normal.X;
double b = normal.Y;
double c = normal.Z;
double d = -(a * v0.X + b * v0.Y + c * v0.Z);
return a * v.X + b * v.Y + c * v.Z + d;
}
internal string DistanceSign(Vector3D v)
{
var distance = ComputeDistance(v);
return (distance > Epsilon ? "left" : (distance < -Epsilon ? "right" : "on"));
}
As shown in the code, I use a fixed Epsilon value.
But I don't trust this fixed epsilon because I don't know the size of the floating point error. If the fp error is bigger than my epsilon interval, then my algorithm will fail.
How can I make it robust? I have searched on the internet but haven't found a solution so far. I have read "What Every Computer Scientist Should Know About Floating-Point Arithmetic", it describes why fp errors occur but not how to solve them practically.
Edit
Here is a shewchuk predicate that doesn't seem work:
double[] pa = {0, 0};
double[] pb = {2 * Double.Epsilon, 0};
double[] pc = { 0, Double.Epsilon };
Assert.IsTrue(GeometricPredicates.Orient2D(pa, pb, pc) > 0);
The assertion fails because Orient2D return 0. The code is here
Edit2
Shouldn't it be possible to calculate an error bound by using the machine epsilon? According to wikipedia, the machine epsilon is an upper bound due to rounding. For double, it is 2^−53. So as I take it, when I have an arithmetic calculation:
double x = y + z
then the maximum error should be 2^−53. Shouldn't this fact enable the possiblity to calculate an appropriate epsilon? So two rewrite my method:
double const Machine_Eps = 1.11022302462516E-16 // (2^-53)
double Epsilon = Machine_Eps;
internal double ComputeDistance(Vector3D v)
{
Vector3D normal = Plane.Normal;
Vertex v0 = Plane.Origin;
double a = normal.X;
double b = normal.Y;
double c = normal.Z;
// 3 multiplications + 2 additions = maximum of 5*Machine_Eps
double d = -(a * v0.X + b * v0.Y + c * v0.Z);
// 3 multiplications + 3 additions = maximum of 6*Machine_Eps
Epsilon = 11 * Machine_Eps;
return a * v.X + b * v.Y + c * v.Z + d;
}
internal string DistanceSign(Vector3D v)
{
var distance = ComputeDistance(v);
return (distance > Epsilon ? "left" : (distance < -Epsilon ? "right" : "on"));
}
Ok now you can tell me how wrong I am. :)

Inverse FFT in C#

I am writing an application for procedural audiofiles, I have to analyze my new file, get its frequency spectrum and change it in its calculated.
I want to do this with the Fast Fourier Transform (FFT). This is my recursive C# FFT:
void ft(float n, ref Complex[] f)
{
if (n > 1)
{
Complex[] g = new Complex[(int) n / 2];
Complex[] u = new Complex[(int) n / 2];
for (int i = 0; i < n / 2; i++)
{
g[i] = f[i * 2];
u[i] = f[i * 2 + 1];
}
ft(n / 2, ref g);
ft(n / 2, ref u);
for (int i = 0; i < n / 2; i++)
{
float a = i;
a = -2.0f * Mathf.PI * a / n;
float cos = Mathf.Cos(a);
float sin = Mathf.Sin(a);
Complex c1 = new Complex(cos, sin);
c1 = Complex.Multiply(u[i], c1);
f[i] = Complex.Add(g[i], c1);
f[i + (int) n / 2] = Complex.Subtract(g[i], c1);
}
}
}
The inspiring example was
I then compared my results with those from wolframalpha for the same input 0.6,0.7,0.8,0.9 but the results aren't be the same. My results are twice as big than Wolfram's and the imaginary part are the -2 times of Wolfram's.
Also, wiki indicates that the inverse of FFT can be computed with
But I compare inputs and outputs and they are different.
Has anyone an idea what's wrong?

Different implementations often use different definitions of the Discrete Fourier Transform (DFT), with correspondingly different results. The correspondence between implementations is usually fairly trivial (such as a scaling factor).
More specifically, your implementation is based on the following definition of the DFT:
On the other hand, Wolfram alpha by default uses a definition, which after adjusting to 0-based indexing looks like:
Correspondingly, it is possible to transform the result of your implementation to match Wolfram alpha's with:
void toWolframAlphaDefinition(ref Complex[] f)
{
float scaling = (float)(1.0/Math.Sqrt(f.Length));
for (int i = 0; i < f.Length; i++)
{
f[i] = scaling * Complex.Conjugate(f[i]);
}
}
Now as far as computing the inverse DFT using the forward transform, a direct implementation of the formula
you provided would be:
void inverseFt(ref Complex[] f)
{
for (int i = 0; i < f.Length; i++)
{
f[i] = Complex.Conjugate(f[i]);
}
ft(f.Length, ref f);
float scaling = (float)(1.0 / f.Length);
for (int i = 0; i < f.Length; i++)
{
f[i] = scaling * Complex.Conjugate(f[i]);
}
}
Calling ft on the original sequence 0.6, 0.7, 0.8, 0.9 should thus get you the transformed sequence 3, -0.2+0.2j, -0.2, -0.2-0.2j.
Further calling inverseFt on this transform sequence should then bring you back to your original sequence 0.6, 0.7, 0.8, 0.9 (within some reasonable floating point error), as shown in this live demo.

Compute the Cartesian Distance between points in high-dimensional space optimally

Summary:
My class SquareDistance computes the square of the Cartesian distance in four ways using methods with these names:
Signed
UnsignedBranching
UnsignedDistribute
CastToSignedLong
The first one is fastest and uses signed integers, but my data must be unsigned (for reasons given below). The other three methods start with unsigned numbers. My goal is to write a method like those in SquareDistance that takes unsigned data and performs better than the three I already wrote, as close as possible in performance to #1. Code with benchmark results follows. (unsafe code is permitted, if you think it will help.)
Details:
I am developing an algorithm to solve K-nearest neighbor problems using an index derived from the Hilbert curve. The execution time for the naive, linear scan algorithm grows in time quadratically with the number of points and linearly with the number of dimensions, and it spends all its time computing and comparing Cartesian distances.
The motivation behind the special Hilbert index is to reduce the number of times that the distance function is called. However, it must still be called millions of times, so I must make it as fast as possible. (It is the most frequently called function in the program. A recent failed attempt to optimize the distance function doubled the program execution time from seven minutes to fifteen minutes, so no, this is not a premature or superfluous optimization.)
Dimensions: The points may have anywhere from ten to five thousand dimensions.
Constraints. I have two annoying constraints:
The Hilbert transformation logic requires that the points be expressed as uint (unsigned integer) arrays. (The code was written by another, is magic and uses shifts, ANDs, ORs, and the like and can't be changed.) Storing my points as signed integers and incessantly casting them to uint arrays produced wretched performance, so I must at the very least store a uint array copy of each point.
To improve efficiency, I made a signed integer copy of each point to speed up the distance calculations. This worked very well, but once I get to about 3,000 dimensions, I run out of memory!
To save on memory, I removed the memoized signed integer arrays and tried to write an unsigned version of the distance calculation. My best results are 2.25 times worse than the signed integer version.
The benchmarks create 1000 random points of 1000 dimensions each and perform distance calculations between every point and every other point, for 1,000,000 comparisons. Since I only care about the relative distance, I save time by not performing the square root.
In debug mode:
SignedBenchmark Ratio: 1.000 Seconds: 3.739
UnsignedBranchingBenchmark Ratio: 2.731 Seconds: 10.212
UnsignedDistributeBenchmark Ratio: 3.294 Seconds: 12.320
CastToSignedLongBenchmark Ratio: 3.265 Seconds: 12.211
In release mode:
SignedBenchmark Ratio: 1.000 Seconds: 3.494
UnsignedBranchingBenchmark Ratio: 2.672 Seconds: 9.334
UnsignedDistributeBenchmark Ratio: 3.336 Seconds: 11.657
CastToSignedLongBenchmark Ratio: 3.471 Seconds: 12.127
The above benchmarks were run on a Dell with an Intel Core i7-4800MQ CPU # 2.70GHz with 16 GB memory. My larger algorithm already uses the Task Parallel library for larger tasks, so it is fruitless to parallelize this inner loop.
Question: Can anyone think of a faster algorithm than UnsignedBranching?
Below is my benchmark code.
UPDATE
This uses loop unrolling (thanks to #dasblinkenlight) and is 2.7 times faster:
public static long UnsignedLoopUnrolledBranching(uint[] x, uint[] y)
{
var distance = 0UL;
var leftovers = x.Length % 4;
var dimensions = x.Length;
var roundDimensions = dimensions - leftovers;
for (var i = 0; i < roundDimensions; i += 4)
{
var x1 = x[i];
var y1 = y[i];
var x2 = x[i+1];
var y2 = y[i+1];
var x3 = x[i+2];
var y3 = y[i+2];
var x4 = x[i+3];
var y4 = y[i+3];
var delta1 = x1 > y1 ? x1 - y1 : y1 - x1;
var delta2 = x2 > y2 ? x2 - y2 : y2 - x2;
var delta3 = x3 > y3 ? x3 - y3 : y3 - x3;
var delta4 = x4 > y4 ? x4 - y4 : y4 - x4;
distance += delta1 * delta1 + delta2 * delta2 + delta3 * delta3 + delta4 * delta4;
}
for (var i = roundDimensions; i < dimensions; i++)
{
var xi = x[i];
var yi = y[i];
var delta = xi > yi ? xi - yi : yi - xi;
distance += delta * delta;
}
return (long)distance;
}
SquareDistance.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace DistanceBenchmark
{
/// <summary>
/// Provide several alternate methods for computing the square of the Cartesian distance
/// to allow study of their relative performance.
/// </summary>
public static class SquareDistance
{
/// <summary>
/// Compute the square of the Cartesian distance between two N-dimensional points
/// with calculations done on signed numbers using signed arithmetic,
/// a single multiplication and no branching.
/// </summary>
/// <param name="x">First point.</param>
/// <param name="y">Second point.</param>
/// <returns>Square of the distance.</returns>
public static long Signed(int[] x, int[] y)
{
var distance = 0L;
var dimensions = x.Length;
for (var i = 0; i < dimensions; i++)
{
var delta = x[i] - y[i];
distance += delta * delta;
}
return distance;
}
/// <summary>
/// Compute the square of the Cartesian distance between two N-dimensional points
/// with calculations done on unsigned numbers using unsigned arithmetic, a single multiplication
/// and a branching instruction (the ternary operator).
/// </summary>
/// <param name="x">First point.</param>
/// <param name="y">Second point.</param>
/// <returns>Square of the distance.</returns>
public static long UnsignedBranching(uint[] x, uint[] y)
{
var distance = 0UL;
var dimensions = x.Length;
for (var i = 0; i < dimensions; i++)
{
var xi = x[i];
var yi = y[i];
var delta = xi > yi ? xi - yi : yi - xi;
distance += delta * delta;
}
return (long)distance;
}
/// <summary>
/// Compute the square of the Cartesian distance between two N-dimensional points
/// with calculations done on unsigned numbers using unsigned arithmetic and the distributive law,
/// which requires four multiplications and no branching.
///
/// To prevent overflow, the coordinates are cast to ulongs.
/// </summary>
/// <param name="x">First point.</param>
/// <param name="y">Second point.</param>
/// <returns>Square of the distance.</returns>
public static long UnsignedDistribute(uint[] x, uint[] y)
{
var distance = 0UL;
var dimensions = x.Length;
for (var i = 0; i < dimensions; i++)
{
ulong xi = x[i];
ulong yi = y[i];
distance += xi * xi + yi * yi - 2 * xi * yi;
}
return (long)distance;
}
/// <summary>
/// Compute the square of the Cartesian distance between two N-dimensional points
/// with calculations done on unsigned numbers using signed arithmetic,
/// by first casting the values into longs.
/// </summary>
/// <param name="x">First point.</param>
/// <param name="y">Second point.</param>
/// <returns>Square of the distance.</returns>
public static long CastToSignedLong(uint[] x, uint[] y)
{
var distance = 0L;
var dimensions = x.Length;
for (var i = 0; i < dimensions; i++)
{
var delta = (long)x[i] - (long)y[i];
distance += delta * delta;
}
return distance;
}
}
}
RandomPointFactory.cs:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace DistanceBenchmark
{
public static class RandomPointFactory
{
/// <summary>
/// Get a random list of signed integer points with the given number of dimensions to use as test data.
/// </summary>
/// <param name="recordCount">Number of points to get.</param>
/// <param name="dimensions">Number of dimensions per point.</param>
/// <returns>Signed integer test data.</returns>
public static IList<int[]> GetSignedTestPoints(int recordCount, int dimensions)
{
var testData = new List<int[]>();
var random = new Random(DateTime.Now.Millisecond);
for (var iRecord = 0; iRecord < recordCount; iRecord++)
{
int[] point;
testData.Add(point = new int[dimensions]);
for (var d = 0; d < dimensions; d++)
point[d] = random.Next(100000);
}
return testData;
}
/// <summary>
/// Get a random list of unsigned integer points with the given number of dimensions to use as test data.
/// </summary>
/// <param name="recordCount">Number of points to get.</param>
/// <param name="dimensions">Number of dimensions per point.</param>
/// <returns>Unsigned integer test data.</returns>
public static IList<uint[]> GetUnsignedTestPoints(int recordCount, int dimensions)
{
var testData = new List<uint[]>();
var random = new Random(DateTime.Now.Millisecond);
for (var iRecord = 0; iRecord < recordCount; iRecord++)
{
uint[] point;
testData.Add(point = new uint[dimensions]);
for (var d = 0; d < dimensions; d++)
point[d] = (uint)random.Next(100000);
}
return testData;
}
}
}
Program.cs:
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace DistanceBenchmark
{
public class Program
{
private static IList<int[]> SignedTestData = RandomPointFactory.GetSignedTestPoints(1000, 1000);
private static IList<uint[]> UnsignedTestData = RandomPointFactory.GetUnsignedTestPoints(1000, 1000);
static void Main(string[] args)
{
var baseline = TimeIt("SignedBenchmark", SignedBenchmark);
TimeIt("UnsignedBranchingBenchmark", UnsignedBranchingBenchmark, baseline);
TimeIt("UnsignedDistributeBenchmark", UnsignedDistributeBenchmark, baseline);
TimeIt("CastToSignedLongBenchmark", CastToSignedLongBenchmark, baseline);
TimeIt("SignedBenchmark", SignedBenchmark, baseline);
Console.WriteLine("Done. Type any key to exit.");
Console.ReadLine();
}
public static void SignedBenchmark()
{
foreach(var p1 in SignedTestData)
foreach (var p2 in SignedTestData)
SquareDistance.Signed(p1, p2);
}
public static void UnsignedBranchingBenchmark()
{
foreach (var p1 in UnsignedTestData)
foreach (var p2 in UnsignedTestData)
SquareDistance.UnsignedBranching(p1, p2);
}
public static void UnsignedDistributeBenchmark()
{
foreach (var p1 in UnsignedTestData)
foreach (var p2 in UnsignedTestData)
SquareDistance.UnsignedDistribute(p1, p2);
}
public static void CastToSignedLongBenchmark()
{
foreach (var p1 in UnsignedTestData)
foreach (var p2 in UnsignedTestData)
SquareDistance.CastToSignedLong(p1, p2);
}
public static double TimeIt(String testName, Action benchmark, double baseline = 0.0)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
benchmark();
stopwatch.Stop();
var seconds = stopwatch.Elapsed.TotalSeconds;
var ratio = baseline <= 0 ? 1.0 : seconds/baseline;
Console.WriteLine(String.Format("{0,-32} Ratio: {1:0.000} Seconds: {2:0.000}", testName, ratio, seconds));
return seconds;
}
}
}

You should be able to shave off a lot of execution time by unrolling your loops:
public static long Signed(int[] x, int[] y)
{
var distance = 0L;
var dimensions = x.Length;
var stop = dimensions - (dimensions % 4);
for (var i = 0; i < stop; i+=4)
{
var delta0 = x[i] - y[i];
var delta1 = x[i+1] - y[i+1];
var delta2 = x[i+2] - y[i+2];
var delta3 = x[i+3] - y[i+3];
distance += (delta0 * delta0)
+ (delta1 * delta1)
+ (delta2 * delta2)
+ (delta3 * delta3);
}
for (var i = stop; i < dimensions; i++)
{
var delta = x[i] - y[i];
distance += delta * delta;
}
return distance;
}
This change alone reduced the execution time from 8.325s to 4.745s on my local system - a 43% improvement!
The idea is to do four points at a time while you can, and then finish off the remaining points in a separate loop.

If you cannot change the hilbert curve you can try a z curve, i.e. a morton curve. Translate the dimensions to a binary and interleave it. Then sort it. You can verify the upper bounds with the mostsignificat bits. Hilbert curve in n-dimension uses a gray code maybe you can search the internet for a faster version. You can find some fast implementation in the hackers cookbook. A morton curve should be similar to a h-tree. When you need the precision you can try copies of the hilbert curves, i.e. a moore curve. For example in 2d you can interleave 4 hilbert curves:
"
,

Best possible improvement I can see isn't going to be a low hanging fruit. This kind of problem is not well suited for the current version of the .net framework (or a CPU in general).
The class of problem you have is called SIMD. You may have heard of Intel Pentium MMX. The MMX instruction set is the marketing term for SIMD instruction set.
There are three pretty good ways to get SIMD to run with your program. In order of slowest to fastest, and in order of easiest to hardest.
RyuJIT (the preview version of the next .net Compiler) to take advantage of CPU SIMD
P/Invoke into C++ AMP onto your GPU
Interop onto a FPGA with a core designed for this calculation.
I would highly recommend trying to take advantage of your GPU with C++ AMP, especially since uint[] should be easily passed to C++ AMP.

In the shower this morning, I came up with a way to improve this further using the dot product, shaving another fifty percent off the time when the data is stored as uint[] arrays. I had investigated this idea before, but failed to recognize a loop invariant that I could optimize by precomputing. The basis of the idea is to distribute the operations:
(x-y)(x-y) = x*x + y*y - 2xy
If I sum this over all coordinates, the result is:
2 2 2
D = |x| + |y| - 2(x·y)
Since I will be performing lots of distance computations, I can store the square length of each vector. Then finding the distance between two vectors amounts to summing their square distances (outside the loop) and computing the dot-product, which has no negative values, so needs no branching!
Why is the branching a problem? It is because with uint vectors you can't subtract values in the Cartesian formula without using a branch operation to test which value is larger. Thus if I want (x-y)*(x-y) I needed to do this in my loop:
var delta = x[i] > y[i] ? x[i] - y[i] : y[i] - x[i];
distance += delta * delta;
In addition, to prevent overflow from uint to ulong, I needed to cast numbers to ulong a lot, which really killed performance. Since most of my coordinates are smallish, I was able to create a test. I also store the max value for each vector. Since I am unrolling my loops by four iterations at a time, if 4*xMax*yMax does not overflow uint, I can dispense with most of my casting operations. If the test fails, I do the more costly version that casts more.
In the end, I had several implementations: naive with casting, with branching, distributed with casting and loop invariants not removed, and dot product with less casting and invariants removed.
The naive approach has a subtraction, a multiplication and an addition in each loop iteration. The dot product distribution with loop invariants removed uses only a multiplication and an addition.
Here are the benchmarks:
For 100000 iterations and 2000 dimensions.
Naive time = 2.505 sec.
Branch time = 0.628 sec.
Distributed time = 6.371 sec.
Dot Product time = 0.288 sec.
Improve vs Naive = 88.5%.
Improve vs Branch = 54.14%.
Here is the code as an NUnit test:
using System;
using System.Diagnostics;
using NUnit.Framework;
using System.Linq;
namespace HilbertTransformationTests
{
[TestFixture]
public class CartesianDistanceTests
{
[Test]
public void SquareDistanceBenchmark()
{
var dims = 2000;
var x = new uint[dims];
var y = new uint[dims];
var xMag2 = 0L;
var yMag2 = 0L;
for (var i = 0; i < dims; i++)
{
x[i] = (uint)i;
xMag2 += x[i] * (long)x[i];
y[i] = (uint)(10000 - i);
yMag2 += y[i] * (long)y[i];
}
var xMax = (long)x.Max();
var yMax = (long)y.Max();
var repetitions = 100000;
var naiveTime = Time(() => SquareDistanceNaive(x, y), repetitions);
var distributeTime = Time(() => SquareDistanceDistributed(x, y), repetitions);
var branchTime = Time(() => SquareDistanceBranching(x, y), repetitions);
var dotProductTime = Time(() => SquareDistanceDotProduct(x, y, xMag2, yMag2, xMax, yMax), repetitions);
Console.Write($#"
For {repetitions} iterations and {dims} dimensions.
Naive time = {naiveTime} sec.
Branch time = {branchTime} sec.
Distributed time = {distributeTime} sec.
Dot Product time = {dotProductTime} sec.
Improve vs Naive = {((int)(10000 * (naiveTime - dotProductTime) / naiveTime)) / 100.0}%.
Improve vs Branch = {((int)(10000 * (branchTime - dotProductTime) / branchTime)) / 100.0}%.
");
Assert.Less(dotProductTime, branchTime, "Dot product time should have been less than branch time");
}
private static double Time(Action action, int repeatCount)
{
var timer = new Stopwatch();
timer.Start();
for (var j = 0; j < repeatCount; j++)
action();
timer.Stop();
return timer.ElapsedMilliseconds / 1000.0;
}
private static long SquareDistanceNaive(uint[] x, uint[] y)
{
var squareDistance = 0L;
for (var i = 0; i < x.Length; i++)
{
var delta = (long)x[i] - (long)y[i];
squareDistance += delta * delta;
}
return squareDistance;
}
/// <summary>
/// Compute the square distance, using ternary operators for branching to keep subtraction operations from going negative,
/// which is inappropriate for unsigned numbers.
/// </summary>
/// <returns>The distance branching.</returns>
/// <param name="x">The x coordinate.</param>
/// <param name="y">The y coordinate.</param>
private static long SquareDistanceBranching(uint[] x, uint[] y)
{
long squareDistanceLoopUnrolled;
// Unroll the loop partially to improve speed. (2.7x improvement!)
var distance = 0UL;
var leftovers = x.Length % 4;
var dimensions = x.Length;
var roundDimensions = dimensions - leftovers;
for (var i = 0; i < roundDimensions; i += 4)
{
var x1 = x[i];
var y1 = y[i];
var x2 = x[i + 1];
var y2 = y[i + 1];
var x3 = x[i + 2];
var y3 = y[i + 2];
var x4 = x[i + 3];
var y4 = y[i + 3];
var delta1 = x1 > y1 ? x1 - y1 : y1 - x1;
var delta2 = x2 > y2 ? x2 - y2 : y2 - x2;
var delta3 = x3 > y3 ? x3 - y3 : y3 - x3;
var delta4 = x4 > y4 ? x4 - y4 : y4 - x4;
distance += delta1 * delta1 + delta2 * delta2 + delta3 * delta3 + delta4 * delta4;
}
for (var i = roundDimensions; i < dimensions; i++)
{
var xi = x[i];
var yi = y[i];
var delta = xi > yi ? xi - yi : yi - xi;
distance += delta * delta;
}
squareDistanceLoopUnrolled = (long)distance;
return squareDistanceLoopUnrolled;
}
private static long SquareDistanceDistributed(uint[] x, uint[] y)
{
long squareDistanceLoopUnrolled;
// Unroll the loop partially to improve speed. (2.7x improvement!)
var distance = 0UL;
var dSubtract = 0UL;
var leftovers = x.Length % 4;
var dimensions = x.Length;
var roundDimensions = dimensions - leftovers;
for (var i = 0; i < roundDimensions; i += 4)
{
ulong x1 = x[i];
ulong y1 = y[i];
ulong x2 = x[i + 1];
ulong y2 = y[i + 1];
ulong x3 = x[i + 2];
ulong y3 = y[i + 2];
ulong x4 = x[i + 3];
ulong y4 = y[i + 3];
distance += x1 * x1 + y1 * y1
+ x2 * x2 + y2 * y2
+ x3 * x3 + y3 * y3
+ x4 * x4 + y4 * y4;
dSubtract += x1 * y1 + x2 * y2 + x3 * y3 + x4 * y4;
}
distance = distance - 2UL * dSubtract;
for (var i = roundDimensions; i < dimensions; i++)
{
var xi = x[i];
var yi = y[i];
var delta = xi > yi ? xi - yi : yi - xi;
distance += delta * delta;
}
squareDistanceLoopUnrolled = (long)distance;
return squareDistanceLoopUnrolled;
}
private static long SquareDistanceDotProduct(uint[] x, uint[] y, long xMag2, long yMag2, long xMax, long yMax)
{
const int unroll = 4;
if (xMax * yMax * unroll < (long) uint.MaxValue)
return SquareDistanceDotProductNoOverflow(x, y, xMag2, yMag2);
// Unroll the loop partially to improve speed. (2.7x improvement!)
var dotProduct = 0UL;
var leftovers = x.Length % unroll;
var dimensions = x.Length;
var roundDimensions = dimensions - leftovers;
for (var i = 0; i < roundDimensions; i += unroll)
{
var x1 = x[i];
ulong y1 = y[i];
var x2 = x[i + 1];
ulong y2 = y[i + 1];
var x3 = x[i + 2];
ulong y3 = y[i + 2];
var x4 = x[i + 3];
ulong y4 = y[i + 3];
dotProduct += x1 * y1 + x2 * y2 + x3 * y3 + x4 * y4;
}
for (var i = roundDimensions; i < dimensions; i++)
dotProduct += x[i] * (ulong)y[i];
return xMag2 + yMag2 - 2L * (long)dotProduct;
}
/// <summary>
/// Compute the square of the Cartesian distance using the dotproduct method,
/// assuming that calculations wont overflow uint.
///
/// This permits us to skip some widening conversions to ulong, making the computation faster.
///
/// Algorithm:
///
/// 2 2 2
/// D = |x| + |y| - 2(x·y)
///
/// Using the dot product of x and y and precomputed values for the square magnitudes of x and y
/// permits us to use two operations (multiply and add) instead of three (subtract, multiply and add)
/// in the main loop, saving one third of the time.
/// </summary>
/// <returns>The square distance.</returns>
/// <param name="x">First point.</param>
/// <param name="y">Second point.</param>
/// <param name="xMag2">Distance from x to the origin, squared.</param>
/// <param name="yMag2">Distance from y to the origin, squared.</param>
private static long SquareDistanceDotProductNoOverflow(uint[] x, uint[] y, long xMag2, long yMag2)
{
// Unroll the loop partially to improve speed. (2.7x improvement!)
const int unroll = 4;
var dotProduct = 0UL;
var leftovers = x.Length % unroll;
var dimensions = x.Length;
var roundDimensions = dimensions - leftovers;
for (var i = 0; i < roundDimensions; i += unroll)
dotProduct += (x[i] * y[i] + x[i+1] * y[i+1] + x[i+2] * y[i+2] + x[i+3] * y[i+3]);
for (var i = roundDimensions; i < dimensions; i++)
dotProduct += x[i] * y[i];
return xMag2 + yMag2 - 2L * (long)dotProduct;
}
}
}

Generate a random number in a Gaussian Range?

I want to use a random number generator that creates random numbers in a gaussian range where I can define the median by myself. I already asked a similar question here and now I'm using this code:
class RandomGaussian
{
private static Random random = new Random();
private static bool haveNextNextGaussian;
private static double nextNextGaussian;
public static double gaussianInRange(double from, double mean, double to)
{
if (!(from < mean && mean < to))
throw new ArgumentOutOfRangeException();
int p = Convert.ToInt32(random.NextDouble() * 100);
double retval;
if (p < (mean * Math.Abs(from - to)))
{
double interval1 = (NextGaussian() * (mean - from));
retval = from + (float)(interval1);
}
else
{
double interval2 = (NextGaussian() * (to - mean));
retval = mean + (float)(interval2);
}
while (retval < from || retval > to)
{
if (retval < from)
retval = (from - retval) + from;
if (retval > to)
retval = to - (retval - to);
}
return retval;
}
private static double NextGaussian()
{
if (haveNextNextGaussian)
{
haveNextNextGaussian = false;
return nextNextGaussian;
}
else
{
double v1, v2, s;
do
{
v1 = 2 * random.NextDouble() - 1;
v2 = 2 * random.NextDouble() - 1;
s = v1 * v1 + v2 * v2;
} while (s >= 1 || s == 0);
double multiplier = Math.Sqrt(-2 * Math.Log(s) / s);
nextNextGaussian = v2 * multiplier;
haveNextNextGaussian = true;
return v1 * multiplier;
}
}
}
Then to verify the results I plotted them with gaussianInRange(0, 0.5, 1) for n=100000000
As one can see the median is really at 0.5 but there isn't really a curve visible. So what I'm doing wrong?
EDIT
What i want is something like this where I can set the highest probability by myself by passing a value.

The simplest way to draw normal deviates conditional on them being in a particular range is with rejection sampling:
do {
retval = NextGaussian() * stdev + mean;
} while (retval < from || to < retval);
The same sort of thing is used when you draw coordinates (v1, v2) in a circle in your unconditional normal generator.
Simply folding in values outside the range doesn't produce the same distribution.
Also, if you have a good implementation of the error function and its inverse, you can calculate the values directly using an inverse CDF. The CDF of a normal distribution is
F(retval) = (1 + erf((retval-mean) / (stdev*sqrt(2)))) / 2
The CDF of a censored distribution is
C(retval) = (F(retval) - F(from)) / (F(to) - F(from)), from ≤ x < to
To draw a random number using a CDF, you draw v from a uniform distribution on [0, 1] and solve C(retval) = v. This gives
double v = random.NextDouble();
double t1 = erf((from - mean) / (stdev*sqrt(2)));
t2 = erf((to - mean) / (stdev*sqrt(2)));
double retval = mean + stdev * sqrt(2) * erf_inv(t1*(1-v) + t2*v);
You can precalculate t1 and t2 for specific parameters. The advantage of this approach is that there is no rejection sampling, so you only need a single NextDouble() per draw. If the [from, to] interval is small this will be faster.
However, it sounds like you might want the binomial distribution instead.

I have similar methods in my Graph generator (had to modify it a bit):
Returns a random floating-point number using a generator function with a specific range:
private double NextFunctional(Func<double, double> func, double from, double to, double height, out double x)
{
double halfWidth = (to - from) / 2;
double distance = halfWidth + from;
x = this.rand.NextDouble() * 2 - 1;// -1 .. 1
double y = func(x);
x = halfWidth * x + distance;
y *= height;
return y;
}
Gaussian function:
private double Gauss(double x)
{
// Graph should look better with double-x scale.
x *= 2;
double σ = 1 / Math.Sqrt(2 * Math.PI);
double variance = Math.Pow(σ, 2);
double exp = -0.5 * Math.Pow(x, 2) / variance;
double y = 1 / Math.Sqrt(2 * Math.PI * variance) * Math.Pow(Math.E, exp);
return y;
}
A method that generates a graph using the random numbers:
private void PlotGraph(Graphics g, Pen p, double from, double to, double height)
{
for (int i = 0; i < 1000; i++)
{
double x;
double y = this.NextFunctional(this.Gauss, from, to, height, out x);
this.DrawPoint(g, p, x, y);
}
}
I would rather used a cosine function - it is much faster and pretty close to the gaussian function for your needs:
double x;
double y = this.NextFunctional(a => Math.Cos(a * Math.PI), from, to, height, out x);
The out double x parameter in the NextFunctional() method is there so you can easily test it on your graphs (I use an iterator in my method).

get closest point to a line

I'd like to have a straight forward C# function to get a closest point (from a point P) to a line-segment, AB. An abstract function may look like this. I've search through SO but not found a usable (by me) solution.
public Point getClosestPointFromLine(Point A, Point B, Point P);

Here's Ruby disguised as Pseudo-Code, assuming Point objects each have a x and y field.
def GetClosestPoint(A, B, P)
a_to_p = [P.x - A.x, P.y - A.y] # Storing vector A->P
a_to_b = [B.x - A.x, B.y - A.y] # Storing vector A->B
atb2 = a_to_b[0]**2 + a_to_b[1]**2 # **2 means "squared"
# Basically finding the squared magnitude
# of a_to_b
atp_dot_atb = a_to_p[0]*a_to_b[0] + a_to_p[1]*a_to_b[1]
# The dot product of a_to_p and a_to_b
t = atp_dot_atb / atb2 # The normalized "distance" from a to
# your closest point
return Point.new( :x => A.x + a_to_b[0]*t,
:y => A.y + a_to_b[1]*t )
# Add the distance to A, moving
# towards B
end
Alternatively:
From Line-Line Intersection, at Wikipedia. First, find Q, which is a second point that is to be had from taking a step from P in the "right direction". This gives us four points.
def getClosestPointFromLine(A, B, P)
a_to_b = [B.x - A.x, B.y - A.y] # Finding the vector from A to B
This step can be combined with the next
perpendicular = [ -a_to_b[1], a_to_b[0] ]
# The vector perpendicular to a_to_b;
This step can also be combined with the next
Q = Point.new(:x => P.x + perpendicular[0], :y => P.y + perpendicular[1])
# Finding Q, the point "in the right direction"
# If you want a mess, you can also combine this
# with the next step.
return Point.new (:x => ((A.x*B.y - A.y*B.x)*(P.x - Q.x) - (A.x-B.x)*(P.x*Q.y - P.y*Q.x)) / ((A.x - B.x)*(P.y-Q.y) - (A.y - B.y)*(P.y-Q.y)),
:y => ((A.x*B.y - A.y*B.x)*(P.y - Q.y) - (A.y-B.y)*(P.x*Q.y - P.y*Q.x)) / ((A.x - B.x)*(P.y-Q.y) - (A.y - B.y)*(P.y-Q.y)) )
end
Caching, Skipping steps, etc. is possible, for performance reasons.

if anyone is interested in a C# XNA function based on the above:
public static Vector2 GetClosestPointOnLineSegment(Vector2 A, Vector2 B, Vector2 P)
{
Vector2 AP = P - A; //Vector from A to P
Vector2 AB = B - A; //Vector from A to B
float magnitudeAB = AB.LengthSquared(); //Magnitude of AB vector (it's length squared)
float ABAPproduct = Vector2.Dot(AP, AB); //The DOT product of a_to_p and a_to_b
float distance = ABAPproduct / magnitudeAB; //The normalized "distance" from a to your closest point
if (distance < 0) //Check if P projection is over vectorAB
{
return A;
}
else if (distance > 1) {
return B;
}
else
{
return A + AB * distance;
}
}

Your point (X) will be a linear combination of points A and B:
X = k A + (1-k) B
For X to be actually on the line segment, the parameter k must be between 0 and 1, inclusive. You can compute k as follows:
k_raw = (P-B).(A-B) / (A-B).(A-B)
(where the period denotes the dot product)
Then, to make sure the point is actually on the line segment:
if k_raw < 0:
k= 0
elif k_raw > 1:
k= 1
else:
k= k_raw

The answer from Justin L. is almost fine, but it doesn't check if the normalized distance is less than 0, or higher than the AB vector magnitude. Then it won't work well when P vector proyection is out of bounds (from the line segment AB).
Here's the corrected pseudocode:
function GetClosestPoint(A, B, P)
{
vectorAP = (p.x - a.x, p.y - a.y) //Vector from A to P
vectorAB = (b.x - a.x, b.y - a.y) //Vector from A to B
magnitudeAB = vectorAB[0]^2 + vectorAB[1]^2
//Magnitude of AB vector (it's length)
ABAPproduct = vectorAB[0]*vectorAP[0] + vectorAB[1]*vectorAP[1]
//The product of a_to_p and a_to_b
distance = ABAPproduct / magnitudeAB
//The normalized "distance" from a to your closest point
if ( distance < 0) //Check if P projection is over vectorAB
{
returnPoint.x = a.x
returnPoint.y = a.y
}
else if (distance > magnitudeAB)
{
returnPoint.x = b.x
returnPoint.y = b.y
}
else
{
returnPoint.x = a.x + vectorAB[0]*distance
returnPoint.y = a.y + vectorAB[1]*distance
}
}

I wrote this a long time ago, it's not much different to what others have said, but it's a copy/paste solution in C# if you have a class (or struct) named PointF with members X and Y:
private static PointF ClosestPointToSegment(PointF P, PointF A, PointF B)
{
PointF a_to_p = new PointF(), a_to_b = new PointF();
a_to_p.X = P.X - A.X;
a_to_p.Y = P.Y - A.Y; // # Storing vector A->P
a_to_b.X = B.X - A.X;
a_to_b.Y = B.Y - A.Y; // # Storing vector A->B
float atb2 = a_to_b.X * a_to_b.X + a_to_b.Y * a_to_b.Y;
float atp_dot_atb = a_to_p.X * a_to_b.X + a_to_p.Y * a_to_b.Y; // The dot product of a_to_p and a_to_b
float t = atp_dot_atb / atb2; // # The normalized "distance" from a to the closest point
return new PointF(A.X + a_to_b.X * t, A.Y + a_to_b.Y * t);
}
Update: Looking at the comments it looks like I adapted it to C# from the same source code mentioned in the accepted answer.

Find the slope a1 of AB by dividing the y-difference with the x-difference; then draw a perpendicular line (with slope a2 = -1/a1, you need to solve for the offset (b2) by putting P's coordinates into y = a2*x + b2); then you have two lines (i.e. two linear equations), and you need to solve the intersection. That will be your closest point.
Do the math right, and the function will be pretty trivial to write.
To elaborate a bit:
Original line:
y = a1 * x + b1
a1 = (By - Ay) / (Bx - Ax) <--
b1 = Ay - a1 * Ax <--
Perpendicular line:
y = a2 * x + b2
a2 = -1/a1 <--
b2 = Py - a2 * Px <--
Now you have P which lies on both lines:
y = a1 * x + b1
y = a2 * x + b2
--------------- subtract:
0 = (a1 - a2) * Px + (b1 - b2)
x = - (b1 - b2) / (a1 - a2) <--
y = a1 * x + b1 <--
Hope I didn't mess up somewhere :) UPDATE Of course I did. Serve me right for not working things out on paper first. I deserved every downvote, but I'd've expected someone to correct me. Fixed (I hope).
Arrows point the way.
UPDATE Ah, the corner cases. Yeah, some languages don't handle infinities well. I did say the solution was language-free...
You can check the special cases, they're quite easy. The first one is when the x difference is 0. That means the line is vertical, and the closest point is on a horizontal perpendicular. Thus, x = Ax, y = Px.
The second one is when y difference is 0, and the opposite is true. Thus, x = Px, y = Ay

This answer is based on ideas from projective geometry.
Compute the cross product (Ax,Ay,1)×(Bx,By,1)=(u,v,w). The resulting vector describes the line connecting A and B: it has the equation ux+vy+w=0. But you can also interpret (u,v,0) as a point infinitely far away in a direction perpendicular to that line. Doing another cross product you get the line joining hat point to P: (u,v,0)×(Px,Py,1). And to intersect that line with the line AB, you do another cross product: ((u,v,0)×(Px,Py,1))×(u,v,w). The result will be a homogenous coordinate vector (x,y,z) from which you can read the coordinates of this closest point as (x/z,y/z).
Take everything together and you get the following formula:
Using a computer algebra system, you can find the resulting coordinates to be the following:
x = ((Ax - Bx)*Px + (Ay - By)*Py)*(Ax - Bx) + (Ay*Bx - Ax*By)*(Ay - By)
y = -(Ay*Bx - Ax*By)*(Ax - Bx) + ((Ax - Bx)*Px + (Ay - By)*Py)*(Ay - By)
z = (Ax - Bx)^2 + (Ay - By)^2
As you notice, there are a lot of recurring terms. Inventing (pretty much arbitrary) names for these, you can get the following final result, written in pseudocode:
dx = A.x - B.x
dy = A.y - B.y
det = A.y*B.x - A.x*B.y
dot = dx*P.x + dy*P.y
x = dot*dx + det*dy
y = dot*dy - det*dx
z = dx*dx + dy*dy
zinv = 1/z
return new Point(x*zinv, y*zinv)
Benefits of this approach:
No case distinctions
No square roots
Only a single division

Here are extension methods that should do the trick:
public static double DistanceTo(this Point from, Point to)
{
return Math.Sqrt(Math.Pow(from.X - to.X, 2) + Math.Pow(from.Y - to.Y, 2));
}
public static double DistanceTo(this Point point, Point lineStart, Point lineEnd)
{
double tI = ((lineEnd.X - lineStart.X) * (point.X - lineStart.X) + (lineEnd.Y - lineStart.Y) * (point.Y - lineStart.Y)) / Math.Pow(lineStart.DistanceTo(lineEnd), 2);
double dP = ((lineEnd.X - lineStart.X) * (point.Y - lineStart.Y) - (lineEnd.Y - lineStart.Y) * (point.X - lineStart.X)) / lineStart.DistanceTo(lineEnd);
if (tI >= 0d && tI <= 1d)
return Math.Abs(dP);
else
return Math.Min(point.DistanceTo(lineStart), point.DistanceTo(lineEnd));
}
Then just call:
P.DistanceTo(A, B);
To get distance of point "P" from line |AB|. It should be easy to modify this for PointF.
Finding the closest point then is just a matter of searching minimal distance. LINQ has methods for this.

The closest point C will be on a line whose slope is the reciprocal of AB and which intersects with P. This sounds like it might be homework, but I'll give some pretty strong hints, in order of increasing spoiler-alert level:
There can be only one such line.
This is a system of two line equations. Just solve for x and y.
Draw a line segment between A and B; call this L. The equation for L is y = mx + b, where m is the ratio of the y-coordinates to the x-coordinates. Solve for b using either A or B in the expression.
Do the same as above, but for CP. Now solve the simultaneous linear system of equations.
A Google search will give you a bevy of examples to choose from.

In case somebody is looking for a way to do this with Java + LibGdx:
Intersector.nearestSegmentPoint

This is the right algorythm to get nearest point of a segment from a point(Tested)(vb.net)
s2 = ClosestPointToSegment(point_x, Point_y, Segment_start_x, Segment_start_y, Segment_end_X, Segment_end_Y)
Public Shared Function DistanceTo(x1 As Double, y1 As Double, x2 As Double, y2 As Double) As Double
Return Math.Sqrt(Math.Pow(x1 - x2, 2) + Math.Pow(y1 - y2, 2))
End Function
Public Shared Function DistanceTo(point_x As Double, point_y As Double, lineStart_x As Double, lineStart_y As Double, lineEnd_x As Double, lineEnd_y As Double) As Double
Dim tI As Double = ((lineEnd_x - lineStart_x) * (point_x - lineStart_x) + (lineEnd_y - lineStart_y) * (point_y - lineStart_x)) / Math.Pow(DistanceTo(lineStart_x, lineStart_y, lineEnd_x, lineEnd_y), 2)
Dim dP As Double = ((lineEnd_x - lineStart_x) * (point_y - lineStart_y) - (lineEnd_y - lineStart_y) * (point_x - lineStart_x)) / DistanceTo(lineStart_x, lineStart_y, lineEnd_x, lineEnd_y)
If tI >= 0R AndAlso tI <= 1.0R Then
Return Math.Abs(dP)
Else
Return Math.Min(DistanceTo(point_x, point_y, lineStart_x, lineStart_y), DistanceTo(point_x, point_y, lineEnd_x, lineEnd_y))
End If
End Function
Private Shared Function ClosestPointToSegment(P_x As Double, p_y As Double, A_x As Double, a_y As Double, B_x As Double, b_y As Double) As Double()
Dim a_to_p As PointF = New PointF(), a_to_b As PointF = New PointF()
Dim rikthex As Double, rikthey As Double
Dim s1(1) As Double
Dim p1_v1_X As Double, p1_v1_y As Double, distanca1 As Double, distanca2 As Double
a_to_p.X = P_x - A_x
a_to_p.Y = p_y - a_y
a_to_b.X = B_x - A_x
a_to_b.Y = b_y - a_y
Dim atb2 As Single = a_to_b.X * a_to_b.X + a_to_b.Y * a_to_b.Y
Dim atp_dot_atb As Single = a_to_p.X * a_to_b.X + a_to_p.Y * a_to_b.Y
Dim t As Single = atp_dot_atb / atb2
rikthex = A_x + a_to_b.X * t
rikthey = a_y + a_to_b.Y * t
If A_x > B_x Then
If rikthex < A_x And rikthex > B_x Then 'pika duhet ne rregulll
If a_y > b_y Then
If rikthey < a_y And rikthey > b_y Then 'pika duhet ne rregulll
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
Else
If rikthey > a_y And rikthey < b_y Then 'pika duhet ne rregulll
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
End If
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
Else
If rikthex > A_x And rikthex < B_x Then 'pika duhet ne rregulll
If a_y > b_y Then
If rikthey < a_y And rikthey > b_y Then 'pika duhet ne rregulll
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
Else
If rikthey > a_y And rikthey < b_y Then 'pika duhet ne rregulll
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
End If
Else
distanca1 = DistanceTo(P_x, p_y, A_x, a_y)
distanca2 = DistanceTo(P_x, p_y, B_x, b_y)
If distanca1 < distanca2 Then
rikthex = A_x
rikthey = a_y
Else
rikthex = B_x
rikthey = b_y
End If
End If
End If
s1(0) = rikthex
s1(1) = rikthey
Return s1
End Function

In case anybody looking for python implementation, here is the code:
p1 and p2 is line, p3 is the point
def p4(p1, p2, p3):
x1, y1 = p1
x2, y2 = p2
x3, y3 = p3
dx, dy = x2-x1, y2-y1
det = dx*dx + dy*dy
a = (dy*(y3-y1)+dx*(x3-x1))/det
x= x1+a*dx, y1+a*dy
# print(x)
if x[0]<x1 or x[1]<y1:
return p1
elif x[0]>x2 or x[1]>y2:
return p2
else:
return x
This was taken from another thread and modified a little.
Python: point on a line closest to third point

The algorithm would be quite easy:
you have 3 points - triangle. From there you should be able to find AB, AC, BC.
Theck this out:
http://www.topcoder.com/tc?d1=tutorials&d2=geometry1&module=Static#line_point_distance