I need to add a very small value to a floating point value to make it insignificantly different so that it fails an equality test.
To avoid issues with precision, instead of adding a very small number, I have opted to add a relatively small number. Is this a good solution? or is there a reliable way to add an even smaller number?
matrix.m00 += matrix.m00 * 0.0000001f;
matrix.m11 += matrix.m11 * 0.0000001f;
matrix.m22 += matrix.m22 * 0.0000001f;
From reading I have found that the best solution is to use the next representable floating point number. Though in C# the process of doing this either a) requires unmanaged/unsafe code, or b) uses BitConverter which is too slow. So I figured that the above solution would work, but I would like to know if there are any gotchyas.
You can add an ulp to any double (depends on the double); that is the smallest number that you can add or subtract to it that will change its value.
Calculate the unit in the last place (ULP) for doubles
next higher/lower IEEE double precision number
Though, those posts all use BitConverter. I discovered a post that discusses how to add an ulp without unsafe code or BitConverter, though:
http://realtimemadness.blogspot.com/2012/06/nextafter-in-c-without-allocations-of.html
Sure there's a gotcha. If any of these values is 0, then you'll be adding exactly 0, i.e. not modifying the value at all.
Is there any reason why you couldn't use unsafe code to do this?
The minimum number you can add to a floating point number such that a different number is produced is a function of the original number, it's not some constant. Call this function Epsilon(x).
Epsilon(0), i.e. the minimum floating point number you can add to floating point 0 such that a distinguishable number is produced, can be found in the static value Double.Epsilon.
Even using a "large" epsilon like 1 will eventually fail, though. For example, this returns true in C#:
var big = 10000000000000000.0;
Console.WriteLine(big == (big + 1.0));
So unless you are sure that your input is in some fixed range of magnitude (e.g. all close to 0), you can't just fudge it with a single constant.
Related
I need to create random numbers between 100,000 and 100,000,000 using Erlang distribution. As it mentioned on the linked page, parameter x could be any number [0, +infinity); hence my range of values is theoretically acceptable.
However, when calculating the probability density function, the returned value is always 0 and that is because raising e to the power of -\lambda * x which with such big x values is always zero.
I need to be able to raise e to such big powers (indeed very small powers (e.g., -40,000)). My application is in .NET4.5 and I checked system.numerics namespace. However, so far I was not able to figure out how to perform this calculation.
For big floating point numbers you could use BigRational. This answer has a nice explanation about it.
The default floating point numbers in .NET don't have the precision that you are looking for, because as you pointed out, the value of such exponents is smaller than the precision of the number (hence the 0 values).
If you want to stick to C#, why not use a wrapper for an arbitrary precision library.
http://gnumpnet.codeplex.com/
Suppose I have equally spaced doubles (64 bit floating point numbers) x0,x1,...,xn. Equally spaced means that for all i, x(i+1) - xi is constant; call it w for width.
Given a number y in the range [x0,xn] I want to find the largest i such that xi <= y.
A naive approach would visit each i in turn (O(n)). Marginally better is to use a binary search (O(log n)).
A constant time lookup would be to calculate (y-x0)/w and cast it to an integer. However, this will occasionally give the wrong result due to floating point inaccuracy. E.g. Suppose there are 100 intervals of width 0.01 starting at 0.
(int)(0.29/0.01) = 28 //want 29 here
Can I retain the constant time lookup but ensure that the results are always identical to the binary search? Performing the calculation with decimals rather than doubles for 'w' and 'x0' seems to work here, but will it always work? I could always follow the direct lookup with a comparison with the xs either side, but this seems ugly and inefficient.
To clarify - I am given the xi and the value y as doubles - I cannot change this. But any intermediate calculation performed before returning the integer index can use any datatypes I like. Additionally, I can perform one-off "preparation" work in order to make the runtime calculation faster.
Edit: Apologies - turns out that I didn't check "equally spaced" properly - these numbers are often not "equally spaced" when their difference is calculated using floating point arithmetic.
Do the following
Calculate (int)(0.29/0.01) = 28 //want 29 here
Next, calculate back i * 0.01 for i between 28-1 and 28+1 and pick up the one that is correct.
What do you mean equally spaced? If can make some assumptions about the numbers, for example - that they increase on an interval, you can actually use median selction that is O(1) in the best case and O(log2(N)) in the worst case.
There was some code in my project that compared two double values to see if their difference exceeded 0, such as:
if (totValue != 1.0)
Resharper complained about this, suggesting I should use "EPSILON" and added just such a constant to the code (when invited to do so). However, it does not create the constant itself or suggest what value it should be. Is this a good solution:
const double EPSILON = double.Epsilon; // see http://msdn.microsoft.com/en-us/library/system.double.epsilon.aspx
. . .
if (Math.Abs(totValue - 1.0) > EPSILON)
compValue = Convert.ToString(totValue*Convert.ToDouble(compValue));
?
UPDATE
I changed it to this:
const double EPSILON = 0.001;
...thinking that's probably both large and small enough to work for typical double vals (not scientific stuff, just "I have 2.5 of these," etc.)
No, it is not a sensible value for your epsilon. The code you have is no different than a straight equality check.
double.Epsilon is the smallest possible difference that there can be between any two doubles. There is no way for any two doubles to be closer to each other than by being double.Epsilon apart, so the only way for this check to be true is for them to be exactly equal.
As for what epsilon value you should actually use, that simply depends, which is why one isn't automatically generated for you. It all depends on what types of operations you're doing to the data (which affects the possible deviation from the "true value") along with how much precision actually care about in your application (and of course if the precision that you care about is greater than your margin of error, you have a problem). Your epsilon needs to be some precision value greater than (or equal to) the precision you need, while being less than the possible margin of error of all operations performed on either numeric value.
Yes. But even better is to not use floating point. Use decimal instad.
However, if, for some reason, you have to stick to double never compare directly, that means, never rely on e.g. a-b == 0 with (a and b being some double values which are meant to be equal).
Floating point arithmetic is fast, but not precise, and taken that into account, R# is correct.
I know this has been discussed time and time again, but I can't seem to get even the most simple example of a one-step division of doubles to result in the expected, unrounded outcome in C# - so I'm wondering if perhaps there's i.e. some compiler flag or something else strange I'm not thinking of. Consider this example:
double v1 = 0.7;
double v2 = 0.025;
double result = v1 / v2;
When I break after the last line and examine it in the VS debugger, the value of "result" is 27.999999999999996. I'm aware that I can resolve it by changing to "decimal," but that's not possible in the case of the surrounding program. Is it not strange that two low-precision doubles like this can't divide to the correct value of 28? Is the only solution really to Math.Round the result?
Is it not strange that two low-precision doubles like this can't divide to the correct value of 28?
No, not really. Neither 0.7 nor 0.025 can be exactly represented in the double type. The exact values involved are:
0.6999999999999999555910790149937383830547332763671875
0.025000000000000001387778780781445675529539585113525390625
Now are you surprised that the division doesn't give exactly 28? Garbage in, garbage out...
As you say, the right result to represent decimal numbers exactly is to use decimal. If the rest of your program is using the wrong type, that just means you need to work out which is higher: the cost of getting the wrong answer, or the cost of changing the whole program.
Precision is always a problem, in case you are dealing with float or double.
Its a known issue in Computer Science and every programming language is affected by it. To minimize these sort of errors, which are mostly related to rounding, a complete field of Numerical Analysis is dedicated to it.
For instance, let take the following code.
What would you expect?
You will expect the answer to be 1, but this is not the case, you will get 0.9999907.
float v = .001f;
float sum = 0;
for (int i = 0; i < 1000; i++ )
{
sum += v;
}
It has nothing to do with how 'simple' or 'small' the double numbers are. Strictly speaking, neither 0.7 or 0.025 may be stored as exactly those numbers in computer memory, so performing calculations on them may provide interesting results if you're after heavy precision.
So yes, use decimal or round.
To explain this by analogy:
Imagine that you are working in base 3. In base 3, 0.1 is (in decimal) 1/3, or 0.333333333'.
So you can EXACTLY represent 1/3 (decimal) in base 3, but you get rounding errors when trying to express it in decimal.
Well, you can get exactly the same thing with some decimal numbers: They can be exactly expressed in decimal, but they CAN'T be exactly expressed in binary; hence, you get rounding errors with them.
Short answer to your first question: No, it's not strange. Floating-point numbers are discrete approximations of the real numbers, which means that rounding errors will propagate and scale when you do arithmetic operations.
Theres' a whole field of mathematics called numerical analyis that basically deal with how to minimize the errors when working with such approximations.
It's the usual floating point imprecision. Not every number can be represented as a double, and those minor representation inaccuracies add up. It's also a reason why you should not compare doubles to exact numbers. I just tested it, and result.ToString() showed 28 (maybe some kind of rounding happens in double.ToString()?). result == 28 returned false though. And (int)result returned 27. So you'll just need to expect imprecisions like that.
After one hour of trying to find a bug in my code I've finally found the reason. I was trying to add a very small float to 1f, but nothing was happening. While trying to figure out why I found that adding that small float to 0f worked perfectly.
Why is this happening?
Does this have to do with 'orders of magnitude'?
Is there any workaround to this problem?
Thanks in advance.
Edit:
Changing to double precision or decimal is not an option at the moment.
Because precision for a single-precision (32 bit) floating-point value is around 7 digits after the decimal point. Which means the value you are adding is essentially zero, at least when added to 1. The value itself, however, can effortlessly stored in a float since the exponent is small in that case. But to successfully add it to 1 you have to use the exponent of the larger number ... and then the digits after the zeroes disappear in rounding.
You can use double if you need more precision. Performance-wise this shouldn't make a difference on today's hardware and memory is often also not as constrained that you have to think about every single variable.
EDIT: As you stated that using double is not an option you could use Kahan summation, as akuhn pointed out in a comment.
Another option may be to perform intermediary calculations in double-precision and afterwards cast to float again. This will only help, however, when there are a few more operations than just adding a very small number to a larger one.
Floating-point arithmetic
This probably happens because the number of digits of precision in a float is constant, but the exponent can obviously vary.
This means that although you can add your small number to 0, you cannot expect to add it to a number that has an exponent different from 0, since there just won't be enough digits of precision left.
You should read What Every Computer Scientist Should Know About Floating-Point Arithmetic.
It looks like it has something to do with floating point precision. If I were you, I'd use a different type, like decimal. That should fix precision errors.
With float, you only get an accuracy of about seven digits. So your number'll be rounded into 1f. If you want to store such number, use double instead
http://msdn.microsoft.com/en-us/library/ayazw934.aspx
In addition to the accepted answer: If you need to sum up many small number and some larger ones, you should use Kahan Summation.
If performance is an issue (because you can't use double), then binary scaling/fixed-point may be an option. floats are stored as integers, but scaled by a large number (say, 2^16). Intermediate arithmetic is done with (relatively fast) integer operations. The final answer can be converted back to floating point at the end, by dividing by the scaling factor.
This is often done if the target processor lacks a hardware floating-point unit.
You're using the f suffix on your literals, which will make these floats instead of doubles. So your very small float will vanish in the bigger float.