I'm trying to migrate a subtle calculation from C#.NET to TSQL and absolutely every result must remain the same, regardless of whether the original C#.NET answer is inaccurate in some cases.
My TSQL script mostly succeeds at this, however, there is some unusual rounding behavior in .NET that I've been unable to capture. I'll demonstrate with two examples:
Here's the .NET code that I'm trying to recreate.
(double)Math.Round(number, 1, MidpointRounding.AwayFromZero );
double number = 6.1499999999999995; // returns 6.1
double number = 6.9499999999999993; // returns 7 (which, to me, seems inconsistent)
Here's the TSQL code that I'm trying to align with the .NET code above.
SELECT ROUND(ROUND(CAST(#Score as decimal(10,1)), 16), 1)
DECLARE #Score DECIMAL(18,16) = 6.1499999999999995 -- returns 6.1
DECLARE #Score DECIMAL(18,16) = 6.9499999999999993 -- returns 6.9 (needs to be 7)
My goal is to get my TSQL code to return 7 instead of 6.9 in the second case. Does anyone know how I can make that happen?
decimal is a decimal-accurate number. double isn't. Since your C# code is using double, only real way to replicate a similar behaviour in T-SQL is to also use a binary floating point number in T-SQL - float (specifically, double corresponds to float(53)). The rounding behaviour isn't "unusual", it follows the IEEE-754 standard.
However, if you care about decimal precision (as it seems you should), you should really push switching the C# code to use decimal instead of double. Binary floating points are a terrible choice when you want decimal precision.
I don't think there a reliable way to get what you want using plain T-SQL.
Your logic is wrong:
CAST(#Score as decimal(10,1)) already rounds before you apply ROUND
DECLARE #Score decimal(18,16) = 6.9499999999999993
SELECT #Score, CAST(#Score as decimal(10,1))
6.9499999999999993 6.9
And you ROUND to 16 digits and then apply another ROUND to 1 digit, but ROUND(ROUND(n, 16), 1) might be a different result than ROUND(#n,1) directly and is definitely not the way .NET rounds.
Another problem is the rounding rule, you specify MidpointRounding.AwayFromZero, but afaik the default for T-SQL is Rounding to Even.
You might check SQL Server Rounding Methods and try to implement Round to Even but then there's still the problem of significant digits, DEC(18,16) has 18, but DOUBLE only 16.
If you can't use a CLR-function in T-SQL or switch to DECIMAL in .NET you're probably doomed...
Related
A proposed fix for a rounding (for reporting) problem is
private double RoundedResult(double v, int digits)
{
// return Math.Round(value, digits);
// sometimes does not do proper rounding per user feedback
// proposed fix relies on a fixed decimal (F) format rendering converted back to double
return
Convert.ToDouble
(
v.ToString("F"+digits);
);
}
I am uncertain as to the vagaries of the Fn format.
Question: Is the above a reasonable fix? , and can it cause other problems later ?
Background
An individual report will have < 1,000 rounded numbers in it, so I am not worried about performance.
I am working with legacy system using SQL Server and ASP NET MVC application code. Data values are double? and values need to be rounded to a varying number of decimals for reporting. I know the system should be using decimal data values instead of double, but I am stuck with the legacy double.
The end-user has complained about this particular situation (and other similar):
The text data file has 0.000065, the data base viewer shows 0.000065 and the reporting (to 5 decimal places) shows 0.00006 instead of 0.00007. Reporting should always round up.
The text file information is
I am working through the specifics of
double? value = 0.000065;
I am aware that 0.000065d must actually be ever so slightly less than 0.000065m:
Math.Round(value.Value,5,...) returns 0.00006, using both MidpointRounding.AwayFromZero and ToEven.
Debugger shows 0.000065d-0.000065f as -1.7695129487117073E-13 double
Single cast of the value rounds up, the debugger watch shows
Math.Round((Single)value,5) 7E-05 double
Slightly perplexing
value.Value.ToString("F5") returns "0.00007"
I don't know if F5 does any sort of internal casting, but the result corresponds with rounding the single cast. ToString() (aka String.Format()) has some sort of mojo going on.
The single case of rounding the single cast of the double 0.000065 'worked' by yielding 0.0007, but probably won't in all cases.
Fiddle code
I wrote a .NET Fiddle to iterate over a large number of 'ending in 5' cases, and it seems that ToString("F5") always yielded the 'human-expected' result.
Mathematically, 0.9 recurring can be shown to be equal to 1. This question however, is not about infinity, convergence, or the maths behind this.
The above assumption can be represented using doubles in C# with the following.
var oneOverNine = 1d / 9d;
var resultTimesNine = oneOverNine * 9d;
Using the code above, (resultTimesNine == 1d) evaluates to true.
When using decimals instead, the evaluation yields false, yet, my question is not about the disparate precision of double and decimal.
Since no type has infinite precision, how and why does double maintain such an equality where decimal does not? What is happening literally 'between the lines' of code above, with regards to the manner in which the oneOverNine variable is stored in memory?
It depends on the rounding used to get the closest representable value to 1/9. It could go either way. You can investigate the issue of representability at Rob Kennedy's useful page: http://pages.cs.wisc.edu/~rkennedy/exact-float
But don't think that somehow double is able to achieve exactness. It isn't. If you try with 2/9, 3/9 etc. you will find cases where the rounding goes the other way. The bottom line is that 1/9 is not exactly representable in binary floating point. And so rounding happens and your calculations are subject to rounding errors.
What is happening literally 'between the lines' of code above, with regards to the manner in which the oneOverNine variable is stored in memory?
What you're asking about is called IEEE 754. This is the spec that C#, it's underlying .Net runtime, and most other programming platforms use to store and manipulate decimal values. This is because support for IEEE 754 is typically implemented directly at the CPU/chipset-level, making it both far more performant than an alternative implemented solely in software and far easier when building compilers, because the operations will map almost directly to specific CPU instructions.
Consider the following code snippet in Python:
m = int(math.sqrt(n))
For n = 25, it should give m = 5 (and it does in my shell). But from my C experience I know that using such expression is a bad idea, as sqrt function may return a slightly lower value than the real value, and then after rounding i may get m = 4 instead of m = 5. Is this limitation also involved in python? And if this is the case, what is be the best way to write such expressions in python? What will happen if I use Java or C#?
Besides, if there is any inaccuracy, what factors controls the amount of it?
For proper rounding, use round(); it rounds to the nearest whole number, but returns a float. Then you may construct an int from the result.
(Most probably your code is not performance-critical and you will never notice any slowdown associated with round(). If you do, you probably should be using numpy anyway.)
If you are very concerned with the accuracy of sqrt, you could use the decimal.Decimal class from the standard library, which provides its own sqrt function. The Decimal class can be set to greater precision than regular Python floats. That said, it may not matter if you are rounding anyways. The Decimal class results in exact numbers (from the docs):
The exactness [of Decimal] carries over into arithmetic. In decimal floating point,
0.1 + 0.1 + 0.1 - 0.3 is exactly equal to zero. In binary floating point, the result is 5.5511151231257827e-017. While near to zero, the
differences prevent reliable equality testing and differences can
accumulate. For this reason, decimal is preferred in accounting
applications which have strict equality invariants.
The solution is easy. If you're expecting an integer result, use int(math.sqrt(n)+.1). If the value is a little more or less than the integer result, it will round to the correct value.
This is a question concerning cross-platform consistency and determinism of floating point operations (IE yield different results on different CPUs/sysetms)
Which one is more likely to stay cross-platform consistent(pseudo code):
float myFloat = float ( myInteger) / float( 1024 )
or
float myFloat = float ( myInteger ) / float( 1000 )
Platforms are C# and AS3.
.
AS3 versions:
var myFloat:Number = myInteger / 1000 // AS3
var myFloat:Number = myInteger / 1024 // AS3
- Ok I've added AS3 version for clarification, which is equivalent to the 'C pseudo code' above . As you can see in AS3 all calculations, even on integers, are performed as Floats automatically, a cast is not required ( and nor can you avoid it or force the runtime to perform true integer divisions )
Hopefully this explains why im 'casting' everything into Floats: I am not! that just simply what happens in one of the target languages!
The first one is likely the same on both platforms, since there are no representation issues. In particular for small integers (highest 8 bits unused) there is one exact result, and it's very likely that this result will be used.
But I wouldn't rely on it. If you need guaranteed determinism, I recommend implementing the required arithmetic yourself on top of plain integers. For example using a fixed point representation.
The second one is likely to be inconsistent, even when using the same C# code on different hardware or .net versions. See the related question Is floating-point math consistent in C#? Can it be?
I suggest you read the IEEE 754-1985 standard. A copy can be purchased for $43. Although superseded by the 2008 version, it is an excellent introduction to floating-point because it is only 20 pages and is quite readable. It will show you why both dividing by 1000 and by 1024 are deterministic and why the former may have error but the latter does not (except in cases of underflow). It will also give you a basis for understanding the answers you have been given and why you are on the wrong track.
Which one is more likely to stay cross-platform consistent(pseudo code):
Dividing by 1024.
Every binary-based floating point systems (which are IEEE754, IBM, VAX, Cray) which applies division by 1024 to all finite numbers will yield exact results in the given representation. The reason is that dividing by 1024 is equivalent to
shifting the bits 10 positions to the right which means
decreasing the binary exponent by 10
If the number is too small (for IEEE754 1E-38/1E-308), you will lose an exact result, but this is not a problem of the operation, but of the limited range of the number...it simply cannot display such small results accurately.
As no rounding is necessary, there can be no difference due to rounding (and yes, while most programming languages use round to even, some enable choosing another rounding mode).
I know this has been discussed time and time again, but I can't seem to get even the most simple example of a one-step division of doubles to result in the expected, unrounded outcome in C# - so I'm wondering if perhaps there's i.e. some compiler flag or something else strange I'm not thinking of. Consider this example:
double v1 = 0.7;
double v2 = 0.025;
double result = v1 / v2;
When I break after the last line and examine it in the VS debugger, the value of "result" is 27.999999999999996. I'm aware that I can resolve it by changing to "decimal," but that's not possible in the case of the surrounding program. Is it not strange that two low-precision doubles like this can't divide to the correct value of 28? Is the only solution really to Math.Round the result?
Is it not strange that two low-precision doubles like this can't divide to the correct value of 28?
No, not really. Neither 0.7 nor 0.025 can be exactly represented in the double type. The exact values involved are:
0.6999999999999999555910790149937383830547332763671875
0.025000000000000001387778780781445675529539585113525390625
Now are you surprised that the division doesn't give exactly 28? Garbage in, garbage out...
As you say, the right result to represent decimal numbers exactly is to use decimal. If the rest of your program is using the wrong type, that just means you need to work out which is higher: the cost of getting the wrong answer, or the cost of changing the whole program.
Precision is always a problem, in case you are dealing with float or double.
Its a known issue in Computer Science and every programming language is affected by it. To minimize these sort of errors, which are mostly related to rounding, a complete field of Numerical Analysis is dedicated to it.
For instance, let take the following code.
What would you expect?
You will expect the answer to be 1, but this is not the case, you will get 0.9999907.
float v = .001f;
float sum = 0;
for (int i = 0; i < 1000; i++ )
{
sum += v;
}
It has nothing to do with how 'simple' or 'small' the double numbers are. Strictly speaking, neither 0.7 or 0.025 may be stored as exactly those numbers in computer memory, so performing calculations on them may provide interesting results if you're after heavy precision.
So yes, use decimal or round.
To explain this by analogy:
Imagine that you are working in base 3. In base 3, 0.1 is (in decimal) 1/3, or 0.333333333'.
So you can EXACTLY represent 1/3 (decimal) in base 3, but you get rounding errors when trying to express it in decimal.
Well, you can get exactly the same thing with some decimal numbers: They can be exactly expressed in decimal, but they CAN'T be exactly expressed in binary; hence, you get rounding errors with them.
Short answer to your first question: No, it's not strange. Floating-point numbers are discrete approximations of the real numbers, which means that rounding errors will propagate and scale when you do arithmetic operations.
Theres' a whole field of mathematics called numerical analyis that basically deal with how to minimize the errors when working with such approximations.
It's the usual floating point imprecision. Not every number can be represented as a double, and those minor representation inaccuracies add up. It's also a reason why you should not compare doubles to exact numbers. I just tested it, and result.ToString() showed 28 (maybe some kind of rounding happens in double.ToString()?). result == 28 returned false though. And (int)result returned 27. So you'll just need to expect imprecisions like that.