We Keep Coding

How can I make the result to be the same in C++ and C# of double values [duplicate]

This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 3 years ago.
Please notice that I am not asking why a double value changes into another precision, but how can I make the results to be the same.
I appreciate all the comments and answers below, but I didn't get a solution to my problem.
My question again.
I just met a problem when rewriting the code from C++ to C# of the precision problem in double value,
I have a method to calculate the result of a financial product of several double values, but when I compare the results, I got different but similar results from C++ and C# program due to the precision of the double value.
I debug the program in C++ I found that after I set the values to a and b
double a = 4.9;
double b = 5.1;
The value of a and b changes to, a is 4.9000000000000004 and b is 5.0999999999999996.
But in C#, a is still 4.9 and b is still 5.1, and finally, I got the results of -0.062579630921710816(C++ result), and -0.062579644244387111(C# result)
I am totally new in C# and C++, so are there any solutions to make the calculation result to be the same, please (I cannot change the C++ code)?
Or is it possible to make the calculation result to be the same?

It's a display issue - the actual value is as you expect and matches your C++ code. Try this code to see the approximate1 value stored:
double a = 4.9; double b = 5.1;
Console.WriteLine(a.ToString("G20"));
Console.WriteLine(b.ToString("G20"));
The result is:
4.9000000000000004
5.0999999999999996
See here to learn more about why I used G20, and what the other number format strings are.
1 I say approximate value because computers can only store fractions composed of 1/pow(2, n). For example 0.75 is 1/2 + 1/4, so this can be represented accurately. In the same way that we can't accurately represent 1/3 in base 10 (we might write 0.3, 0.33, 0.333, 0.3333, etc.), you can't represent some fractions in base 2 (binary) because they can't be composed of powers of two. See this question for more on that.

Short answer, probably because of compiler optimisations.
Following the IEEE Float spec to the letter would produce exactly the same results. But compilers often make decisions to sacrifice correctness for speed. Applying algebraic rearrangements to your code, fusing multiple operations together, all kinds of optimisations will change the result.
Most C compilers provide compiler flags that allow you to tune which of these optimisations are allowed. I'm not aware if C# provides any explicit guarantees.

Computations in C# as accurately as with Windows Calculator

When I do the following double multiplication in C# 100.0 * 1.005 I get 100,49999999999999 as a result. I believe this is because the exact number (or some intermedia result when evaluting the expression) cannot be represented. When I do the same computation in calc.exe I get 100.5 as expected.
Another example is the ninefold incrementation of 0.001 (that is the first time a deviation occurs) so basically 9d * 0.001d = 0,0090000000000000011. When I do the same computation in calc.exe I get 0.009 as expected.
Now one can argue, that I should choose decimal instead. But with decimal I get the problem with other computations for example with ((1M / 3M) * 3M) = 0,9999999999999999999999999999 while calc.exe says 1.
With calc.exe I can divide 1 by 3 several times until some real small number and then multiply with 3 again as many several times and then I reach exacty 1. I therefore suspect, that calc.exe computes internally with fractions, but obviously with real big ones, because it computes
(677605234775492641 / 116759166847407000) + (932737194383944703 / 2451942503795547000)
where the common denominator is -3422539506717149376 (an overflow occured) when doing a long computation, so it must be at least ulong. Does anybody know how computation in calc.exe is implemented? Is this implementation made somewhere public for reuse?

As described here, calc uses an arbitrary-precision engine for its calculations, while double is standard IEEE-754 arithmetic, and decimal is also floating-point arithmetic, just in decimal, which, as you point out, has the same problems, just in another base.
You can try finding such an arbitrary-precision arithmetic library for C# and use it, e.g. this one (no idea whether it's good; was the first result). The one inside calc is not available as an API, so you cannot use it.
Another point is that when you round the result to a certain number of places (less than 15), you'd also get the intuitively "correct" result in a lot of cases. C# already does some rounding to hide the exact value of a double from you (where 0.3 is definitely not exactly 0.3, but probably something like 0.30000000000000004). By reducing the number of digits you display you lessen the occurrence of such very small differences from the correct value.

Recreating .NET rounding behavior in SQL

I'm trying to migrate a subtle calculation from C#.NET to TSQL and absolutely every result must remain the same, regardless of whether the original C#.NET answer is inaccurate in some cases.
My TSQL script mostly succeeds at this, however, there is some unusual rounding behavior in .NET that I've been unable to capture. I'll demonstrate with two examples:
Here's the .NET code that I'm trying to recreate.
(double)Math.Round(number, 1, MidpointRounding.AwayFromZero );
double number = 6.1499999999999995; // returns 6.1
double number = 6.9499999999999993; // returns 7 (which, to me, seems inconsistent)
Here's the TSQL code that I'm trying to align with the .NET code above.
SELECT ROUND(ROUND(CAST(#Score as decimal(10,1)), 16), 1)
DECLARE #Score DECIMAL(18,16) = 6.1499999999999995 -- returns 6.1
DECLARE #Score DECIMAL(18,16) = 6.9499999999999993 -- returns 6.9 (needs to be 7)
My goal is to get my TSQL code to return 7 instead of 6.9 in the second case. Does anyone know how I can make that happen?

decimal is a decimal-accurate number. double isn't. Since your C# code is using double, only real way to replicate a similar behaviour in T-SQL is to also use a binary floating point number in T-SQL - float (specifically, double corresponds to float(53)). The rounding behaviour isn't "unusual", it follows the IEEE-754 standard.
However, if you care about decimal precision (as it seems you should), you should really push switching the C# code to use decimal instead of double. Binary floating points are a terrible choice when you want decimal precision.

I don't think there a reliable way to get what you want using plain T-SQL.
Your logic is wrong:
CAST(#Score as decimal(10,1)) already rounds before you apply ROUND
DECLARE #Score decimal(18,16) = 6.9499999999999993
SELECT #Score, CAST(#Score as decimal(10,1))
6.9499999999999993 6.9
And you ROUND to 16 digits and then apply another ROUND to 1 digit, but ROUND(ROUND(n, 16), 1) might be a different result than ROUND(#n,1) directly and is definitely not the way .NET rounds.
Another problem is the rounding rule, you specify MidpointRounding.AwayFromZero, but afaik the default for T-SQL is Rounding to Even.
You might check SQL Server Rounding Methods and try to implement Round to Even but then there's still the problem of significant digits, DEC(18,16) has 18, but DOUBLE only 16.
If you can't use a CLR-function in T-SQL or switch to DECIMAL in .NET you're probably doomed...

Consistency: Dividing an integer by the powers of 2 vs powers of 10?

This is a question concerning cross-platform consistency and determinism of floating point operations (IE yield different results on different CPUs/sysetms)
Which one is more likely to stay cross-platform consistent(pseudo code):
float myFloat = float ( myInteger) / float( 1024 )
or
float myFloat = float ( myInteger ) / float( 1000 )
Platforms are C# and AS3.
.
AS3 versions:
var myFloat:Number = myInteger / 1000 // AS3
var myFloat:Number = myInteger / 1024 // AS3
- Ok I've added AS3 version for clarification, which is equivalent to the 'C pseudo code' above . As you can see in AS3 all calculations, even on integers, are performed as Floats automatically, a cast is not required ( and nor can you avoid it or force the runtime to perform true integer divisions )
Hopefully this explains why im 'casting' everything into Floats: I am not! that just simply what happens in one of the target languages!

The first one is likely the same on both platforms, since there are no representation issues. In particular for small integers (highest 8 bits unused) there is one exact result, and it's very likely that this result will be used.
But I wouldn't rely on it. If you need guaranteed determinism, I recommend implementing the required arithmetic yourself on top of plain integers. For example using a fixed point representation.
The second one is likely to be inconsistent, even when using the same C# code on different hardware or .net versions. See the related question Is floating-point math consistent in C#? Can it be?

I suggest you read the IEEE 754-1985 standard. A copy can be purchased for $43. Although superseded by the 2008 version, it is an excellent introduction to floating-point because it is only 20 pages and is quite readable. It will show you why both dividing by 1000 and by 1024 are deterministic and why the former may have error but the latter does not (except in cases of underflow). It will also give you a basis for understanding the answers you have been given and why you are on the wrong track.

Which one is more likely to stay cross-platform consistent(pseudo code):
Dividing by 1024.
Every binary-based floating point systems (which are IEEE754, IBM, VAX, Cray) which applies division by 1024 to all finite numbers will yield exact results in the given representation. The reason is that dividing by 1024 is equivalent to
shifting the bits 10 positions to the right which means
decreasing the binary exponent by 10
If the number is too small (for IEEE754 1E-38/1E-308), you will lose an exact result, but this is not a problem of the operation, but of the limited range of the number...it simply cannot display such small results accurately.
As no rounding is necessary, there can be no difference due to rounding (and yes, while most programming languages use round to even, some enable choosing another rounding mode).

Calculate factorials in C#

How can you calculate large factorials using C#? Windows calculator in Win 7 overflows at Factorial (3500). As a programming and mathematical question I am interested in knowing how you can calculate factorial of a larger number (20000, may be) in C#. Any pointers?
[Edit] I just checked with a calc on Win 2k3, since I could recall doing a bigger factorial on Win 2k3. I was surprised by the way things worked out.
Calc on Win2k3 worked with even big numbers. I tried !50000 and I got an answer, 3.3473205095971448369154760940715e+213236
It was very fast while I did all this.
The main question here is not only to find out the appropriate data type, but also a bit mathematical. If I try to write a simple factorial code in C# [recursive or loop], the performance is really bad. It takes multiple seconds to get an answer. How is the calc in Windows 2k3 (or XP) able to perform such a huge factorial in less than 10 seconds? Is there any other way of calculating factorial programmatically in C#?

Have a look at the BigInteger structure:
http://msdn.microsoft.com/en-us/library/system.numerics.biginteger.aspx
Maybe this can help you implement this functionality.
CodeProject has an implementation for older versions of the framework at http://www.codeproject.com/KB/cs/biginteger.aspx.

If I try to write a simple factorial code in C# [recursive or loop], the performance is really bad. It takes multiple seconds to get an answer.
Let's do a quick order-of-magnitude calculation here for a naive implementation of factorial that performs n multiplications. Suppose we are on the last step. 19999! is about 218 bits. 20000 is about 25 bits; we'll assume that it is a 32 bit integer. The final multiplication therefore involves the addition of up to 25 partial results each roughly 218 bits long. The number of bit operations will therefore be on the order of 223.
That's for the last stage; there will be 20000 = 216 such operations at each stage, so that is a total of about 239 operations. Some of them will of course be cheaper, but we're going for an order of magnitude here.
A modern processor does about 232 operations per second. Therefore it will take about 27 seconds to get the result.
Of course, the big integer library writers were not naive; they take advantage of the ability of the chip to do many bit operations in parallel. They're probably doing the math in 32 bit chunks, giving speedups of a factor of 25. So our total order-of-magnitude calculation is that it should take about 22 seconds to get a result.
22 is 4. So your observation that it takes a few seconds to get a result is expected.
How is the calc in Windows 2k3 (or XP) able to perform such a huge factorial in less than 10 seconds?
I don't know. Extreme cleverness in exploiting the math operations on the chip probably. Or, using a non-naive algorithm for calculating factorial. Or, possibly they are using Stirling's Approximation and getting an inexact result.
Is there any other way of calculating factorial programmatically in C#?
Sure. If all you care about is the order of magnitude then you can use Stirling's Approximation. If you care about the exact value then you're going to have to compute it.

There exist sophisticated computational algorithms for efficiently computing the factorials of large, arbitrary precision numbers. The Schönhage–Strassen algorithm, for instance, allows you to perform asymptotically fast multiplication for arbitrarily large integers.
Case in point, Mathematica computes 22000! on my machine in less than 1 second. The Implementation Notes page at reference.wolfram.com states:
(Mathematica's) n! uses an O(log(n) M(n)) algorithm of Schönhage based on dynamic decomposition to prime powers.
Unfortunately, the implementation of such algorithms is both complicated and error prone. Rather than trying to roll your own implementation, it may be wiser for you to license a copy of Mathematica (or a similar product that meets your functional and performance needs) and either use it, or a .NET programming interface to it, to perform your computation.

Have you looked at System.Numerics.BigInteger?

Using System.Numerics BigInteger
var bi = new BigInteger(1);
var factorial = 171;
for (var i = 1; i <= factorial; i++)
{
bi *= i;
}
will be calculated to
1241018070217667823424840524103103992616605577501693185388951803611996075221691752992751978120487585576464959501670387052809889858690710767331242032218484364310473577889968548278290754541561964852153468318044293239598173696899657235903947616152278558180061176365108428800000000000000000000000000000000000000000
For 50000! it takes a couple seconds to calculate but it seems to work and the result is a 213237 digit number and that's also what Wolfram says.

You will probably have to implement your own arbitrary precision numeric type.
There are various approaches. probably not the most efficient, but perhaps the simplest is to have variable length arrays of byte (unsigned char). Each element represents a digit. ideally this would be included in a class, and you can then add a method which let's you multiply the number with another arbitrary precision number. A multiply with a standard C# integer would probably also be a good idea, but a little trickier to implement.

Since they don't give you the result down to the last digit, they may be "cheating" using some approximation.
Check out http://mathworld.wolfram.com/StirlingsApproximation.html
Using Stirling's formula you can calculate (an approximation of) the factorial of n in logn time. Of course, they might as well have a dictionary with pre-calculated values of factorial(n) for every n up to one million, making the calculator show the result extremely fast.

This answer covers limits for basic .Net types to compute and represent n!
Basic code to calculate factorial for "SomeType" that supports multiplication:
SomeType factorial = 1;
int n = 35;
for (int i = 1; i <= n; i++)
{
factorial *= i;
}
Limits for built in number types:
short - correct results up to 7!, incorrect results afterwards, code returns 0 starting 18 (similar to int)
int - correct results up to 12!, incorrect results afterwards, code returns 0 starting at 34 (Why computing factorial of realtively small numbers (34+) returns 0)
float - precise results up to 14!, correct but not precise afterwards, returns infinity starting at 35
long - correct results up to 20!, incorrect results afterwards, code returns 0 starting at 66 (similar to int)
double - precise results up to 22!, correct but not precise afterwards, returns infinity starting at 171
BigInteger - precise and upper limit is set by memory usage only.
Note: integer types overflow pretty quickly and start producing incorrect results. Realistically if you need factorials for any practical usage long is the type to go (up to 20!), if you can't expect limited numbers - BigInteger is the only type provided in .Net Framework to provide precise results (albeit slow for large numbers as there is no built-in optimized n! method)

You need a special big-number library for this. This link introduces the System.Numeric.BigInteger class, and incidentally has an example program that calculates factorials. But don't use the example! If you recurse like that, your stack will grow horribly. Just write a for-loop to do the multiplication.

I don't know how you could do this in a language without arbitrary precision arithmetic. I guess a start could be to count factors of 5 and 2, removing them from the product, and add on these zeroes at the end.
As you can see there are many.
>>> factorial(20000)
<<non-zeroes removed>>0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000L