What is the best data type to use when storing geopositional data in C#? I would use decimal for its exactness, but operations on decimal floating point numbers are slower then binary floating point numbers (double).
I read that most of the time you won't need any more than 6 or 7 digits of precision for latitude or longitude. Does the inexactness of doubles even matter then or can it be ignored?
Go for double, there are several reasons.
Trigonometric functions are available only for double
Precision of double (range of 100 nanometers) is far beyond anything you'll ever require for Lat/Lon values
GeoCoordinate Class and third-Party modules (e.g. DotSpatial) also use double for coordinates
A double has up to 15 decimal digits of precision. So, lets assume three of those digits are going to be on the left of the decimal point for lat/long values (max of 180deg). This leaves 12 digits of precision on the right. Since a degree of lat/long is ~111km, 5 of those 12 digits would give us precision to the meter. 3 more digits would give us precision to the millimeter. The remaining 4 digits would get us precision to around 100 nanometers. Since double will win from the perspective of performance and memory, I see no reason to even consider using decimal.
I faced this question quite a while ago when i started with spacial programming.
I read a book a while ago that led me to this.
//sql server has a really cool dll that deals with spacial data such like
//geography points and so on.
//add this namespace
Using Microsoft.SqlServer.Types;
//SqlGeography.Point(dblLat, dblLon, srid)
var lat_lon_point = Microsoft.SqlServer.Types.SqlGeography.Point(lat, lon, 4326);
This is the best way when working in your application with spacial data.
then to save the data use this in sql
CREATE TABLE myGeoTable
{
LatLonPoint GEOMETRY
}
else, if you are using something else that isnt sql just convert the point to hexadecimal and store it. I know after a long time using spacial that this is the safest.
Double
Combining the answers, it is how Microsoft represents it itself in SqlGeography library
[get: Microsoft.SqlServer.Server.SqlMethod(IsDeterministic=true, IsPrecise=true)]
public System.Data.SqlTypes.SqlDouble Lat { get; }
Property Value
SqlDouble
A SqlDouble value that specifies the latitude.
If you are using .net ef core, I would recommend you the NetTopologySuite library.
Read the full documentation at below link:
https://learn.microsoft.com/en-us/ef/core/modeling/spatial
Related
This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 7 years ago.
If I execute the following expression in C#:
double i = 10*0.69;
i is: 6.8999999999999995. Why?
I understand numbers such as 1/3 can be hard to represent in binary as it has infinite recurring decimal places but this is not the case for 0.69. And 0.69 can easily be represented in binary, one binary number for 69 and another to denote the position of the decimal place.
How do I work around this? Use the decimal type?
Because you've misunderstood floating point arithmetic and how data is stored.
In fact, your code isn't actually performing any arithmetic at execution time in this particular case - the compiler will have done it, then saved a constant in the generated executable. However, it can't store an exact value of 6.9, because that value cannot be precisely represented in floating point point format, just like 1/3 can't be precisely stored in a finite decimal representation.
See if this article helps you.
why doesn't the framework work around this and hide this problem from me and give me the
right answer,0.69!!!
Stop behaving like a dilbert manager, and accept that computers, though cool and awesome, have limits. In your specific case, it doesn't just "hide" the problem, because you have specifically told it not to. The language (the computer) provides alternatives to the format, that you didn't choose. You chose double, which has certain advantages over decimal, and certain downsides. Now, knowing the answer, you're upset that the downsides don't magically disappear.
As a programmer, you are responsible for hiding this downside from managers, and there are many ways to do that. However, the makers of C# have a responsibility to make floating point work correctly, and correct floating point will occasionally result in incorrect math.
So will every other number storage method, as we do not have infinite bits. Our job as programmers is to work with limited resources to make cool things happen. They got you 90% of the way there, just get the torch home.
And 0.69 can easily be represented in
binary, one binary number for 69 and
another to denote the position of the
decimal place.
I think this is a common mistake - you're thinking of floating point numbers as if they are base-10 (i.e decimal - hence my emphasis).
So - you're thinking that there are two whole-number parts to this double: 69 and divide by 100 to get the decimal place to move - which could also be expressed as:
69 x 10 to the power of -2.
However floats store the 'position of the point' as base-2.
Your float actually gets stored as:
68999999999999995 x 2 to the power of some big negative number
This isn't as much of a problem once you're used to it - most people know and expect that 1/3 can't be expressed accurately as a decimal or percentage. It's just that the fractions that can't be expressed in base-2 are different.
but why doesn't the framework work around this and hide this problem from me and give me the right answer,0.69!!!
Because you told it to use binary floating point, and the solution is to use decimal floating point, so you are suggesting that the framework should disregard the type you specified and use decimal instead, which is very much slower because it is not directly implemented in hardware.
A more efficient solution is to not output the full value of the representation and explicitly specify the accuracy required by your output. If you format the output to two decimal places, you will see the result you expect. However if this is a financial application decimal is precisely what you should use - you've seen Superman III (and Office Space) haven't you ;)
Note that it is all a finite approximation of an infinite range, it is merely that decimal and double use a different set of approximations. The advantage of decimal is it produces the same approximations that you would if you were performing the calculation yourself. For example if you calculated 1/3, you would eventually stop writing 3's when it was 'good enough'.
For the same reason that 1 / 3 in a decimal systems comes out as 0.3333333333333333333333333333333333333333333 and not the exact fraction, which is infinitely long.
To work around it (e.g. to display on screen) try this:
double i = (double) Decimal.Multiply(10, (Decimal) 0.69);
Everyone seems to have answered your first question, but ignored the second part.
I have a class that does some length calculations based on a height on a ticket. It's been in place for years and working quite well... Until we got a unique ticket size.
They are entered by sales people in inches and are normally nice numbers like 3, 4 or 3.5 and store in a database - This one is however 3.66666 recurring (or 11/3) But it is being entered as 3.666 and causing the calculation to fail due to lost precision.
I have thought of a bit of a hack to restore precision for certain numbers, but thought maybe someone knows of a better way of getting a 3.666 or a 93.1333 back to it's number + two thirds status?
Thanks,
Mick.
As you explained in comments I see your point now. I've checked the numbers:
168000 / 3.666 = 45826.5139
168000 / 3.666666 = 45818.1901488
168000 * 3 / 11 = 45818.1818182
It makes a difference of 8 tickets. I have a feeling that your issue can be solved in many ways. On the side of user input for example. Or on the side of database. But back to your question:
How do I convert 3.666 or a 93.1333 back to it's number + two thirds
status?
You are looking for converting decimal (or double) to fraction.
There is already a question on SO: Algorithm for simplifying decimal to fractions which has many answeres. I've tested some of them, and none of them were satisfying. Some of them don't even hanlde recurrence. Perhaps I've missed the correct one, you can look by yourself.
Anyway, I believe you don't need to fully implement a conversion from 1.666 to 3/2, since it's not easy and you have a real-world sizes. You've said, that most of the time numbers are aroung 3, 3.5, 4 etc. So I suggest you to take a look at a question I've linked above and search for an algorythm of detecting the recurrence number. It was also discussed here How to know the repeating decimal in a fraction?
After what just convert 1.666 to 1.666666, since 1/1000000 of inch won't mess your calculations, as numbers above show.
It would be difficult to get the accurate value of double as double is floating point.
The MSDN says:
Remember that a floating-point number
can only approximate a decimal number,
and that the precision of a
floating-point number determines how
accurately that number approximates a
decimal number. By default, a Double
value contains 15 decimal digits of
precision, although a maximum of 17
digits is maintained internally. The
precision of a floating-point number
has several consequences:
Two floating-point numbers that appear equal for a particular
precision might not compare equal
because their least significant digits
are different.
A mathematical or comparison operation that uses a floating-point
number might not yield the same result
if a decimal number is used because
the floating-point number might not
exactly approximate the decimal
number.
I want to avoid any precision errors when doing math on bitcoin values.
1) Is 'decimal' the best option to use for bitcoin currency in C#?
2) When converting from string to number, are there any precision errors I need to be aware of?
Thanks
Bitcoin amounts can range from 1 Satoshi (0.00000001 BTC) to nearly
2,100,000,000,000,000 (21,000,000 BTC). To avoid rounding errors, you
must make sure your PHP implementation supports the full range of
Bitcoin values without losing precision. Most PHP implementations use
IEEE 64-bit double-precision floating point numbers with 53 bits of
precision, which is enough to correctly represent the full range of
bitcoin values.
Even-though this is related to PHP, it is still relevant. As suggested, you should use Decimal value type.
The last block that will generate coins will be block #6,929,999 which
should be generated at or near the year 2140. The total number of
coins in circulation will then remain static at 20,999,999.9769 BTC.
Even if the allowed precision is expanded from the current 8 decimals,
the total BTC in circulation will always be slightly below 21 million
(assuming everything else stays the same). For example, with 16
decimals of precision, the end total would be 20,999,999.999999999496
BTC.
An example of a String being converted to Decimal and keeping precision:
var maxBtc = "20999999.999999999496";
var maxBtcDecimal = Decimal.Parse(maxBtc, NumberStyles.AllowDecimalPoint);
Converting back:
var maxBtcString = Convert.ToString(maxBtcDecimal);
Simple math:
var oneBtc = new decimal(1.000000000000);
var newBtcValue = maxBtcDecimal - oneBtc;
Decimal is the best option available.
Take a look at this working C# bitcoin library that uses Decimal for bitcoin values: https://github.com/GeorgeKimionis/BitcoinLib
You need to be aware that in C# Decimal is not fixed-floating so you need to configure your O/RM or your DB to keep exactly 8 decimal points.
From MSDN
Compared to floating-point types, the decimal type has more precision and a smaller range, which makes it appropriate for financial and monetary calculations.
If you are doing mathematical operations on Bitcoin amounts, you should use Decimal.
Decimal.Parse()
should be used to convert from string to Decimal. See also this other Stackoverflow question
I have little experience with MongoDB. I am usual working on large scale SQL server DBs.
MongoDB only supports double and there is no decimal. The C# driver serializes decimals as strings.
What functionality do I miss if I store decimals as strings in
MongoDB?
Is there a way to set a default serialization of decimals as double
(AllowTruncation) without having to put an Attribute on each
property?
What do I lose in precision if I used Bson double?
Thanks for your help!
UPDATE
I have an existing application model that uses decimals in C#. I want to use MongoDB as a new DB layer and change as little in the existing app as possible. Thats why I am looking for a way to map decimals in C# to double in MongoDB.
I understand that I loose precision and would have to analyze the side effects of it. My only remaining question is to know if there is a way to set a default serialization of decimals as double.
Thanks again. Great answers and comments so far.
As of Mongodb 3.4 and the 2.4 Mongodb C# driver, decimal types are supported.
The properties of your document must have the [BsonRepresentation(BsonType.Decimal128)] attribute found in the MongoDB.Bson.Serialization.Attributes namespace.
this will map to "YourDecimalValue" : NumberDecimal("100.0000") in MongodDB. Robomongo supports the new decimal type from version 1.1 Beta.
I will answer your question partially (because I do not know much about C#).
So what will you lose if you will store decimals as strings.
your numbers on average would weight more (each double number cost 8 bytes to store which means that every string that has more then 8 chars will weight more). Because of these your indexes (if they will be built on this field would grow)
you will not be able to use operators which takes numbers as arguments $inc, $bit, $mod, $min, $max and in 2.6 version $mul. (May be I forgot something)
you will not be able to compare numbers (may be '1.65' and '1.23' is comparable as a string, but definitely not numbers with e and minuses somewhere in between). Because of this operations which build on top of comparison like $sort, and all these $gte, $gt, $lte, $lt will not work correctly.
What will you lose in precision if you store decimal as double:
based on this, Decimal in C# has 28-29 significant digits, whereas looking at my first link and checking the spec for double precision you see that it has 15-17 significant digits. This is basically what you will lose
another really important thing which people sometimes forget when dealing with double floats is illustrated below:
.
db.c.insert({_id : 1, b : 3.44})
db.c.update({_id : 1},{$inc : {b : 1}})
db.c.find({b: 4.44}) // WTf, where is my document? There is nothing there
Regarding the 2-nd subquestion:
Is there a way to set a default serialization of decimals as double
(AllowTruncation) without having to put an Attribute on each property?
I do not really understood it, so I hope someone would be able to answer it.
I looked at decimal in C# but I wasnt 100% sure what it did.
Is it lossy? in C# writing 1.0000000000001f+1.0000000000001f results in 2 when using float (double gets you 2.0000000000002 which is correct) is it possible to add two things with decimal and not get the correct answer?
How many decimal places can I use? I see the MaxValue is 79228162514264337593543950335 but if i subtract 1 how many decimal places can I use?
Are there quirks I should know of? In C# its 128bits, in other language how many bits is it and will it work the same way as C# decimal does? (when adding, dividing, multiplication)
What you're showing isn't decimal - it's float. They're very different types. f is the suffix for float, aka System.Single. m is the suffix for decimal, aka System.Decimal. It's not clear from your question whether you thought this was actually using decimal, or whether you were just using float to demonstrate your fears.
If you use 1.0000000000001m + 1.0000000000001m you'll get exactly the right value. Note that the double version wasn't able to express either of the individual values exactly, by the way.
I have articles on both kinds of floating point in .NET, and you should read them thoroughly, along other resources:
Binary floating point (float/double)
Decimal floating point (decimal)
All floating point types have their limits of course, but in particular you should not expect binary floating point to accurately represent decimal values such as 0.1. It still can't represent anything that isn't exactly representable in 28/29 decimal digits though - so if you divide 1 by 3, you won't get the exact answer of course.
You should also note that the range of decimal is considerably smaller than that of double. So while it can have 28-29 decimal digits of precision, you can't represent truly huge numbers (e.g. 10200) or miniscule numbers (e.g. 10-200).
Decimals in programming are (almost) never 100% accurate. Sometimes it's even better to multiply the decimal value with a very high number and then calculate, but that's only if you're for example sure that the value is always between 0 and 100(so it won't get out of range of the maxvalue)
Floting point is inherently imprecise. Some numbers can't be represented faithfully. Decimal is a large floating point with high precision. If you look on the page at msdn you can see there are "28-29 significant digits." The .net framework classes are language agnostic. they will work the same in every language that uses .net.
edit (in response to Jon Skeet): If you initialize the Decimal class with the numbers above, which are less than 28 digits each after the decimal point, the number will be stored faithfully as long as the binary representation is exact. Since it works in 64-bit format, I assume the 128-bit will handle it perfectly fine. Some numbers, such as 0.1, will never be exactly representable because they are a repeating sequence in binary.