This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
In Excel I have a simple calculation:
total = £103000
percent: 2.14%
charges = £2,199.05 - Excel formula (total*percent)/100
In C# I can't get this to calculate correctly:
double percent = 2.14;
double total = 103000;
double charges = (total * percent) / 100;
returns £2,204.20
I'm sure there is some rounding going on somewhere which is making the calculation incorrect.
I wouldnt expect the spreadsheet to be incorrect, as it was provided by a financial advisor/expert!
I've uploaded a version of the spreadsheet here:
See Page/Tab 2 for the calculations, cell K20 is where to charges appear
I did the algebra and the real value is 2.135%.
Examining the spreadsheet provided via Google Docs confirms that the actual percentage is 2.135%. 2.14% is displayed due to format settings.
You should double-check your Excel values and formula because it is giving you the wrong results. Floats or not, it shoudln't be off that much on those numbers in that calculation.
You could change it to a float for more accuracy if required, but according to my calc.exe 2,204.20 is spot on.
This is likely because Excel uses 32-bit floats for their calculations.
The answer you received in C# is correct.
If you want the Excel answer to match, you need to change the data type of the cell you are looking at.
Related
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
I've got a SQL Server table with a DECIMAL(10,2) column. This should fit numbers up to 99999999.99. But when I insert any number >= 1000 into it via the following C# code, I get the exception
Error converting data type double to float.
If the value being inserted is < 1000, it works fine. The C# value is a double by the way.
Can anyone explain this please?
// ...
Database.AddInParameter(cmd, "#AssessmentScaleScore", DbType.Double,
(goal.StudentAssessmentInstanceID > 0
&& goal.AssessmentScaleScore > 0)
? goal.AssessmentScaleScore
: System.Data.SqlTypes.SqlDouble.Null);
// ...
Database.ExecuteDataSet(cmd);
So again, the goal.AssessmentScaleScore is a double, and I'm inserting it into a DECIMAL(10,2) column in the DB. If the value is < 1000 it works fine, but if its >= 1000 I get an exception.
I don't get it.
Sorry, I jumped the gun asking this here.
Turns out I'm an idiot and my parameter in my stored procedure has it as DECIMAL(5,2) instead of DECIMAL(10,2) like the DB column.
So I was overflowing the parameter in the procedure, not the column in the DB.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 9 years ago.
Can someone tell me why this C# code
item.Price = Convert.ToDouble(rdr["Ar"]);
gives me an error:
Cannot implicitly convert type 'double' to 'int'. An explicit conversion exists (are you missing a cast?)
The Price item is double, rdr is a SqlDataReader and Ar is a float type column of a table ... I thought that I should use float too in C# but I think that has other representation.
Can someone help me with this? I am trying to get some prices from the DB but it's not working. If you have any suggestions?
The problem is that item.Price is defined as an integer.
One possible answer is that the Convert.ToDouble() method is seeing your DBs float as a double, yet the expected parameter is an int (seeing as you wouldnt convert from double to double). Have you tried this?:
item.Price = (double)rdr["Ar"];
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
I came across this C# literal and was wondering what does it mean?
Especially, in the following case:
string.Format("{0:x}", byteArray[i]);
Thanks
It means format the first argument (index 0) as hexadecimal: http://msdn.microsoft.com/en-us/library/s8s7t687(v=vs.80).aspx
It means the first argument will be output as hexadecimal (in lowercase !!).
To output uppercase you could use "{0:X}".
Look msdn for more info about string formatting : MSDN Custom string format
This represents the hexadecimal format.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 11 years ago.
After reading these questions:
Code is behaving differently in Release vs Debug Mode
C# - Inconsistent math operation result on 32-bit and 64-bit
Double precision problems on .NET
Why does this floating-point calculation give different results on different machines?
I suspect that the reason my method for determining FPS which works while in Debug mode and no longer works in Release mode is because I'm using Long to hold time values. Here's the relevant code:
public void ActualFPS()
{
if (Stopwatch.GetTimestamp() >= lastTicks + Stopwatch.Frequency)
{
actualFPS = runsThisSecond;
lastTicks = Stopwatch.GetTimestamp();
runsThisSecond = 0;
}
}
runsThisSecond is incremented by one every time the method I'm tracing is called. Granted this isn't an overly accurate way to determine FPS, but it works for what I need it to.
lastTicks is a variable of type Long, and I believe that Stopwatch.GetTimestamp() is returned as a Long as well(?). Is this my problem? If so: any suggestions as to how to work around this?
EDIT: Stopwatch is using the High Resolution timer.
EDIT2: The problem has resolved itself. Without any changes to any of my code. At all. None. I have no idea what caused it to break, or to fix itself. Perhaps my computer decided to spontaneously consider my feelings?
You have a very accurate interval measurement available (gettimestamp - lastticks), but you are not using it all to compute the frame rate. You assume the interval is a second, it won't be. It will be more, by a random amount that's determined by how often you call ActualFPS(). In Release mode you'll call ActualFPS() more frequently so the error is less.
Divide runsThisSecond by (gettimestamp - lastticks) converted to seconds.
This question is unlikely to help any future visitors; it is only relevant to a small geographic area, a specific moment in time, or an extraordinarily narrow situation that is not generally applicable to the worldwide audience of the internet. For help making this question more broadly applicable, visit the help center.
Closed 10 years ago.
How do I check if the string reader has passed a certain line number, or has passed a line number which contains some text? I put this in the line processing code of a string reader:
if (currentline.Contains("123"))
currentbank = "123";
else if (currentline.Contains("456"))
currentbank = "456";
else if (currentline.Contains("789"))
currentbank = "789";
I want to change the contents of a string based on what range of line numbers it is in, with my code it always gives 123. Like for example if it's from lines 10-20 (or from 123 to 456) then the string should have 123, 20-30 (or 456 to 789) it should have 456 and 30-40 have 789. How can I do this using a StringReader?
Fixed it myself, problem being I used upper case (e.g. TEST) instead of lower case.