I have a question regarding why .NET calculates something strange. We encountered this by accident. Consider the two below lines of code:
int
iOK = (int)(16383.51 * 100);int iNotOK = (int)(16384.51 * 100);
When this is compiled and run, the variable iOK will hold the value 1638351. The value iNokOK will hold the value 1638450 - note the 0 in the end when 1 was expected.
I tried different values for some time, and it breaks just at that number 16384, which is nowhere near the limits of int.
How can this happen
.NET calculations around the number 16384.51
Marcela Moreno
int iOK = (int)(16383.51m * 100);
int iNotOK = (int)(16384.51m * 100);
ismaelc
16384 is a power of 2, perhaps that has something to do with it
A. Weisberg
Not so much ‘default behavior’ as it is ‘natural properties of integers and floats’.
Mr Pilley
Bart371
This is a floating point precision problem.
The compiler most likely evaluates the expression 16384.51 * 100 as doubles. The result of this is expression is 1638450.9999999998. Casting this to an int results in 1638450.
One possible solution is as followed
int rightValue = (int)Math.Round (16384.51 * 100);
RJMPhD
Shajeel
thak you for your reply, I feel a bit uneducated when a waste the other time asking basic things I coul find in manuals, sorry, it's not my abit.
Btw, I don't agree with you that is a "natural properties". It depends by the semantic choosen by the language designer.
It's more common to observer the truncating which, for performance issue, is better (C, C++, Java, C#, etc... essentially alla C-derived languages) but other language adopt a "default behaviour" different like Pascal (if I right recall).
I posted that just because sometimes I like to be publicly silly :).
Bye guys