Oddities of Computer Math in Java

Here Be Dragons

[pic of dragon] [pic of dragon]

Fixed Point Oddities

Example of post-increment side effect: int i = 1; i = i++ + i + i; System.output.print( "i = " + i ); Output: i = 5
Example of overflow: byte b = 100; b += 100; System.output.print( "b = " + b ); Output: b = -56
Example of conversion error: byte b = -100; char c = (char) b; int i = c; System.out.print( "b=" + b + ", c=" + c + "i=" + i ); Output: b=-100, c=ワ, i=65436 (FYI: The letter is Japanese Katakana)
Example of division: int i = 9 / 10; System.output.print( "i = " + i ); Output: i = 0

Floating Point Oddities

Example of round-off error: double d1 = 0.1, d2 = 0.3, d3; d3 = d1 + d1 + d1; if ( d2 == d3 ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d3 - d2 = 5.551115123125783E-17 To work around this use fixed point, the "BigDecimal" class, or code like this: final double EPSILON = 1E-14; ... if ( d2 >= d3 - EPSILON && d2 <= d3 + EPSILON ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d2 equals d3 The similar looking test below isn't quite the same and should be avoided: if ( Math.abs(d3 - d2) < EPSILON )
Examples of loss of precision: final float EPSILON = 0.1f; float f1 = 50000000.0f; float f2 = f1 + 2.0f; if ( f1 >= f2 - EPSILON && f1 <= f2 + EPSILON ) System.out.print( "f1 equals f2" ); else System.out.print( "f2 - f1 = " + (f2 - f1) ); Output: f1 equals f2 float f3 = f1 + 5.0f + 5.0f + 5.0f + 5.0f; float f4 = 5.0f + 5.0f + 5.0f + 5.0f + f1; System.out.printf( "f3 = %,f, f4 = %,f\n", f3, f4 ); Output: f3 = 50,000,016.0, f4 = 50,000,020.0
Logic errors that look like compiler errors: int a = 1; a += 0.2; System.out.println( "a == 1 + 0.2 ==> " + (a == 1 + 0.2) ); System.out.println( "1/10f == 1/10.0 ==> " + (1/10f == 1/10.0) ); System.out.println( "1/10f == 1/10 ==> " + (1/10f == 1/10) ); System.out.println( "1/2f == 1/2.0 ==> " + (1/2f == 1/2.0) ); Output: a == 1 + 0.2 ==> false 1/10f == 1/10.0 ==> false 1/10f == 1/10 ==> false 1/2f == 1/2.0 ==> true
Example of overflow: double d1 = 1.6e308; double d2 = d1 * 2.0 / 2.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); Output: d1 = 1.6E308 d2 = Infinity
Examples of division by zero: double d1 = 0.0 / 0.0; double d2 = 0.0 / 0.0; double d3 = d1; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d1 == d2 is " + (d1 == d2) ); System.out.println( "d1 == d3 is " + (d1 == d3) ); int i = 1 / 0; Output: d1 = NaN d2 = NaN d3 = NaN d1 == d2 is false d1 == d3 is false java.lang.ArithmeticException: / by zero at MathOddities.addContent(MathOddities.java:177) at MathOddities.init(MathOddities.java:22) at sun.applet.AppletPanel.run(AppletPanel.java:424) at java.lang.Thread.run(Thread.java:619)
Examples of Negative zero, infinity, and Not a Number: double d1 = 1.0 / 0.0; double d2 = -1.0 / 0.0; double d3 = 0.0 / 1.0; double d4 = 0.0 / -1.0; double d5 = -0.0 / 0.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d4 = " + d4 ); System.out.println( d3 == d4 ); System.out.println( "d5 = " + d5 ); System.out.println( "d5 == d5: " + (d5 == d5) ); System.out.println( "Double.isNaN( d5 ) = " + Double.isNaN( d5 ) ); Output: d1 = Infinity d2 = -Infinity d3 = 0.0 d4 = -0.0 true d5 = NaN d5 == d5: false Double.isNaN( d5 ) = true
Example of BigDecimal error: BigDecimal zero1 = new BigDecimal("0"); BigDecimal zero2 = new BigDecimal("0.0"); BigDecimal zero3 = zero1.stripTrailingZeros(); System.out.println(zero1.equals(zero2)); System.out.println(zero1.compareTo(zero2)); System.out.println(zero1.equals(zero3)); Output: false true true

When comparing very large floating-point numbers, if the difference is less than the number of significant digits, you usually consider the two values equal regardless of the value assigned to EPSILON.  For example, 60_000_000.0f and 60_000_003.0f should usually compare equal, even though the difference is greater than (say) an EPSILON value of 0.05.  In other words, the test using abs is only testing if the round-off error is significant, whereas the first (correct) test shown above is testing if two floating-point numbers should be considered equal, to the limit of the precision or to the value of EPSILON, whichever is greater.

See also What Every Computer Scientist Should Know About Floating-Point Arithmetic

xkcd.com cartoon about overflow

Note that this page does not describe all the weird ways computer floating point math differs from real number math.  Certain math operations may take hundreds of times longer for some inputs than others. (See subnormal numbers.)  Not only is performance terrible for subnormals, but how they are handled differs by hardware and compiler; in some cases, these are simply rounded to zero (known as FTZ).  There's only about 8 million (224) distinct 32-bit float values, and they are not evenly spaced.  (The situation is similar for 64-bit doubles.)  So trying to generate a random number in a range is especially difficult to do if you want a uniform distribution of values.  And so on.

When developing software that uses floating point math, as in the fields of chemistry, physics, engineering, gambling, gaming, modeling, and others, you must become knowledgeable if you plan on using floating point types!