Oddities of Computer Math in Java

Here Be Dragons

[pic of dragon] [pic of dragon]

Fixed Point Oddities

Example of post-increment side effect: int i = 1; i = i++ + i + i; System.output.print( "i = " + i ); Output: i = 5
Example of overflow: byte b = 100; b += 100; System.output.print( "b = " + b ); Output: b = -56
Example of conversion error: byte b = -100; char c = (char) b; int i = c; System.out.print( "b=" + b + ", c=" + c + "i=" + i ); Output: b=-100, c=ワ, i=65436 (FYI: The letter is Japanese Katakana)
Example of division: int i = 9 / 10; System.output.print( "i = " + i ); Output: i = 0

Floating Point Oddities

Example of round-off error: double d1 = 0.1, d2 = 0.3, d3; d3 = d1 + d1 + d1; if ( d2 == d3 ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d3 - d2 = 5.551115123125783E-17 To work around this use fixed point, the "BigDecimal" class, or code like this: final double EPSILON = 1E-14; ... if ( d2 >= d3 - EPSILON && d2 <= d3 + EPSILON ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d2 equals d3 The similar looking test below isn't quite the same and should be avoided: if ( Math.abs(d3 - d2) < EPSILON )
Examples of loss of precision: final float EPSILON = 0.1f; float f1 = 50000000.0f; float f2 = f1 + 2.0f; if ( f1 >= f2 - EPSILON && f1 <= f2 + EPSILON ) System.out.print( "f1 equals f2" ); else System.out.print( "f2 - f1 = " + (f2 - f1) ); Output: f1 equals f2 float f3 = f1 + 5.0f + 5.0f + 5.0f + 5.0f; float f4 = 5.0f + 5.0f + 5.0f + 5.0f + f1; System.out.printf( "f3 = %,f, f4 = %,f\n", f3, f4 ); Output: f3 = 50,000,016.0, f4 = 50,000,020.0
Logic errors that look like compiler errors: int a = 1; a += 0.2; System.out.println( "a == 1 + 0.2 ==> " + (a == 1 + 0.2) ); System.out.println( "1/10f == 1/10.0 ==> " + (1/10f == 1/10.0) ); System.out.println( "1/10f == 1/10 ==> " + (1/10f == 1/10) ); System.out.println( "1/2f == 1/2.0 ==> " + (1/2f == 1/2.0) ); Output: a == 1 + 0.2 ==> false 1/10f == 1/10.0 ==> false 1/10f == 1/10 ==> false 1/2f == 1/2.0 ==> true
Example of overflow: double d1 = 1.6e308; double d2 = d1 * 2.0 / 2.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); Output: d1 = 1.6E308 d2 = Infinity
Examples of division by zero: double d1 = 0.0 / 0.0; double d2 = 0.0 / 0.0; double d3 = d1; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d1 == d2 is " + (d1 == d2) ); System.out.println( "d1 == d3 is " + (d1 == d3) ); int i = 1 / 0; Output: d1 = NaN d2 = NaN d3 = NaN d1 == d2 is false d1 == d3 is false java.lang.ArithmeticException: / by zero at MathOddities.addContent(MathOddities.java:177) at MathOddities.init(MathOddities.java:22) at sun.applet.AppletPanel.run(AppletPanel.java:424) at java.lang.Thread.run(Thread.java:619)
Examples of Negative zero, infinity, and Not a Number: double d1 = 1.0 / 0.0; double d2 = -1.0 / 0.0; double d3 = 0.0 / 1.0; double d4 = 0.0 / -1.0; double d5 = -0.0 / 0.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d4 = " + d4 ); System.out.println( d3 == d4 ); System.out.println( "d5 = " + d5 ); System.out.println( "d5 == d5: " + (d5 == d5) ); System.out.println( "Double.isNaN( d5 ) = " + Double.isNaN( d5 ) ); Output: d1 = Infinity d2 = -Infinity d3 = 0.0 d4 = -0.0 true d5 = NaN d5 == d5: false Double.isNaN( d5 ) = true

When comparing very large floating-point numbers, if the difference is less than the number of significant digits, you usually consider the two values equal regardless of the value assigned to EPSILON.  For example, 60_000_000.0f and 60_000_003.0f should usually compare equal, even though the difference is greater than (say) an EPSILON value of 0.05.  In other words, the test using abs is only testing if the round-off error is significant, whereas the first (correct) test shown above is testing if two floating-point numbers should be considered equal, to the limit of the precision or to the value of EPSILON, whichever is greater.

See also What Every Computer Scientist Should Know About Floating-Point Arithmetic

xkcd.com cartoon about overflow