﻿ Oddities of Computer Math in Java

# Oddities of Computer Math in Java

## Here Be Dragons

### Fixed Point Oddities

Example of post-increment side effect: int i = 1; i = i++ + i + i; System.output.print( "i = " + i ); Output: i = 5
Example of overflow: byte b = 100; b += 100; System.output.print( "b = " + b ); Output: b = -56
Example of conversion error: byte b = -100; char c = (char) b; int i = c; System.out.print( "b=" + b + ", c=" + c + "i=" + i ); Output: b=-100, c=ﾜ, i=65436 (FYI: The letter is Japanese Katakana)
Example of division: int i = 9 / 10; System.output.print( "i = " + i ); Output: i = 0

### Floating Point Oddities

Example of round-off error: double d1 = 0.1, d2 = 0.3, d3; d3 = d1 + d1 + d1; if ( d2 == d3 ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d3 - d2 = 5.551115123125783E-17 To work around this use fixed point, the "BigDecimal" class, or code like this: final double EPSILON = 1E-14; ... if ( d2 >= d3 - EPSILON && d2 <= d3 + EPSILON ) System.out.print( "d2 equals d3" ); else System.out.print( "d3 - d2 = " + (d3 - d2) ); Output: d2 equals d3 The similar looking test below isn't quite the same and should be avoided: if ( Math.abs(d3 - d2) < EPSILON )
Examples of loss of precision: final float EPSILON = 0.1f; float f1 = 50000000.0f; float f2 = f1 + 2.0f; if ( f1 >= f2 - EPSILON && f1 <= f2 + EPSILON ) System.out.print( "f1 equals f2" ); else System.out.print( "f2 - f1 = " + (f2 - f1) ); Output: f1 equals f2 float f3 = f1 + 5.0f + 5.0f + 5.0f + 5.0f; float f4 = 5.0f + 5.0f + 5.0f + 5.0f + f1; System.out.printf( "f3 = %,f, f4 = %,f\n", f3, f4 ); Output: f3 = 50,000,016.0, f4 = 50,000,020.0
Logic errors that look like compiler errors: int a = 1; a += 0.2; System.out.println( "a == 1 + 0.2 ==> " + (a == 1 + 0.2) ); System.out.println( "1/10f == 1/10.0 ==> " + (1/10f == 1/10.0) ); System.out.println( "1/10f == 1/10 ==> " + (1/10f == 1/10) ); System.out.println( "1/2f == 1/2.0 ==> " + (1/2f == 1/2.0) ); Output: a == 1 + 0.2 ==> false 1/10f == 1/10.0 ==> false 1/10f == 1/10 ==> false 1/2f == 1/2.0 ==> true
Example of overflow: double d1 = 1.6e308; double d2 = d1 * 2.0 / 2.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); Output: d1 = 1.6E308 d2 = Infinity
Examples of division by zero: double d1 = 0.0 / 0.0; double d2 = 0.0 / 0.0; double d3 = d1; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d1 == d2 is " + (d1 == d2) ); System.out.println( "d1 == d3 is " + (d1 == d3) ); int i = 1 / 0; Output: d1 = NaN d2 = NaN d3 = NaN d1 == d2 is false d1 == d3 is false java.lang.ArithmeticException: / by zero at MathOddities.addContent(MathOddities.java:177) at MathOddities.init(MathOddities.java:22) at sun.applet.AppletPanel.run(AppletPanel.java:424) at java.lang.Thread.run(Thread.java:619)
Examples of Negative zero, infinity, and Not a Number: double d1 = 1.0 / 0.0; double d2 = -1.0 / 0.0; double d3 = 0.0 / 1.0; double d4 = 0.0 / -1.0; double d5 = -0.0 / 0.0; System.out.println( "d1 = " + d1 ); System.out.println( "d2 = " + d2 ); System.out.println( "d3 = " + d3 ); System.out.println( "d4 = " + d4 ); System.out.println( d3 == d4 ); System.out.println( "d5 = " + d5 ); System.out.println( "d5 == d5: " + (d5 == d5) ); System.out.println( "Double.isNaN( d5 ) = " + Double.isNaN( d5 ) ); Output: d1 = Infinity d2 = -Infinity d3 = 0.0 d4 = -0.0 true d5 = NaN d5 == d5: false Double.isNaN( d5 ) = true

When comparing very large floating-point numbers, if the difference is less than the number of significant digits, you usually consider the two values equal regardless of the value assigned to EPSILON.  For example, 60_000_000.0f and 60_000_003.0f should usually compare equal, even though the difference is greater than (say) an EPSILON value of 0.05.  In other words, the test using `abs` is only testing if the round-off error is significant, whereas the first (correct) test shown above is testing if two floating-point numbers should be considered equal, to the limit of the precision or to the value of EPSILON, whichever is greater.