FLP05-J. Do not attempt comparisons with NaN

According to the Java Language Specification [[JLS 2005]], Section 4.2.3, "Floating-Point Types, Formats, and Values"

NaN is unordered, so the numerical comparison operators <, <=, >, and >= return false if either or both operands are NaN. The equality operator == returns false if either operand is NaN, and the inequality operator != returns true if either operand is NaN.

Problems can arise when the programmer uses such operators on NaN values in comparison operations. There is also a possibility that the input validation condition does not expect a NaN value as input.

Noncompliant Code Example

This noncompliant code example attempts a direct comparison with NaN. As per the semantics of NaN, all comparisons with NaN yield false (with the exception of the != operator, which returns true). Consequently, the comparison must always return false, and the "Both are equal" message is never printed.

public class NaNComparison {
  public static void main(String[] args) {
    double x = 0.0;
    double result = Math.cos(1/x); // returns NaN if input is infinity
    if (result == Double.NaN) { // comparison is always false!
      System.out.println("Both are equal");
    }
  }
}

Compliant Solution

This compliant solution uses the method Double.isNaN() to check whether the expression corresponds to a NaN value.

public class NaNComparison {
  public static void main(String[] args) {
    double x = 0.0;	  
    double result = Math.cos(1/x); // returns NaN when input is infinity
    if (Double.isNaN(result)) { 
      System.out.println("Both are equal");
    }
  }
}

Risk Assessment

Comparisons with NaN values may lead to unexpected results.

Automated Detection

Automated detection of floating point comparison operators is straightforward. Sound determination of whether the possibility of an unordered result has been correctly handled is not feasible in the general case. Heuristic checks may be useful.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this guideline on the CERT website.

Bibliography

[[FindBugs 2008]] FE: Doomed test for equality to NaN
[[JLS 2005]] Section 4.2.3, Floating-Point Types, Formats, and Values

FLP04-J. Use the strictfp modifier for floating point calculation consistency      07. Floating Point (FLP)      FLP06-J. Check floating point inputs for exceptional values