FLP10-J. Beware of precision loss when casting primitive integers to floating-point

Narrower primitive types may be converted to wider types without affecting the magnitude of numeric values. See JLS, Section 5.1.2, "Widening Primitive Conversion" for more information.

The following 19 specific conversions on primitive types are called the widening primitive conversions:

• byte to {{short, int, long, float, or double
• short to int, long, float, or double
• char to int, long, float, or double
• int to long, float, or double
• long to float or double
• float to double

Conversion from int or long to float, or long to double may lead to loss of precision (loss of least significant bits). In this case, the resulting floating-point value will be a
correctly rounded version of the integer value, using IEEE 754 round-to-nearest mode. No runtime exception occurs despite this loss.

Note that conversions from float to double can also lose information about the overall magnitude of the converted value. (See guideline FLP04-J. Use the strictfp modifier for floating point calculation consistency for additional information.)

Noncompliant Code Example

In this noncompliant code example, a value of type int is converted to the type float because of numeric promotions (see NUM10-J. Be aware of numeric promotion behavior). Numbers of type float have 23 mantissa bits, a sign bit, and an 8 bit exponent. The exponent allows type float to represent a larger range than that of type int. Nevertheless, integers whose representation requires more than 23 bits can only be represented approximately by a float. Consequently, the result of subtracting the original from this value is -46, not zero.

class WideSample {
  public static float addFloatToInt(int op1, float op2) {
    return op1 + op2;
  }

  public static void main(String[] args) {
    float result = addFloatToInt(1234567890, 1234567890);
    // This prints -46, and not 0 as may be expected
    System.out.println(result);  
  }

}

Compliant Solution (wider type)

This compliant solution replaces the float type double. Numbers of type double have 52 mantissa bits, a sign bit, and an 11 bit exponent. Consequently, integer values of type int and narrower can be converted to double without a loss of precision.

class WideSample {
  public static void main(String[] args) {
    int big = 1234567890;
                  
    // The significand can store at most 23 bits
    if ((big > 0x007fffff) || (big < -0x800000)) { 
      throw new ArithmeticException("Insufficient precision");	
    }

    float approx = big;
    System.out.println(big - (int)approx);  // Prints zero when no precision is lost
  }
}

Compliant Solution (ArithmeticException)

Numbers of type float The most significant bit of a float or double is its sign bit. The mantissa occupies the 23 least significant bits of a float and the 52 least significant bits of a double. The exponent, 8 bits in a float and 11 bits in a double, sits between the sign and mantissa. . The exponent allows type float to represent a larger range than that of type int. Nevertheless, integers whose representation requires more than 23 bits can only be represented approximately by a float.

class WideSample {
  public static void main(String[] args) {
    int big = 1234567890;
                  
    // The significand can store at most 23 bits
    if ((big > 0x007fffff) || (big < -0x800000)) { 
      throw new ArithmeticException("Insufficient precision");	
    }

    float approx = big;
    System.out.println(big - (int)approx);  // Prints zero when no precision is lost
  }
}

Risk Assessment

Casting integer values to floating-point types whose mantissa has fewer bits than the original integer value will lose precision.

Automated Detection

Automatic detection of casts that can lose precision is straightforward. Sound determination of whether those casts correctly reflect the intent of the programmer is infeasible in the general case. Heuristic warnings could be useful.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this guideline on the CERT website.

Related Guidelines

C Secure Coding Standard: FLP36-C. Beware of precision loss when converting integral types to floating point

C++ Secure Coding Standard: FLP36-CPP. Beware of precision loss when converting integral types to floating point

Bibliography

[[JLS 2005]] Section 5.1.2, "Widening Primitive Conversion"

INT02-J. Do not assume that the remainder operator always returns a non-negative result      03. Integers (INT)      INT04-J. Avoid using the char integral type to hold signed values