Due to the nature of floating path arithmetic, almost all floating point arithmetic is imprecise. The computer can only maintain a finite number of digits. As a result, it is impossible to precisely represent repeating binary-representation values, such as 1/3 or 1/5.
When precise computations are necessary, consider alternative representations that may be able to completely represent your values. For example, if you are performing arithmetic on decimal values and need an exact rounding mode based on decimal values, represent your values in binary decimal instead of using floating point.
When precise computation is necessary, carefully and methodically evaluate the cumulative error of the computations, regardless of whether decimal or binary is used, to ensure that the resulting error is within tolerances. Consider using numerical analysis to properly understand the numerical properties of the problem. A useful introduction can be found in Goldberg 91.
Risk Analysis
Using an alternative representation besides floating point may allow for more precision and accuracy for critical arithmetic.
Recommendation |
Severity |
Likelihood |
Remediation Cost |
Priority |
Level |
|---|---|---|---|---|---|
FLP00-A |
1 (low) |
2 (probable) |
2 (medium) |
P4 |
L3 |
Search for vulnerabilities resulting from the violation of this rule on the CERT website.
References
[[IEEE 754 2006]]
[[ISO/IEC JTC1/SC22/WG11]]
[[Goldberg 91]]
05. Floating Point (FLP) 05. Floating Point (FLP) FLP01-A. Take care in rearranging floating point expressions