The C Standard, 7.12.1 [ISO/IEC 9899:2024], defines three types of errors that relate specifically to math functions in <math.h>.  Paragraph 2 states

A domain error occurs if an input

It is important to prevent or detect domain errors and range errors in the C99 math functions (as defined in math.h) before using the results in further computations.

...

argument is outside the domain over which the mathematical function is defined.

...

Many math errors can be prevented by carefully bounds-checking the arguments before calling functions, and taking alternative action if the bounds are violated. In particular, the following functions should be bounds checked as follows:

However, for some functions it is not practical to use bounds-checking to prevent all errors.  In the above pow example, the bounds check does not prevent the pow(10., 1e6) range error. In these cases detection must be used, either in addition to bounds checking or instead of bounds checking.

...

Paragraph 3 states

A pole error (also known as a singularity or infinitary) occurs if the mathematical function has an exact infinite result as the finite input argument(s) are approached in the limit.

Paragraph 4 states

arange error occurs if and only if the result overflows or underflows

An example of a domain error is the square root of a negative number, such as sqrt(-1.0), which has no meaning in real arithmetic. Contrastingly, 10 raised to the 1-millionth power, pow(10., 1e6), cannot be represented in many floating-point implementations because of the limited range of the type double and consequently constitutes a range error. In both cases, the function will return some value, but the value returned is not the correct result of the computation. An example of a pole error is log(0.0), which results in negative infinity.

Programmers can prevent domain and pole errors by carefully bounds-checking the arguments before calling mathematical functions and taking alternative action if the bounds are violated.

Range errors usually cannot be prevented because they are dependent on the implementation of floating-point numbers as well as on the function being applied. Instead of preventing range errors, programmers should attempt to detect them and take alternative action if a range error occurs.

The following table lists the double forms of standard mathematical functions, along with checks that should be performed to ensure a proper input domain, and indicates whether they can also result in range or pole errors, as reported by the C Standard. Both float and long double forms of these functions also exist but are omitted from the table for brevity. If a function has a specific domain over which it is defined, the programmer must check its input values. The programmer must also check for range errors where they might occur. The standard math functions not listed in this table, such as fabs(), have no domain restrictions and cannot result in range or pole errors.

Domain and Pole Checking

The most reliable way to handle domain and pole errors is to prevent them by checking arguments beforehand, as in the following exemplar:

double safe_sqrt(double x) {
  if (x < 0) {
    fprintf(stderr, "sqrt requires a nonnegative argument");
    /* Handle domain / pole error */
  }
  return sqrt (x);
}

Range Checking

Programmers usually cannot prevent range errors, so the most reliable way to handle them is to detect when they have occurred and act accordingly.

The exact treatment of error conditions from math functions is tedious. The C Standard, 7.12.1 paragraph 5 [ISO/IEC 9899:2024], defines the following behavior for floating-point overflow:

A floating result overflows if a finite result value with ordinary accuracy would have magnitude (absolute value) too large for the representation with full precision in the specified type. A result that is exactly an infinity does not overflow. If a floating result overflows and default rounding is in effect, then the function returns the value of the macro HUGE_VAL, HUGE_VALF, or HUGE_VALL according to the return type, with the same sign as the correct value of the function; however, for the types with reduced-precision representations of numbers beyond the overflow threshold, the function may return a representation of the result with less than full precision for the type. If a floating resultoverflowsanddefaultroundingisineffectandtheintegerexpressionmath_errhandling & MATH_ERRNO is nonzero, then the integer expression errno acquires the value ERANGE. If a floating result overflows, and the integer expression math_errhandling & MATH_ERREXCEPT is nonzero, the "overflow" floating-point exception is raised (regardless of whether default rounding is in effect).

It is preferable not to check for errors by comparing the returned value against HUGE_VAL or 0 for several reasons:

• These are, in general, valid (albeit unlikely) data values.
• Making such tests requires detailed knowledge of the various error returns for each math function.
• Multiple results aside from HUGE_VAL and 0 are possible, and programmers must know which are possible in each case.
• Different versions of the library have varied in their error-return behavior.

It can be unreliable to check for math errors using errno because an implementation might not set errno. For real functions, the programmer determines if the implementation sets errno by checking whether math_errhandling & MATH_ERRNO is nonzero. 

The C Standard, 7.3.2, paragraph 1 [ISO/IEC 9899:2024],  states:

 an implementation may set errno but is not required to.

The obsolete System V Interface Definition (SVID3) [UNIX 1992] provides more control over the treatment of errors in the math library. The programmer can define a function named matherr() that is invoked if errors occur in a math function. This function can print diagnostics, terminate the execution, or specify the desired return value. The matherr() function has not been adopted by C or POSIX, so it is not generally portable.

The following error-handing template uses C Standard functions for floating-point errors when the C macro math_errhandling is defined and indicates that they should be used; otherwise, it examines errno:

#include <math.h>
#include <fenv.h>
#include <errno.h>
 
/* ... */
/* Use to call a math function and check errors */
{
  #pragma STDC FENV_ACCESS ON

  if (math_errhandling & MATH_ERREXCEPT) {
    feclearexcept(FE_ALL_EXCEPT);
  }
  errno = 0;

  /* Call the math function */

  if ((math_errhandling & MATH_ERRNO) && errno != 0) {
    /* Handle range error */
  } else if ((math_errhandling & MATH_ERREXCEPT) &&
             fetestexcept(FE_INVALID | FE_DIVBYZERO |
                          FE_OVERFLOW | FE_UNDERFLOW) != 0) {
    /* Handle range error */
  }
}

See FLP03-C. Detect and handle floating-point errors for more details on how to detect floating-point errors.

Subnormal Numbers

A subnormal number is a nonzero number that does not use all of its precision bits [IEEE 754 2006]. These numbers can be used to represent values that are closer to 0 than the smallest normal number (one that uses all of its precision bits). However, the asin(), asinh(), atan(), atanh(), and erf() functions may produce range errors, specifically when passed a subnormal number. When evaluated with a subnormal number, these functions can produce an inexact, subnormal value, which is an underflow error.

The C Standard, 7.12.1, paragraph 6 [ISO/IEC 9899:2024], defines the following behavior for floating-point underflow:

The result underflows if a nonzero result value with ordinary accuracy would have magnitude (absolute value) less than the minimum normalized number in the type; however a zero result that is specified to be an exact zero does not underflow. Also, a result with ordinary accuracy and the magnitude of the minimum normalized number may underflow.269) If the result underflows, the function returns an implementation-defined value whose magnitude is no greater than the smallest normalized positive number in the specified type; if the integer expression math_errhandling & MATH_ERRNO is nonzero, whether errno acquires the value ERANGE is implementation-defined; if the integer expression math_errhandling & MATH_ERREXCEPT s nonzero, whether the"underflow" floating-point exception is raised is implementation-defined. 

Implementations that support floating-point arithmetic but do not support subnormal numbers, such as IBM S/360 hex floating-point or nonconforming IEEE-754 implementations that skip subnormals (or support them by flushing them to zero), can return a range error when calling one of the following families of functions with the following arguments:

• fmod((min+subnorm), min)
• remainder((min+subnorm), min)
• remquo((min+subnorm), min, quo)

where min is the minimum value for the corresponding floating point type and subnorm is a subnormal value.

If Annex F is supported and subnormal results are supported, the returned value is exact and a range error cannot occur. The C Standard, F.10.7.1 paragraph 2 [ISO/IEC 9899:2024], specifies the following for the fmod(), remainder(), and remquo() functions:

When subnormal results are supported, the returned value is exact and is independent of the current rounding direction mode.

Annex F, subclause F.10.7.2, paragraph 2, and subclause F.10.7.3, paragraph 2, of the C Standard identify when subnormal results are supported.

Noncompliant Code Example (sqrt())

This noncompliant code example determines the square root of x:

#include <math.h>
 
void func(double x) {
  double result;
  result = sqrt(x);
}

However, this code may produce a domain error if x is negative.

Compliant Solution (sqrt())

Because this function has domain errors but no range errors, bounds checking can be used to prevent domain errors:

#include <math.h>
 
void func(double x) {
  double result;

  if (isless(x, 0.0)) {
    /* Handle domain error */
  }

  result = sqrt(x);
}

Noncompliant Code Example (sinh(), Range Errors)

This noncompliant code example determines the hyperbolic sine of x:

#include <math.h>
 
void func(double x) {
  double result;
  result = sinh(x);
}

This code may produce a range error if x has a very large magnitude.

Compliant Solution (sinh(), Range Errors)

Because this function has no domain errors but may have range errors, the programmer must detect a range error and act accordingly:

#include <math.h>
#include <fenv.h>
#include <errno.h>
 
void func(double x) { 
  double result;
  {
    #pragma STDC FENV_ACCESS ON
    if (math_errhandling & MATH_ERREXCEPT) {
      feclearexcept(FE_ALL_EXCEPT);
    }
    errno = 0;

    result = sinh(x);

    if ((math_errhandling & MATH_ERRNO) && errno != 0) {
      /* Handle range error */
    } else if ((math_errhandling & MATH_ERREXCEPT) &&
               fetestexcept(FE_INVALID | FE_DIVBYZERO |
                            FE_OVERFLOW | FE_UNDERFLOW) != 0) {
      /* Handle range error */
    }
  }
 
  /* Use result... */
}

Noncompliant Code Example (pow())

This noncompliant code example raises x to the power of y:

#include <math.h>
 
void func(double x, double y) {
  double result;
  result = pow(x, y);
}

This code may produce a domain error if x is negative and y is not an integer value or if x is 0 and y is 0. A domain error or pole error may occur if x is 0 and y is negative, and a range error may occur if the result cannot be represented as a double.

Compliant Solution (pow())

Because the pow() function can produce domain errors, pole errors, and range errors, the programmer must first check that x and y lie within the proper domain and do not generate a pole error and then detect whether a range error occurs and act accordingly:

#include <math.h>
#include <fenv.h>
#include <errno.h>
 
void func(double x, double y) {
  double result;

  if (((x == 0.0f) && islessequal(y, 0.0)) || isless(x, 0.0)) {
    /* Handle domain or pole error */
  }

  {
    #pragma STDC FENV_ACCESS ON
    if (math_errhandling & MATH_ERREXCEPT) {
      feclearexcept(FE_ALL_EXCEPT);
    }
    errno = 0;

    result = pow(x, y);
 
    if ((math_errhandling & MATH_ERRNO) && errno != 0) {
      /* Handle range error */
    } else if ((math_errhandling & MATH_ERREXCEPT) &&
               fetestexcept(FE_INVALID | FE_DIVBYZERO |
                            FE_OVERFLOW | FE_UNDERFLOW) != 0) {
      /* Handle range error */
    }
  }

  /* Use result... */
}

Noncompliant Code Example (asin(), Subnormal Number)

This noncompliant code example determines the inverse sine of x:

#include <math.h>
 
void func(float x) {
  float result = asin(x);
  /* ... */
}

Compliant Solution (asin(), Subnormal Number)

Because this function has no domain errors but may have range errors, the programmer must detect a range error and act accordingly:

#include <math.h>
#include <fenv.h>
#include <errno.h>
void func(float x) { 
  float result;

  {
    #pragma STDC FENV_ACCESS ON
    if (math_errhandling & MATH_ERREXCEPT) {
      feclearexcept(FE_ALL_EXCEPT);
    }
    errno = 0;

    result = asin(x);

    if ((math_errhandling & MATH_ERRNO) && errno != 0) {
      /* Handle range error */
    } else if ((math_errhandling & MATH_ERREXCEPT) &&
               fetestexcept(FE_INVALID | FE_DIVBYZERO |
                            FE_OVERFLOW | FE_UNDERFLOW) != 0) {
      /* Handle range error */
    }
  }

  /* Use result... */
}

Risk Assessment

Failure to prevent or detect domain and range errors in math functions may cause unexpected results.

Automated Detection

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Related Guidelines

Key here (explains table format and definitions)

CERT-CWE Mapping Notes

Key here for mapping notes

CWE-391 and FLP32-C

Intersection( CWE-391, FLP32-C) =

• Failure to detect range errors in floating-point calculations

CWE-391 - FLP32-C

• Failure to detect errors in functions besides floating-point calculations

FLP32-C – CWE-391 =

• Failure to detect domain errors in floating-point calculations

CWE-682 and FLP32-C

Independent( INT34-C, FLP32-C, INT33-C) CWE-682 = Union( FLP32-C, list) where list =

• Incorrect calculations that do not involve floating-point range errors

Bibliography

...

Image Added Image Added Image Added

...

acos( x ), asin( x )

Non-Compliant Code Example

The following non-compliant code computes the arc cosine of the variable x

double x, result;

/* Set the value for x */

result = acos(x);

However, this code may produce a domain error if {{x}} is not in the range \[-1, \+1\].

Compliant Solution

The following compliant solution uses bounds checking to ensure that there is not a domain error.

double x, result;

/* Set the value for x */

if ( islessequal(x,-1) || isgreaterequal(x, 1) ){
     /* handle domain error */
}

result = acos(x);

...

atan2( y, x )

Non-Compliant Code Example

The following non-compliant code computes the arc tangent of the two variables x and y.

double x, y, result;

/* Set the value for x and y */

result = atan2(y, x);

However, this code may produce a domain error if both x and y are zero.

Compliant Solution

The following compliant solution tests the arguments to ensure that there is not a domain error.

double x, y, result;

/* Set the value for x and y */

if ( (x == 0.f) && (y == 0.f) ) {
     /* handle domain error */
}

result = atan2(y, x);

...

log( x ), log10( x )

Non-Compliant Code Example

The following non-compliant code determines the natural logarithm of x.

double result, x;

/* Set the value for x */

result = log(x);

However, this code may produce a domain error if x is negative and a range error if x is zero.

Compliant Solution

The following compliant solution tests the suspect arguments to ensure that no domain or range errors are raised.

double result, x;

/* Set the value for x */

if (islessequal(x, 0)) {
  /* handle domain and range errors */
}

result = log(x);

...

pow( x, y )

Non-Compliant Code Example

The following non-compliant code raises x to the {{y}}th power.

double x, y, result;

result = pow(x, y);

However, this code may produce a domain error if x is negative and y is not an integer, or if x is zero and y is zero. A domain error or range error may occur if x is zero and y is negative, and a range error may occur if the result cannot be represented as a double.

Non-Compliant Code Example

This code only performs bounds-checking on x and y. It prevents domain errors and some range errors, but does not prevent range errors where the result cannot be represented as a double (see the Error Checking section below regarding ways to mitigate the effects of a range error).

double x, y, result;

if (((x == 0.f) && islessequal(y, 0)) ||
    (isless(x, 0) && !isInteger(y))) {
  /* handle domain and range errors */
}

result = pow(x, y);

...

sqrt( x )

Non-Compliant Code Example

The following non-compliant code determines the square root of x

double x, result;

result = sqrt(x);

However, this code may produce a domain error if x is negative.

Compliant Solution

The following compliant solution tests the suspect argument to ensure that no domain error is raised.

double x, result;

if (isless(x, 0)){
  /* handle domain error */
}

result = sqrt(x);

Non-Compliant Coding Example (Error Checking)

The exact treatment of error conditions from math functions is quite complicated (see C99 Section 7.12.1, "Treatment of error conditions").

The {{sqrt()}} function returns zero when given a negative argument. The {{pow()}} function returns a very large number (appropriately positive or negative) on an overflow range error.   This very large number is defined in {{<math.h>}} as described by C99 \[[ISO/IEC 9899-1999|AA. C References#ISO/IEC 9899-1999]\]:

The macro

Code Block
HUGE_VAL

expands to a positive double constant expression, not necessarily representable as a float. The macros

Code Block
HUGE_VALF HUGE_VALL

are respectively float and long double analogs of HUGE_VAL.

It is best not to check for errors by comparing the returned value against HUGE_VAL or zero for several reasons. First of all, these are in general valid (albeit unlikely) data values. Secondly, making such tests requires detailed knowledge of the various error returns for each math function. Also, there are three different possibilities, -HUGE_VAL, 0, and HUGE_VAL, and you must know which are possible in each case. Finally, different versions of the library have differed in their error-return behavior.

It is also difficult to check for math errors using errno because an implementation might not set it.  For real functions, the programmer can test whether the implementation sets errno because in that case math_errhandling & MATH_ERRNO is nonzero.  For complex functions, the C Standard Section 7.3.2 simply states that setting errno is optional.

Compliant Solution (Error Checking)

The most reliable way to test for errors is by checking arguments beforehand, as in the following compliant solution:

if (/* arguments will cause a domain or range error */) {
  /* handle the error */
}
else {
  /* perform computation */
}

For functions where argument validation is difficult, including pow(), erfc(), lgamma(), and tgamma(), one can employ the following approach.

#include <math.h>
#if defined(math_errhandling) && (math_errhandling & MATH_ERREXCEPT)
#include <fenv.h>
#endif

/* ... */

#if defined(math_errhandling) && (math_errhandling & MATH_ERREXCEPT)
  feclearexcept(FE_ALL_EXCEPT);
#endif
errno = 0;

/* call the function */

#if !defined(math_errhandling) || (math_errhandling & MATH_ERRNO)
if (errno != 0) {
  /* handle error */
}
#endif
#if defined(math_errhandling) && (math_errhandling & MATH_ERREXCEPT)
if (fetestexcept(FE_INVALID | FE_DIVBYZERO | FE_OVERFLOW) != 0) {
  /* handle error */
}
#endif

Implementation Details

System V Interface Definition, Third Edition

The System V Interface Definition, Third Edition (SVID3) provides more control over the treatment of errors in the math library. The user can provide a function named matherr that is invoked if errors occur in a math function. This function could print diagnostics, terminate the execution, or specify the desired return-value. The matherr() function has not been adopted by the C standard, so its use is not generally portable.

Risk Assessment

Failure to properly verify arguments supplied to math functions may result in unexpected results.

Automated Detection

Fortify SCA Version 5.0 with CERT C Rule Pack can detect violations of this rule.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

\[[ISO/IEC 9899-1999|AA. C References#ISO/IEC 9899-1999]\] Section 7.3, "Complex arithmetic <complex.h>", and Section 7.12, "Mathematics <math.h>"
\[[Plum 85|AA. C References#Plum 85]\] Rule 2-2
\[[Plum 89|AA. C References#Plum 91]\] Topic 2.10, "conv - conversions and overflow"

FLP31-C. Do not call functions expecting real values with complex values      05. Floating Point (FLP)       FLP33-C. Convert integers to floating point for floating point operations