Signed integer overflow is undefined behavior (see undefined behavior 36 in Annex J of the C Standard). This means that implementations have a great deal of latitude in how they deal with signed integer overflow.

An implementation may define the same modulo arithmetic for both unsigned and signed integers. On such an implementation, signed integers overflow by wrapping around to 0. An example of such an implementation is GNU GCC invoked with the -fwrapv command-line option.

Other implementations may cause a hardware trap (also called an exceptional condition) to be generated when a signed integer overflows. On such implementations, a program that causes a signed integer to overflow will most likely abnormally exit. On a UNIX system, the result of such an event may be a signal sent to the process. An example of such an implementation is GNU GCC invoked with the -ftrapv command-line option.

Still other implementations may simply assume that signed integers never overflow and may generate object code accordingly. An example of such an implementation is GNU GCC invoked without either the -fwrapv or the -ftrapv option.

It is also possible for the same conforming implementation to emit code that exhibits different behavior in different contexts. For example, an implementation may determine that a signed integer loop control variable declared in a local scope cannot overflow and may emit efficient code on the basis of that determination, while the same implementation may avoid making that assumption in another function when the variable is a global object.

For these reasons, it is important to ensure that operations on signed integers do no result in overflow. (See MSC15-C. Do not depend on undefined behavior.) Of particular importance are operations on signed integer values that originate from untrusted sources and are used in any of the following ways:

• As an array index
• In any pointer arithmetic
• As a length or size of an object
• As the bound of an array (for example, a loop counter)
• As an argument to a memory allocation function
• In security-critical code

Most integer operations can result in overflow if the resulting value cannot be represented by the underlying representation of the integer. The following table indicates which operators can result in overflow:

The following sections examine specific operations that are susceptible to integer overflow. When operating on small integer types (smaller than int), integer promotions are applied. The usual arithmetic conversions may also be applied to (implicitly) convert operands to equivalent types before arithmetic operations are performed. Programmers should understand integer conversion rules before trying to implement secure arithmetic operations. (See INT02-C. Understand integer conversion rules.)

Addition

Addition is between two operands of arithmetic type or between a pointer to an object type and an integer type. (See ARR37-C. Do not add or subtract an integer to a pointer to a non-array object and ARR38-C. Do not add or subtract an integer to a pointer if the resulting value does not refer to a valid array element.) Incrementing is equivalent to adding 1.

Noncompliant Code Example

This noncompliant code example may result in a signed integer overflow during the addition of the signed operands si_a and si_b. If this behavior is unanticipated, it can lead to an exploitable vulnerability.

int si_a;
int si_b;
int sum;

void func(void) {
  /* Initialize si_a and si_b */
  sum = si_a + si_b;

  /* ... */
}

Compliant Solution

This compliant solution tests the suspect addition operation to ensure no overflow occurs regardless of representation:

#include <limits.h>
 
signed int si_a;
signed int si_b;
signed int sum;

void (func) {
  /* Initialize si_a, si_b and sum */

  if (((si_b > 0) && (si_a > (INT_MAX - si_b))) ||
      ((si_b < 0) && (si_a < (INT_MIN - si_b)))) {
    /* Handle error condition */
  } else {
    sum = si_a + si_b;
  }
  /* ... */
}

Subtraction

Subtraction is between two operands of arithmetic type, two pointers to qualified or unqualified versions of compatible object types, or a pointer to an object type and an integer type. See ARR36-C. Do not subtract or compare two pointers that do not refer to the same array, ARR37-C. Do not add or subtract an integer to a pointer to a non-array object, and ARR38-C. Do not add or subtract an integer to a pointer if the resulting value does not refer to a valid array element for information about pointer subtraction. Decrementing is equivalent to subtracting 1.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the subtraction of the signed operands si_a and si_b. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner that can lead to an exploitable vulnerability.

signed int si_a;
signed int si_b;
signed int result;

void func(void) {
  /* Initialize si_a and si_b */

  result = si_a - si_b;

  /* ... */
}

Compliant Solution

This compliant solution tests the operands of the subtraction to guarantee there is no possibility of signed overflow, regardless of representation:

#include <limits.h>
 
signed int si_a;
signed int si_b;
singed int result;

void func(void) {
  /* Initialize si_a, si_b and result */

  if ((si_b > 0 && si_a < INT_MIN + si_b) ||
      (si_b < 0 && si_a > INT_MAX + si_b)) {
    /* Handle error condition */
  } else {
    result = si_a - si_b;
  }

  /* ... */
}

Multiplication

Multiplication is between two operands of arithmetic type.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the multiplication of the signed operands si_a and si_b. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner that can lead to an exploitable vulnerability.

signed int si_a;
signed int si_b;
signed int result;

void func(void) {
  /* Initialize si_a and si_b */
  result = si_a * si_b;

  /* ... */
}

Compliant Solution

This compliant solution guarantees there is no possibility of signed overflow on systems where long long is at least twice the width of int:

#include <assert.h>
 
signed int si_a;
signed int si_b;
signed int result;

void func(void) {
  /* Initialize si_a, si_b and result */
  assert( UWIDTH( signed long long, ULLONG_MAX) >=
          2 * UWIDTH( int, UINT_MAX));
  signed long long tmp = (signed long long)si_a *
                         (signed long long)si_b;
  /*
   * If the product cannot be represented as a 32-bit integer,
   * handle as an error condition.
   */
  if ( (tmp > INT_MAX) || (tmp < INT_MIN) ) {
    /* Handle error condition */
  } else {
    result = (int)tmp;
  }
  /* ... */
}

The assertion fails if long long has less than twice the width of int. It relies on the UWIDTH() macro, which is defined in INT19-C. Correctly compute integer widths.

On systems where this assertion would fail, the following compliant solution may be used to ensure signed overflow does not occur:

#include <limits.h>
 
signed int si_a;
signed int si_b;
signed int result;

void func(void) {
  /* Initialize si_a, si_b and result */

  if (si_a > 0) {  /* si_a is positive */
    if (si_b > 0) {  /* si_a and si_b are positive */
      if (si_a > (INT_MAX / si_b)) {
        /* Handle error condition */
      }
    } else { /* si_a positive, si_b nonpositive */
      if (si_b < (INT_MIN / si_a)) {
        /* Handle error condition */
      }
    } /* si_a positive, si_b nonpositive */
  } else { /* si_a is nonpositive */
    if (si_b > 0) { /* si_a is nonpositive, si_b is positive */
      if (si_a < (INT_MIN / si_b)) {
        /* Handle error condition */
      }
    } else { /* si_a and si_b are nonpositive */
      if ( (si_a != 0) && (si_b < (INT_MAX / si_a))) {
        /* Handle error condition */
      }
    } /* End if si_a and si_b are nonpositive */
  } /* End if si_a is nonpositive */

  result = si_a * si_b;

}

Division

Division is between two operands of arithmetic type. Overflow can occur during two's complement signed integer division when the dividend is equal to the minimum (negative) value for the signed integer type and the divisor is equal to −1. Division operations are also susceptible to divide-by-zero errors. (See INT33-C. Ensure that division and modulo operations do not result in divide-by-zero errors.)

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the division of the signed operands s_a and s_b or in a divide-by-zero error. The IA-32 architecture, for example, requires that both conditions result in a fault, which can easily result in a denial-of-service attack.

signed long s_a;
signed long s_b;
signed long result;

void func(void) {
  /* Initialize s_a and s_b */
  result = s_a / s_b;

  /* ... */
}

Compliant Solution

This compliant solution guarantees there is no possibility of signed overflow or divide-by-zero errors:

#include <limits.h>
 
signed long s_a;
signed long s_b;
signed long result;

void func(void) {
  /* Initialize s_a, s_b and result */
  if ( (s_b == 0) || ( (s_a == LONG_MIN) && (s_b == -1) ) ) {
    /* Handle error condition */
  } else {
    result = s_a / s_b;
  }

  /* ... */
}

Modulo

The modulo operator provides the remainder when two operands of integer type are divided.

Noncompliant Code Example

This noncompliant code example can result in a divide-by-zero error. Furthermore, many hardware platforms implement modulo as part of the division operator, which can overflow. Overflow can occur during a modulo operation when the dividend is equal to the minimum (negative) value for the signed integer type and the divisor is equal to −1. This occurs despite that the result of such a modulo operation should theoretically be 0.

signed long s_a;
signed long s_b;
signed long result;

void func(void) {
  /* Initialize s_a and s_b */
  result = s_a % s_b;

  /* ... */
}

Implementation Details

On x86 platforms, the modulo operator for signed integers is implemented by the idiv instruction code, along with the divide operator. Because INT_MIN / -1 overflows, this code will throw a floating-point exception on INT_MIN % -1.

On MSVC++, taking the modulo of INT_MIN by −1 yields the value 0. On GCC/Linux, taking the modulo of INT_MIN by −1 produces a floating-point exception. However, on GCC 4.2.4 and newer, with optimization enabled, taking the modulo of INT_MIN by −1 yields the value 0.

Compliant Solution (Overflow Prevention)

This compliant solution tests the modulo operand to guarantee there is no possibility of a divide-by-zero error or an (internal) overflow error:

#include <limits.h>
 
signed long s_a;
signed long s_b;
signed long result;

void func(void) {
  /* Initialize s_a, s_b and result */
  if ((s_b == 0) || ((s_a == LONG_MIN) && (s_b == -1))) {
    /* Handle error condition */
  } else {
    result = s_a % s_b;
  }

  /* ... */
}

Compliant Solution (Absolute Value)

This compliant solution is based on the fact that both the division and modulo operators truncate toward 0, as specified in subclause 6.5.5, footnote 105, of the C Standard [ISO/IEC 9899:2011], which guarantees that

i % j

and

i % -j

are always equivalent.

Furthermore, it guarantees that the minimum signed value modulo −1 yields 0:

#include <limits.h>
 
signed long s_a;
signed long s_b;
signed long result;

void func(void) {
  /* Initialize s_a, s_b and result */

  if (s_b == 0) {
    /* Handle error condition */
  } else {
    if ((s_b < 0) && (s_b != LONG_MIN)) {
      s_b = -s_b;
    }
    result = s_a % s_b;
  }

  /* ... */
}

Unary Negation

The unary negation operator takes an operand of arithmetic type. Overflow can occur during two's complement unary negation when the operand is equal to the minimum (negative) value for the signed integer type.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the unary negation of the signed operand s_a:

signed long s_a;
signed long result;

void func(void) {
  /* Initialize s_a */
  result = -s_a;

  /* ... */
}

Compliant Solution

This compliant solution tests the suspect negation operation to guarantee there is no possibility of signed overflow:

#include <limits.h>
 
signed long s_a;
signed long result;

void func(void) {
  /* Initialize s_a and result*/
  if (s_a == INT_MIN) {
    /* Handle error condition */
  } else {
    result = -s_a;
  }
  /* ... */
}

Left-Shift Operator

The left-shift operator is between two operands of integer type.

Noncompliant Code Example

This noncompliant code example can result in signed integer overflow:

int si_a;
int si_b;
int sresult;

void func(void) {
  /* Initialize si_a and si_b */
  sresult = si_a << si_b;

  /* ... */
}

Compliant Solution

This compliant solution eliminates the possibility of overflow resulting from a left-shift operation:

#include <limits.h>

 
signed long s_a;
signed long result;

void func(void) {
  if ((si1 < 0) || (si2 < 0) ||
      (si2 >= UWIDTH(signed long, ULONG_MAX)) ||
      (si1 > (INT_MAX >> si2))) {
    /* Handle error condition */
  } else {
    sresult = si1 << si2;
  }


  /* ... */
}

The UWIDTH() macro provides the correct width for an unsigned integer type, and is defined in INT19-C. Correctly compute integer widths...see that rule for more information. This solution also complies with INT34-C. Do not shift a negative number of bits or more bits than exist in the operand.

Atomic Integers

The C Standard defines the behavior of arithmetic on atomic signed integer types to use two's complement representation with silent wraparound on overflow; there are no undefined results. However, although defined, these results may be unexpected and therefore carry similar risks to unsigned integer wrapping (see INT30-C. Ensure that unsigned integer operations do not wrap). Consequently, signed integer overflow of atomic integer types should also be prevented or detected. 

This section includes an example for the addition of atomic integer types only. For other operations, tests similar to the precondition tests for two’s complement integers used for nonatomic integer types can be used.

Noncompliant Code Example

This noncompliant code example using atomic integers can result in unexpected signed integer overflow:

#include <stdatomic.h>
 
atomic_int i;
int si_a;

void func(void) {
  /* Initialize i, si_a */
  atomic_fetch_add(&i, si_a);

  /* ... */
}

Compliant Solution

This compliant solution tests the operands to guarantee there is no possibility of signed overflow. It loads the value stored in the atomic integer and tests for possible overflow before performing the addition:

#include <limits.h>
#include <stdatomic.h>
 
atomic_int i;
int si_a;

void func(void) {
  /* Initialize si_a, i */
  int si_b = atomic_load(&i);

  if (((si_b>0) && (si_a > (INT_MAX-si_b))) ||
      ((si_b<0) && (si_a < (INT_MIN-si_b)))) {
    /* Handle error condition */
  } else {
    atomic_store(&i, si_a + si_b);
  }

  /* ... */
}

Risk Assessment

Integer overflow can lead to buffer overflows and the execution of arbitrary code by an attacker.

Automated Detection

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Related Guidelines

Bibliography