Integer overflow is undefined behavior. This means that implementations have a great deal of latitude in how they deal with signed integer overflow. An implementation that defines signed integer types as being modulo, for example, need not detect integer overflow. Implementations may also trap on signed arithmetic overflows, or simply assume that overflows will never happen and generate object code accordingly (see MSC15-C. Do not depend on undefined behavior). For these reasons, it is important to ensure that operations on signed integers do no result in signed overflow. Of particular importance, however, are operations on signed integer values that originate from untrusted sources and are used in any of the following ways:

• as an array index
• in any pointer arithmetic
• as a length or size of an object
• as the bound of an array (for example, a loop counter)
• as an argument to a memory allocation function
• in security-critical code

Most integer operations can result in overflow if the resulting value cannot be represented by the underlying representation of the integer. The following table indicates which operators can result in overflow:

The following sections examine specific operations that are susceptible to integer overflow. When operating on small integer types (smaller than int), integer promotions are applied. The usual arithmetic conversions may also be applied to (implicitly) convert operands to equivalent types before arithmetic operations are performed. Make sure you understand integer conversion rules before trying to implement secure arithmetic operations (see INT02-C. Understand integer conversion rules).

Addition

Addition is between two operands of arithmetic type or between a pointer to an object type and an integer type (see ARR37-C. Do not add or subtract an integer to a pointer to a non-array object and ARR38-C. Do not add or subtract an integer to a pointer if the resulting value does not refer to a valid array element for rules about adding a pointer to an integer). Incrementing is equivalent to adding one.

Noncompliant Code Example

This noncompliant code example may result in a signed integer overflow during the addition of the signed operands si1 and si2. If this behavior is unanticipated, it can lead to an exploitable vulnerability.

int si1, si2, sum;

/* Initialize si1 and si2 */

sum = si1 + si2;

Compliant Solution (Pre-condition Test, Two's Complement)

This compliant solution performs a pre-condition test of the operands of the addition to ensure no overflow occurs, assuming two's complement representation.

signed int si1, si2, sum;

/* Initialize si1 and si2 */

if ( ((si1^si2) | (((si1^(~(si1^si2) & INT_MIN)) + si2)^si2)) >= 0) {
   /* handle error condition */
} else {
  sum = si1 + si2;
}

This compliant solution works only on architectures that use two's complement representation. While most modern platforms use two's complement representation, it is best not to introduce unnecessary platform dependencies (see MSC14-C. Do not introduce unnecessary platform dependencies). This solution can also be more expensive than a post-condition test, especially on RISC CPUs.

Compliant Solution (Post-condition Test, Two's Complement)

For addition and subtraction of signed integers using two's-compliment representation in which both operands have like signs, a change of sign in the result indicates an overflow condition. If the signs of the operands are different there can be no overflow.

This compliant solution performs a post-condition test following the addition operation to detect if an overflow occurred, assuming two's complement representation and the absence of traps.

signed int si1, si2, sum;

/* Initialize si1 and si2 */

sum = si1 + si2;
if ((si1 > 0 && sum < 0) || (si1 < 0 && sum > 0)) {
  /* handle error condition */
} 

This compliant solution works only on architectures that use two's complement representation. While most modern platforms use two's complement representation, it is best not to introduce unnecessary platform dependencies (see MSC14-C. Do not introduce unnecessary platform dependencies).

This compliant solution also depends on undefined behavior (see MSC15-C. Do not depend on undefined behavior). Conforming implementations are free to perform any action in the presence of undefined behavior, including trapping. If the addition operation traps, it may not be possible to handle the error in the preferred application-defined manner. Also, a compiler has the license to eliminate any code that depends on the presence of undefined behavior, so it is possible that a compiler will optimize out the entire if statement containing the overflow test.

Compliant Solution (General)

This compliant solution tests the suspect addition operation to ensure no overflow occurs regardless of representation.

signed int si1, si2, sum;

/* Initialize si1 and si2 */

if (((si1>0) && (si2>0) && (si1 > (INT_MAX-si2))) 
 || ((si1<0) && (si2<0) && (si1 < (INT_MIN-si2)))) {
   /* handle error condition */
}
else {
  sum = si1 + si2;
}

This solution is more readable but may be less efficient than the solution that is specific to two's complement representation.

Subtraction

Subtraction is between two operands of arithmetic type, two pointers to qualified or unqualified versions of compatible object types, or between a pointer to an object type and an integer type. See ARR36-C. Do not subtract or compare two pointers that do not refer to the same array, ARR37-C. Do not add or subtract an integer to a pointer to a non-array object, and ARR38-C. Do not add or subtract an integer to a pointer if the resulting value does not refer to a valid array element for rules about pointer subtraction. Decrementing is equivalent to subtracting one.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the subtraction of the signed operands si1 and si2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner that can lead to an exploitable vulnerability.

signed int si1, si2, result;

/* Initialize si1 and si2 */

result = si1 - si2;

Compliant Solution (Two's Complement)

This compliant solution tests the operands of the subtraction to guarantee there is no possibility of signed overflow, presuming two's complement representation.

signed int si1, si2, result;

/* Initialize si1 and si2 */

if (((si1^si2) 
  & (((si1 ^ ((si1^si2) 
    & (1 << (sizeof(int)*CHAR_BIT-1))))-si2)^si2)) < 0) {
  /* handle error condition */
}
else {
  result = si1 - si2;
}

This compliant solution only works on architectures that use two's complement representation. While most modern platforms use two's complement representation, it is best not to introduce unnecessary platform dependencies (see MSC14-C. Do not introduce unnecessary platform dependencies).

Multiplication

Multiplication is between two operands of arithmetic type.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the multiplication of the signed operands si1 and si2. If this behavior is unanticipated, the resulting value may be used to allocate insufficient memory for a subsequent operation or in some other manner that can lead to an exploitable vulnerability.

signed int si1, si2, result;

/* ... */

result = si1 * si2;

Compliant Solution

This compliant solution guarantees there is no possibility of signed overflow on systems where long long is at least twice the size of int.

signed int si1, si2, result;

/* Initialize si1 and si2 */

static_assert(
  sizeof(long long) >= 2 * sizeof(int), 
  "Unable to detect overflow after multiplication"
);

signed long long tmp = (signed long long)si1 * 
                       (signed long long)si2;
/*
 * If the product cannot be represented as a 32-bit integer, 
 * handle as an error condition.
 */
if ( (tmp > INT_MAX) || (tmp < INT_MIN) ) {
  /* handle error condition */
}
else {
  result = (int)tmp;
}

The compliant solution uses a static assertion to ensure that the overflow detection will succeed. See DCL03-C. Use a static assertion to test the value of a constant expression for a discussion of static assertions.

On systems where this relationship does not exist, the following compliant solution may be used to ensure signed overflow does not occur.

signed int si1, si2, result;

/* Initialize si1 and si2 */

if (si1 > 0){  /* si1 is positive */
  if (si2 > 0) {  /* si1 and si2 are positive */
    if (si1 > (INT_MAX / si2)) {
      /* handle error condition */
    }
  } /* end if si1 and si2 are positive */
  else { /* si1 positive, si2 non-positive */
    if (si2 < (INT_MIN / si1)) {
        /* handle error condition */
    }
  } /* si1 positive, si2 non-positive */
} /* end if si1 is positive */
else { /* si1 is non-positive */
  if (si2 > 0) { /* si1 is non-positive, si2 is positive */
    if (si1 < (INT_MIN / si2)) {
      /* handle error condition */
    }
  } /* end if si1 is non-positive, si2 is positive */
  else { /* si1 and si2 are non-positive */
    if ( (si1 != 0) && (si2 < (INT_MAX / si1))) {
      /* handle error condition */
    }
  } /* end if si1 and si2 are non-positive */
} /* end if si1 is non-positive */

result = si1 * si2;

Division

Division is between two operands of arithmetic type. Overflow can occur during two's-complement signed integer division when the dividend is equal to the minimum (negative) value for the signed integer type and the divisor is equal to — 1. Division operations are also susceptible to divide-by-zero errors (see INT33-C. Ensure that division and modulo operations do not result in divide-by-zero errors).

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the division of the signed operands sl1 and sl2 or in a divide-by-zero error. The IA-32 architecture, for example, requires that both conditions result in a fault, which can easily result in a denial-of-service attack.

signed long sl1, sl2, result;

/* Initialize sl1 and sl2 */

result = sl1 / sl2;

Compliant Solution

This compliant solution guarantees there is no possibility of signed overflow or divide-by-zero errors.

signed long sl1, sl2, result;

/* Initialize sl1 and sl2 */

if ( (sl2 == 0) || ( (sl1 == LONG_MIN) && (sl2 == -1) ) ) {
  /* handle error condition */
}
else {
  result = sl1 / sl2;
}

Modulo

The modulo operator provides the remainder when two operands of integer type are divided.

Noncompliant Code Example

This noncompliant code example can result in a divide-by-zero error. Furthermore, many hardware platforms implement modulo as part of the division operator, which can overflow. Overflow can occur during a modulo operation when the dividend is equal to the minimum (negative) value for the signed integer type and the divisor is equal to — 1. This occurs in spite of the fact that the result of such a modulo operation should theoretically be 0.

signed long sl1, sl2, result;

result = sl1 % sl2;

Implementation Details

On x86 platforms, the modulo operator for signed ints is implemented by the idiv instruction code, along with the divide operator. Since INT_MIN / -1 overflows, this code will throw a floating-point exception on INT_MIN % -1.

On MSVC++, taking the modulo of INT_MIN by -1 yields the value 0. On gcc/Linux, taking the modulo of INT_MIN by -1 produces a floating-point exception. However, on gcc versions 4.2.4 and newer, with optimization enabled, taking the modulo of INT_MIN by -1 yields the value 0.

Compliant Solution (Overflow Prevention)

This compliant solution tests the modulo operand to guarantee there is no possibility of a divide-by-zero error or an (internal) overflow error.

signed long sl1, sl2, result;

/* Initialize sl1 and sl2 */

if ( (sl2 == 0 ) || ( (sl1 == LONG_MIN) && (sl2 == -1) ) ) {
  /* handle error condition */
}
else {
  result = sl1 % sl2;
}

Compliant Solution (Absolute Value)

This compliant solution is based on the fact that both the division and modulo operators truncate towards zero, as specified in a footnote in paragraph 6.5.5 of the C99 standard. This guarantees that:

i % j

and

i % -j

are always equivalent.

Furthermore, it guarantees that the minumum signed value modulo -1 yields 0.

signed long sl1, sl2, result;

/* Initialize sl1 and sl2 */

if (sl2 == 0) {
  /* handle error condition */
}
else {
  if ((sl2 < 0) && (sl2 != LONG_MIN)) {
    sl2 = -sl2;
  }
  result = sl1 % sl2;
}

Unary Negation

The unary negation operator takes an operand of arithmetic type. Overflow can occur during two's complement unary negation when the operand is equal to the minimum (negative) value for the signed integer type.

Noncompliant Code Example

This noncompliant code example can result in a signed integer overflow during the unary negation of the signed operand si1.

signed int si1, result;

/* Initialize si1 */

result = -si1;

Compliant Solution

This compliant solution tests the suspect negation operation to guarantee there is no possibility of signed overflow.

signed int si1, result;

/* Initialize si1 */

if (si1 == INT_MIN) {
  /* handle error condition */
}
else 
  result = -si1;
}

Left-Shift Operator

The left-shift operator is between two operands of integer type.

Noncompliant Code Example

This noncompliant code example can result in signed integer overflow.

int si1, si2, sresult;

/* Initialize si1 and si2 */

sresult = si1 << si2;

Compliant Solution

This compliant solution eliminates the possibility of overflow resulting from a left-shift operation.

int si1, si2, sresult;

/* Initialize si1 and si2 */

if ( (si1 < 0) || (si2 < 0) ||
     (si2 >= sizeof(int)*CHAR_BIT) ||
     (si1 > (INT_MAX >> si2)) 
) {
  /* handle error condition */
}
else {
  sresult = si1 << si2;
}

This solution is also compliant with INT34-C. Do not shift a negative number of bits or more bits than exist in the operand.

Risk Assessment

Integer overflow can lead to buffer overflows and the execution of arbitrary code by an attacker.

Automated Detection

Fortify SCA Version 5.0 with CERT C Rule Pack can detect violations of this rule.

Compass/ROSE can detect some violations of this rule. Specifically, it checks to ensure that the operand of a unary negation is compared to the type's minimum value immediately before the operation.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Other Languages

This rule appears in the C++ Secure Coding Standard as INT32-CPP. Ensure that operations on signed integers do not result in overflow.

This rule appears in the Java Secure Coding Standard as INT34-J. Perform explicit range checking to ensure integer operations do not overflow.

References

[[Dowd 06]] Chapter 6, "C Language Issues" (Arithmetic Boundary Conditions, pp. 211-223)
[[ISO/IEC 9899:1999]] Section 6.5, "Expressions," and Section 7.10, "Sizes of integer types <limits.h>"
[[ISO/IEC PDTR 24772]] "XYY Wrap-around Error"
[[MITRE 07]] CWE ID 129, "Unchecked Array Indexing" and CWE ID 190, "Integer Overflow (Wrap or Wraparound)"
[[Seacord 05]] Chapter 5, "Integers"
[[Viega 05]] Section 5.2.7, "Integer overflow"
[[VU#551436]]
[[Warren 02]] Chapter 2, "Basics"

      04. Integers (INT)      INT33-C. Ensure that division and modulo operations do not result in divide-by-zero errors