MEM07-A. Ensure that size arguments to calloc() do not result in an integer overflow

The calloc() function takes two arguments: the number of elements to allocate and the storage size of those elements. Typically, calloc() function implementations multiply these arguments together to determine how much memory to allocate. Historically, some implementations failed to check if this multiplication could result in an integer overflow. If the result of multiplying the number of elements to allocate and the storage size cannot be represented as a size_t, less memory is allocated than was requested. As a result, it is necessary to ensure that these arguments, when multiplied, do not result in an integer overflow.

According to RUS-CERT Advisory 2002-08:02, the following C/C++ implementations of calloc() are defective:

• GNU libc 2.2.5
• Microsoft Visual C++ versions 4.0 and 6.0 (including the C++ new allocator)
• GNU C++ Compiler (GCC versions 2.95, 3.0, and 3.1.1)
• HP-UX 11 implementations prior to 2004-01-14
• dietlibc CVS implementations prior to 2002-08-05
• libgcrypt 1.1.10 (GNU Crypto Library)

Non-Compliant Code Example

In this example, the user-defined function get_size() (not shown) is used to calculate the size requirements for a dynamic array of long int that is assigned to the variable num_elements. When calloc() is called to allocate the buffer, num_elements is multiplied by sizeof(long) to compute the overall size requirements. If the number of elements multiplied by the size cannot be represented as a size_t, calloc() may allocate a buffer of insufficient size. When data is copied to that buffer, a buffer overflow may occur.

size_t num_elements = get_size();
long *buffer = calloc(num_elements, sizeof(long));
if (buffer == NULL) {
  /* handle error condition */
}
/*...*/
free(buffer);

Compliant Solution

In this compliant solution, the multiplication of the two arguments num_elements and sizeof(long) is evaluated before the call to calloc() to determine if an overflow will occur. The multsize_t() function sets errno to a non-zero value if the multiplication operation overflows.

long *buffer;
size_t num_elements = calc_size();
(void) multsize_t(num_elements, sizeof(long));
if (errno) {
  /* handle error condition */
}
buffer = calloc(num_elements, sizeof(long));
if (buffer == NULL) {
  /* handle error condition */
}

Note that the maximum amount of allocatable memory is typically limited to a value less than SIZE_MAX (the maximum value of size_t). Always check the return value from a call to any memory allocation function.

Risk Assessment

Integer overflow in memory allocation functions can lead to buffer overflows that can be exploited by an attacker to execute arbitrary code with the permissions of the vulnerable process. Most implementations of calloc() now check to make sure integer overflow does not occur, but it is not always safe to assume the version of calloc() being used is secure, particularly when using dynamically linked libraries.

Comments

A modern implementation of the C standard library should check for overflows. If the libraries being used for a particular implementation properly handle possible integer overflows on the multiplication, that is sufficient to comply with this recommendation.

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

References

[[ISO/IEC 9899-1999]] Section 7.18.3, "Limits of other integer types"
[[Seacord 05]] Chapter 4, "Dynamic Memory Management"
[RUS-CERT Advisory 2002-08:02] "Flaw in calloc and similar routines"
[Secunia Advisory SA10635] "HP-UX calloc Buffer Size Miscalculation Vulnerability"