EXP39-C. Do not access a variable through a pointer of an incompatible type

Modifying a variable through a pointer of an incompatible type (other than unsigned char) can lead to unpredictable results. Subclause 6.2.7 of the C Standard states that two types may be distinct yet compatible and addresses precisely when two distinct types are compatible.

This problem is often caused by a violation of aliasing rules. The C Standard, 6.5, paragraph 7 [ISO/IEC 9899:2011], specifies those circumstances in which an object may or may not be aliased.

An object shall have its stored value accessed only by an lvalue expression that has one of the following types:

• a type compatible with the effective type of the object,
• a qualified version of a type compatible with the effective type of the object,
• a type that is the signed or unsigned type corresponding to the effective type of the object,
• a type that is the signed or unsigned type corresponding to a qualified version of the effective type of the object,
• an aggregate or union type that includes one of the aforementioned types among its members (including, recursively, a member of a subaggregate or contained union), or
• a character type.

Accessing an object by means of any other lvalue expression (other than unsigned char) is undefined behavior 37.

Noncompliant Code Example

In this noncompliant example, an object of type float is incremented through an int *. The programmer can use the unit in the last place to get the next representable value for a floating-point type.  However, accessing an object through a pointer of an incompatible type is undefined behavior.

#include <stdio.h>
 
void f(void) {
  if (sizeof(int) == sizeof(float)) {
    float f = 0.0f;
    int *ip = (int *)&f;
    (*ip)++;
    printf("float is %f\n", f);
  }
}

Compliant Solution

In this compliant solution, the standard C function nextafterf() is used to round toward the highest representable floating-point value:

#include <float.h>
#include <math.h>
#include <stdio.h>
 
void f(void) {
  float f = 0.0f;
  f = nextafterf(f, FLT_MAX);
  printf("float is %f\n", f);
}

Noncompliant Code Example

In this noncompliant code example, an array of two values of type short is treated as an integer and assigned an integer value. The resulting values are indeterminate.

#include <stdio.h>
 
void func(void) {
  short a[2];
  a[0]=0x1111;
  a[1]=0x1111;

  *(int *)a = 0x22222222;

  printf("%x %x\n", a[0], a[1]);
}

When translating this code, an implementation can assume that no access through an integer pointer can change the array a, consisting of shorts. Consequently, printf() may be called with the original values of a[0] and a[1].

Implementation Details

Recent versions of GCC turn on the option -fstrict-aliasing (which allows alias-based optimizations) by default with -O2. Some architectures then print "1111 1111" as a result. Without optimization, the executable generates the expected output "2222 2222."

To disable optimizations based on alias analysis for faulty legacy code, the option -fno-strict-aliasing can be used as a workaround. The option -Wstrict-aliasing (which is included in -Wall) warns about some, but not all, violations of aliasing rules when -fstrict-aliasing is active.

When GCC 3.4.6 compiles this code with optimization, the assignment through the aliased pointer is effectively eliminated.

Compliant Solution

This compliant solution uses a union type that includes a type compatible with the effective type of the object:

#include <stdio.h>
 
void func(void) {
  union {
    short a[2];
    int i;
  } u;

  u.a[0]=0x1111;
  u.a[1]=0x1111;
  u.i = 0x22222222;

  printf("%x %x\n", u.a[0], u.a[1]);

  /* ... */
}

The printf() behavior in this compliant solution is unspecified, but it is commonly accepted as an implementation extension (see unspecified behavior 11).

This function typically outputs "2222 2222." However, there is no guarantee that this will be true, even on implementations that defined the unspecified behavior; values of type short need not have the same representation as values of type int.

Noncompliant Code Example

In this noncompliant code example, a gadget object is allocated, then realloc() is called to create a widget object using the memory from the gadget object. Although reusing memory to change types is acceptable, accessing the memory copied from the original object is undefined behavior.

#include <stdlib.h>
 
struct gadget {
  int i;
  double d;
  char *p;
};
 
struct widget {
  char *q;
  int j;
  double e;
};
 
void func(void) {
  struct gadget *gp;
  struct widget *wp;
 
  gp = (struct gadget *)malloc(sizeof(struct gadget));
  if (!gp) {
    /* Handle error */
  }
  /* ... Initialize gadget ... */
  wp = (struct widget *)realloc(gp, sizeof(struct widget));
  if (!wp) {
    free(gp);
    /* Handle error */
  }
  if (wp->j == 12) {
    /* ... */
  }
}

Compliant Solution

This compliant solution reuses the memory from the gadget object but reinitializes the memory to a consistent state before reading from it:

#include <stdlib.h>
#include <string.h>
 
struct gadget {
  int i;
  double d;
  char *p;
};
 
struct widget {
  char *q;
  int j;
  double e;
};
 
void func(void) {
  struct gadget *gp;
  struct widget *wp;
 
  gp = (struct gadget *)malloc(sizeof (struct gadget));
  if (!gp) {
    /* Handle error */
  }
  /* ... */
  wp = (struct widget *)realloc(gp, sizeof(struct widget));
  if (!wp) {
    free(gp);
    /* Handle error */
  }
  memset(wp, 0, sizeof(struct widget));
  /* ... Initialize widget ... */

  if (wp->j == 12) {
    /* ... */
  }
}

Noncompliant Code Example

According to the C Standard, 6.7.6.2 [ISO/IEC 9899:2011], using two or more incompatible arrays in an expression is undefined behavior (see also undefined behavior 76).

For two array types to be compatible, both should have compatible underlying element types, and both size specifiers should have the same constant value. If either of these properties is violated, the resulting behavior is undefined.

In this noncompliant code example, the two arrays a and b fail to satisfy the equal size specifier criterion for array compatibility. Because a and b are not equal, writing to what is believed to be a valid member of a might exceed its defined memory boundary, resulting in an arbitrary memory overwrite.

enum { ROWS = 10, COLS = 15 };
 
void func(void) {
  int a[ROWS][COLS];
  int (*b)[ROWS] = a;
}

Most compilers will produce a warning diagnostic if the two array types used in an expression are incompatible.

Compliant Solution

In this compliant solution, b is declared to point to an array with the same number of elements as a, satisfying the size specifier criterion for array compatibility:

enum { ROWS = 10, COLS = 15 };
 
void func(void) {
  int a[ROWS][COLS];
  int (*b)[COLS] = a;
}

Risk Assessment

Optimizing for performance can lead to aliasing errors that can be quite difficult to detect. Furthermore, as in the preceding example, unexpected results can lead to buffer overflow attacks, bypassing security checks, or unexpected execution.

Automated Detection

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Related Guidelines

Bibliography