You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 8 Next »

Modifying a variable through a pointer of an incompatible type can lead to unpredictable results. This is often caused by a violation of aliasing rules, which are part of the ISO C standard.

C99 [ISO/IEC 9899:1999]  states that

7. An object shall have its stored value accessed only by an lvalue expression that has one of
the following types: (78)

  • a type compatible with the effective type of the object,
  • a qualified version of a type compatible with the effective type of the object,
  • a type that is the signed or unsigned type corresponding to the effective type of the
    object,
  • a type that is the signed or unsigned type corresponding to a qualified version of the
    effective type of the object,
  • an aggregate or union type that includes one of the aforementioned types among its
    members (including, recursively, a member of a subaggregate or contained union), or
  • a character type.

/78) The intent of this list is to specify those circumstances in which an object may or may not be aliased./

These rules say that a program is invalid if you try to access a variable through a pointer of an incompatible type. This is happening in the following example where a short is accessed through a pointer to integer (the code assumes 16-bit shorts and 32-bit integers).

Noncompliant Code Example

In this example an array of two shorts is treated as an integer, and assigned an integer value. The resulting value of the two shorts is undefined.

#include <stdio.h>

int main()
{
    short a[2];

    a[0]=0x1111;
    a[1]=0x1111;

    *(int *)a = 0x22222222;  /* violation of aliasing rules */

    printf("%x %x\n", a[0], a[1]);
    return 0;
}

Implementation Details

In the case above, the compiler may assume that no access through an integer pointer can change the array a, consisting of shorts. Thus, printf may be called with the original values of a[0] and a[1]. What really happens is up to the compiler and may change with architecture and optimization level.

Recent versions of GCC turn on the option -fstrict-aliasing (which allows alias-based optimizations) by default with -O2. And some architectures then really print "1111 1111" as result. Without optimization the executable will generate the "expected" output "2222 2222".

To disable optimizations based on alias-analysis for faulty legacy code, the option -fno-strict-aliasing can be used as a work-around. The option -Wstrict-aliasing (which is included in -Wall) warns about some - but not all - cases of violation of aliasing rules when -fstrict-aliasing is active.

When GCC 3.4.6 compiles this code with optimization, it yeilds an executable that behaves like the following code:

#include <stdio.h>

int main()
{
    short a[2];

    a[0]=0x1111;
    a[1]=0x1111;

    printf("%x %x\n", a[0], a[1]);
    return 0;
}

In other words, GCC 3.4.6 effectively removes the assignment.

Compliant Solution

To fix the code above, you can use a union instead of a cast.

#include <stdio.h>

int main()
{
    union
    {
        short a[2];
        int i;
    }u;

    u.a[0]=0x1111;
    u.a[1]=0x1111;

    u.i = 0x22222222;

    printf("%x %x\n", u.a[0], u.a[1]);
    return 0;
}

Now the result will always be "2222 2222".

Risk Assessment

Optimizing for performance can lead to such aliasing errors which can be quite difficult to detect. Furthermore as in the case above unexpected results can lead to buffer overflow attacks and/or bypassing security checks and/or unexpected execution.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

EXP39-C

medium

probable

high

P12

L1

References

GCC Known Bugs C bugs, Aliasing issues while casting to incompatible types

Aliasing, pointer casts and gcc 3.3 Aliasing issue

  • No labels