You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 4 Next »


Casting does not work as expected when optimization is turned on. This is often caused by a violation of aliasing rules, which are part of the ISO C standard.

C99 [ISO/IEC 9899:1999]  states that

7. An object shall have its stored value accessed only by an lvalue expression that has one of
the following types: (78)

— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the
object,
— a type that is the signed or unsigned type corresponding to a qualified version of the
effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its
members (including, recursively, a member of a subaggregate or contained union), or
— a character type.

/78) The intent of this list is to specify those circumstances in which an object may or may not be aliased./

These rules say that a program is invalid if you try to access a variable through a pointer of an incompatible type. This is happening in the following example where a short is accessed through a pointer to integer (the code assumes 16-bit shorts and 32-bit integers).

Noncompliant Code Example


#include <stdio.h>

int main()
{
    short a\[2\];

    a[0]=0x1111;
    a[1]=0x1111;

    *(int *)a = 0x22222222;  /* violation of aliasing rules */

    printf("%x %x\n", a[0], a[1]);
    return 0;
}

The aliasing rules were designed to allow compilers more aggressive optimization. Basically, a compiler can assume that all changes to variables happen through pointers or references to variables of a type compatible to the accessed variable. Dereferencing a pointer that violates the aliasing rules results in undefined behavior and can be optimized out as follows.

#include <stdio.h>

int main()
{
    short a\[2\];

    a[0]=0x1111;
    a[1]=0x1111;

    printf("%x %x\n", a[0], a[1]);
    return 0;
}

In the case above, the compiler may assume that no access through an integer pointer can change the array a, consisting of shorts. Thus, printf may be called with the original values of a[0] and a[1]. What really happens is up to the compiler and may change with architecture and optimization level.

Recent versions of GCC turn on the option -fstrict-aliasing (which allows alias-based optimizations) by default with -O2. And some architectures then really print "1111 1111" as result. Without optimization the executable will generate the "expected" output "2222 2222".

To disable optimizations based on alias-analysis for faulty legacy code, the option -fno-strict-aliasing can be used as a work-around. The option -Wstrict-aliasing (which is included in -Wall) warns about some - but not all - cases of violation of aliasing rules when -fstrict-aliasing is active.

Compliant Solution

To fix the code above, you can use a union instead of a cast (note that this is a GCC extension which might not work with other compilers).

#include <stdio.h>

int main()
{
    union
    {
        short a[2];
        int i;
    }u;

    u.a[0]=0x1111;
    u.a[1]=0x1111;

    u.i = 0x22222222;

    printf("%x %x\n", u.a[0], u.a[1]);
    return 0;
}

Now the result will always be "2222 2222".

Risk Assessment

Optimizing for performance can lead to such aliasing errors which can be quite difficult to detect. Furthermore as in the case above unexpected results can lead to buffer overflow attacks and/or bypassing security checks and/or unexpected execution.

Rule

Severity

Likelihood

Remediation Cost

Priority

Level

OBJ31-J

medium

probable

low

P4

L3

References

GCC Known Bugs C bugs, Aliasing issues while casting to incompatible types

Aliasing, pointer casts and gcc 3.3 Aliasing issue

  • No labels