Casting does not work as expected when optimization is turned on. This is often caused by a violation of aliasing rules, which are part of the ISO C standard.
C99 [[ISO/IEC 9899:1999|AA. C References#ISO/IEC 9899-1999]] states that
7. An object shall have its stored value accessed only by an lvalue expression that has one of
the following types: (78)— a type compatible with the effective type of the object,
— a qualified version of a type compatible with the effective type of the object,
— a type that is the signed or unsigned type corresponding to the effective type of the
object,
— a type that is the signed or unsigned type corresponding to a qualified version of the
effective type of the object,
— an aggregate or union type that includes one of the aforementioned types among its
members (including, recursively, a member of a subaggregate or contained union), or
— a character type./78) The intent of this list is to specify those circumstances in which an object may or may not be aliased./
These rules say that a program is invalid if you try to access a variable through a pointer of an incompatible type. This is happening in the following example where a short is accessed through a pointer to integer (the code assumes 16-bit shorts and 32-bit ints).
Noncompliant Code Example
\#include <stdio.h>
int main()
{
short a\[2\];
a\[0\]=0x1111;
a\[1\]=0x1111;
\*(int \*)a = 0x22222222; /\* violation of aliasing rules \*/
printf("%x %x\n", a\[0\], a\[1\]);
return 0;
}
The aliasing rules were designed to allow compilers more aggressive optimization. Basically, a compiler can assume that all changes to variables happen through pointers or references to variables of a type compatible to the accessed variable. Dereferencing a pointer that violates the aliasing rules results in undefined behavior.
In the case above, the compiler may assume that no access through an integer pointer can change the array a, consisting of shorts. Thus, printf may be called with the original values of a[0] and a[1]. What really happens is up to the compiler and may change with architecture and optimization level.
Recent versions of GCC turn on the option -fstrict-aliasing (which allows alias-based optimizations) by default with -O2. And some architectures then really print "1111 1111" as result. Without optimization the executable will generate the "expected" output "2222 2222".
To disable optimizations based on alias-analysis for faulty legacy code, the option -fno-strict-aliasing can be used as a work-around. The option -Wstrict-aliasing (which is included in -Wall) warns about some - but not all - cases of violation of aliasing rules when -fstrict-aliasing is active.
Compliant Solution
To fix the code above, you can use a union instead of a cast (note that this is a GCC extension which might not work with other compilers).
\#include <stdio.h>
int main()
{
union
{
short a\[2\];
int i;
} u;
u.a\[0\]=0x1111;
u.a\[1\]=0x1111;
u.i = 0x22222222;
printf("%x %x\n", u.a\[0\], u.a\[1\]);
return 0;
}
Now the result will always be "2222 2222".
Risk Assessment
Unauthorized modifications to public static variables can result in unexpected behavior and can bypass important security checks and/or invoke malicious code.
|| Rule || Severity || Likelihood || Remediation Cost || Priority || Level ||
| OBJ31-J | high
| likely
| low
| P9 | L2 |
References