CON32-C. Prevent data races when accessing adjacent data by multiple threads

When accessing an object, a thread may inadvertently access a separate object in adjacent memory. This is an artifact of objects being stored compactly, with one byte possibly holding multiple variables.  This is a common optimization on word-addressed machines. Bit-fields are especially prone to this behavior because compilers are allowed to store multiple bit-fields in one addressable byte or word. Consequently, data races may exist not just on an object accessed by multiple threads but also on other objects sharing the same byte or word address.

One approach for preventing data races in concurrent programming is the mutex. When properly observed by all threads, a mutex can provide safe and secure access to a shared object. However, mutexs provide no guarantees with regard to other objects that might be accessed when the mutex is not controlled by the accessing thread. Unfortunately, there is no portable way to determine which adjacent variables may be stored along with a certain variable.

Another approach is to embed a concurrently accessed object inside a union alongside a long object or other padding to ensure that the object is the only one accessed at that address. This technique effectively guarantee that no two object are accessed simultaneously.

Noncompliant Code Example (Bit-field)

Adjacent bit-fields may be stored in a single memory location. Consequently, modifying adjacent bit-fields in different threads is undefined behavior:

struct multi_threaded_flags {
  unsigned int flag1 : 2;
  unsigned int flag2 : 2;
};

struct multi_threaded_flags flags;

int thread1(void *arg) {
  flags.flag1 = 1;
  return 0;
}

int thread2(void *arg) {
  flags.flag2 = 2;
  return 0;
}

The C Standard, subclause 3.14.3 [ISO/IEC 9899:2011], states:

NOTE 2 A bit-field and an adjacent non-bit-field member are in separate memory locations. The same applies to two bit-fields, if one is declared inside a nested structure declaration and the other is not, or if the two are separated by a zero-length bit-field declaration, or if they are separated by a non-bit-field member declaration. It is not safe to concurrently update two non-atomic bit-fields in the same structure if all members declared between them are also (non-zero-length) bit-fields, no matter what the sizes of those intervening bit-fields happen to be.

For example, the following sequence of events can occur:

Thread 1: register 0 = flags
Thread 1: register 0 &= ~mask(flag1)
Thread 2: register 0 = flags
Thread 2: register 0 &= ~mask(flag2)
Thread 1: register 0 |= 1 << shift(flag1)
Thread 1: flags = register 0
Thread 2: register 0 |= 2 << shift(flag2)
Thread 2: flags = register 0

Compliant Solution (Bit-field, C11, Mutex)

This compliant solution protects all accesses of the flags with a mutex, thereby preventing any data races. Finally, the flags are embedded in a union alongside a long, and a static assertion guarantees that the flags do not occupy more space than the long. This technique prevents any data not checked by the mutex from being accessed or modified with the bit-fields on platforms that do not comply with C11.

#include <threads.h>
#include <assert.h>
 
struct multi_threaded_flags {
  unsigned int flag1 : 2;
  unsigned int flag2 : 2;
};

union mtf_protect {
  struct multi_threaded_flags s;
  long padding;
};
struct mtf_mutex {
  union mtf_protect u;
  mtx_t mutex;
};

struct mtf_mutex flags;

void chk_flags(void) {
  static_assert(
    sizeof(long) >= sizeof(struct multi_threaded_flags),
    "A long type will not hold the flags on this architecture."
  );
}

int thread1(void *arg) {
  if (thrd_success != mtx_lock(&flags.mutex)) {
    /* Handle error */
  }
  flags.u.s.flag1 = 1;
  if (thrd_success != mtx_unlock(&flags.mutex)) {
    /* Handle error */
  }
  return 0;
}
 
int thread2(void *arg) {
  if (thrd_success != mtx_lock(&flags.mutex)) {
    /* Handle error */
  }
  flags.u.s.flag2 = 2;
  if (thrd_success != mtx_unlock(&flags.mutex)) {
    /* Handle error */
  }
  return 0;
}

Static assertions are discussed in detail in DCL03-C. Use a static assertion to test the value of a constant expression.

Compliant Code Example (C11)

In this compliant code example, two threads simultaneously modify two distinct members of a structure:

struct multi_threaded_flags {
  unsigned char flag1;
  unsigned char flag2;
};
 
struct multi_threaded_flags flags;
 
int thread1(void *arg) {
  flags.flag1 = 1;
  return 0;
}

int thread2(void *arg) {
  flags.flag2 = 2;
  return 0;
}

Unlike C99, C11 explicitly defines a memory location and provides the following note in subclause 3.14.2 [ISO/IEC 9899:2011]:

NOTE 1 Two threads of execution can update and access separate memory locations without interfering with each other.

In a C99 or earlier compliant compiler it is possible that flag1 and flag2 are stored in the same word. If both assignments occur on a thread-scheduling interleaving that ends with both stores occurring after one another, it is possible that only one of the flags will be set as intended, and the other flag will equal its previous value, because both chars are represented by the same word, which is the smallest unit the processor can work on. Before the changed made to the C Standard for C11, the Standard made no guarantees that these flags can be modified concurrently.

Even though each thread is modifying a separate object, they may be modifying the same word in memory. A similar problem is discussed in CON00-C. Avoid race conditions with multiple threads, but this example can be harder to diagnose because it is not immediately obvious that the same memory location is being modified.

Risk Assessment

Although the race window is narrow, having an assignment or an expression evaluate improperly because of misinterpreted data can result in a corrupted running state or unintended information disclosure.

Automated Detection

Related Vulnerabilities

Search for vulnerabilities resulting from the violation of this rule on the CERT website.

Bibliography