Part 9 · 1 chapters · ~8 min

Undefined Behaviour and Sanitisers

What undefined behaviour means, common sources, how optimisers exploit it, a measured silent heap overflow, AddressSanitizer, UndefinedBehaviorSanitizer, ThreadSanitizer and MemorySanitizer, Valgrind, fuzzing with libFuzzer, and safer habits (bounds-checked APIs, -fstack-protector, _FORTIFY_SOURCE).

10

Silent bugs, loud tools

code
char *name = malloc(8);
strcpy(name, "Adeolaaa");     // 8 characters + zero byte = 9 bytes into 8
free(name);
// measured: clang -O0 asan.c && ./a.out → exit status 0. The overflow went unnoticed.

// with AddressSanitizer:
clang -g -fsanitize=address,undefined asan.c && ./a.out
// ASan stops at the bad write and reports "heap-buffer-overflow", the access size,
// how far past the 8-byte region it landed, and stack traces for the write and the allocation

// overflow-safe arithmetic
int64_t total;
if (__builtin_add_overflow(balance, amount, &total)) return ERR_OVERFLOW;

// fuzzing: libFuzzer calls this with millions of generated inputs, under ASan
int LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { parse_request(data, size); return 0; }
// clang -g -fsanitize=fuzzer,address fuzz.c parser.c && ./a.out

Habits: compile with -Wall -Wextra -Werror; use snprintf, never sprintf or strcpy into fixed buffers; pass lengths with every buffer; build with -D_FORTIFY_SOURCE=2 -fstack-protector-strong in release; run tests under ASan and UBSan in CI. This is also the argument for Rust (Rust course): the compiler rules out most of this list.

UNDEFINED BEHAVIOUR
the compiler may assume it never happens
out of boundsReading or writing past an arrayor allocation.use after freeTouching memory after free; oftenexploitable.signed overflowINT_MAX + 1 is undefined;compilers optimise assuming itcannot happen.uninitialised readsUsing a local before assigning it.data racesTwo threads, one writing, nosynchronisation.null dereferenceUndefined in C, even though itusually crashes.
swipe the figure sideways, or tap expand for full screen
1/4
the meaning
Undefined behaviour means the C standard places no requirements at all. The program may crash, work, or do something else entirely, and may change with the optimisation level.
no requirements at allmay change with -O2