The VFS and the Page Cache
The virtual file system layer, superblocks, inodes, dentries and files, path lookup and the dentry cache, file operations tables, the page cache and readahead, dirty pages and writeback tuning, fsync semantics and the 2018 fsyncgate lesson, O_DIRECT, and pseudo file systems (proc, sysfs, tmpfs).
Files in memory
cat /proc/sys/vm/dirty_background_ratio # start background writeback at this % of memory dirty cat /proc/sys/vm/dirty_ratio # writers block and flush themselves above this % grep -E 'Dirty|Writeback' /proc/meminfo vmtouch -v /var/lib/postgresql/data/base # how much of a directory is in the page cache (tool) echo 3 | sudo tee /proc/sys/vm/drop_caches # drop clean caches: for benchmarks only, never in production
fsyncgate (2018): PostgreSQL developers found that when fsync reported a writeback error on Linux, the failed dirty pages could be marked clean and a retried fsync would succeed without the data reaching disk. Postgres changed to treat an fsync failure as fatal and recover from the WAL (PANIC on fsync error). The lesson: on an fsync error, assume the data may be lost.
O_DIRECT bypasses the page cache for engines that manage their own buffer pool (InnoDB can, ScyllaDB does); Postgres relies on the page cache plus its own shared buffers, which is why its memory is usually tuned as shared_buffers plus OS cache.