Part 5 · 1 chapters · ~8 min

The Block Layer

bios and requests, the multi-queue block layer (blk-mq), I/O schedulers (none, mq-deadline, bfq, kyber), merging and plugging, queue depth, device mapper (LVM, dm-crypt), io_uring and asynchronous I/O, and measuring with iostat, biolatency and fio.

From bio to device

code

cat /sys/block/nvme0n1/queue/scheduler       # [none] mq-deadline kyber bfq
cat /sys/block/nvme0n1/queue/nr_requests     # queue depth
iostat -x 1                                  # r/s, w/s, await (ms), aqu-sz, %util per device
biolatency-bpfcc 1 5                         # histogram of block I/O latency (BCC tools, Systems Performance P8)

# fio: measure a device the way a database uses it (small random writes with fsync)
fio --name=wal --rw=write --bs=8k --size=1G --fsync=1 --ioengine=psync --filename=/data/fio.test

%util lies for SSDs: it reports the share of time the device had any I/O in flight. NVMe drives serve many requests in parallel, so 100% utilisation does not mean saturated; look at latency (await) and queue size instead. Device mapper stacks virtual block devices: LVM volumes, dm-crypt encryption and snapshots all live between the file system and the disk.

THE BLOCK LAYER

from page cache writeback to the device queue

swipe the figure sideways, or tap expand for full screen

1/4

bios

File systems turn dirty pages into bios: requests describing which memory goes to which disk sectors, in which direction.

a bio describes one I/Omemory pages ↔ sectors