Part 4 · 1 chapters · ~8 min

RAID and Erasure Coding

RAID 0, 1, 5, 6 and 10, parity and the RAID 5 write hole, rebuild times and the risk of a second failure during rebuild, unrecoverable read errors, software RAID (mdadm) and ZFS RAID-Z, Reed-Solomon erasure codes, local reconstruction codes, and the durability arithmetic behind "eleven nines".

Parity and erasure codes

code

# parity in one line: P = D1 xor D2 xor D3 ; lose D2 → D2 = P xor D1 xor D3
mdadm --create /dev/md0 --level=6 --raid-devices=6 /dev/sd[b-g]    # RAID 6: survives two disks
cat /proc/mdstat                                                    # rebuild progress

# durability arithmetic (simplified): k + m fragments, independent failures
#   data is lost only if more than m fragments fail before repair completes
#   shorter repair windows and more parity both raise durability sharply
# Amazon S3 is designed for 99.999999999% (eleven nines) durability per object per year (AWS documentation)

Rebuild risk: large disks take many hours to rebuild, and a second failure (or an unrecoverable read error) during that window loses a RAID 5 array; this is why RAID 6, RAID-Z2 or erasure coding are preferred for large drives. RAID is not backup: it faithfully replicates deletions, corruption and ransomware.

STORAGE OVERHEAD FOR REDUNDANCY

raw bytes stored per byte of data, and failures survived

swipe the figure sideways, or tap expand for full screen

1/4

replication

Three full copies survive two failures, are simple, and read fast from any copy, at 3× the storage. HDFS and many databases default to it.

simple, 3× costsurvives 2 losses