Part 2 · 2 chapters · ~20 min

The Memory Hierarchy

Registers to RAM with real latencies, cache lines and the prefetcher, and measured costs from Node: sequential 0.75 ns, random 6.5 ns, pointer chasing 88.8 ns per element; then virtual memory: pages, page tables, the TLB, faults, what translation buys, and behaviour under memory pressure.

5

Caches and locality

the constant that big-O hides
  1. The levels: each is 3 to 10 times slower than the one above, and RAM is about 300 additions away.
  2. Cache lines: memory moves 64 bytes at a time.
  3. The prefetcher keeps sequential scans fed.
  4. Random access is about 8.5× slower, because independent misses can overlap.
  5. Pointer chasing is about 127× slower, because every dependent hop is a trip to RAM.
  6. Locality is a design choice: arrays of values, structure of arrays, typed arrays.
code
// the pointer-chase experiment (Node, 16M nodes, 128 MB): the same list, two layouts
const N = 1 << 24;
const perm = shuffle(Uint32Array.from({ length: N }, (_, i) => i));
const randomList = new Uint32Array(N); for (let k = 0; k < N; k++) randomList[perm[k]] = perm[(k + 1) % N];
const orderedList = Uint32Array.from({ length: N }, (_, i) => (i + 1) % N);
const chase = (next: Uint32Array, steps: number) => { let p = 0; for (let k = 0; k < steps; k++) p = next[p]; return p; };
// measured on Apple silicon, Node 25:
//   list in memory order   12 ms     0.7 ns/hop
//   list in random order 1489 ms    88.8 ns/hop    ← a cache miss per hop
the frontend version
A virtualised list (the FSD course part 6) wins partly because a small DOM fits in cache. An array of plain numbers in a Float64Array beats an array of { price } objects because the numbers sit side by side, while the objects are pointers to separate heap cells. Chart data, order books and large tables belong in typed arrays.
THE MEMORY HIERARCHY
registers to RAM, each level bigger and slower, and the cache line that moves between them
swipe the figure sideways, or tap expand for full screen
1/6
the levels
The levels: registers (a few hundred bytes, under a cycle), L1 (32 to 128 KB per core, about 1 ns), L2 (256 KB to 2 MB per core, about 3 to 4 ns), L3 (8 to 64 MB shared, about 10 to 15 ns), RAM (gigabytes, 80 to 100 ns), SSD (terabytes, 50 to 100 microseconds). Each step down is roughly 3 to 10 times slower and much bigger.
6

Virtual memory, pages and the TLB

every address is translated
  1. Pages are 4 KB (16 KB on Apple silicon). An address is a page number plus an offset.
  2. Page tables are a multi-level tree with permissions on each entry.
  3. The TLB caches translations, and huge pages extend how much it covers.
  4. Page faults: a minor fault maps a page already in memory, a major fault reads it from disk.
  5. What it buys: isolation, lazy allocation, copy-on-write, mmap, overcommit.
  6. Under memory pressure, desktops reclaim pages and phones kill processes.
code
# watch page faults from the shell (macOS / Linux)
/usr/bin/time -l node -e "new Float64Array(1<<26).fill(1)"      # macOS: "page reclaims", "page faults"
/usr/bin/time -v node -e "new Float64Array(1<<26).fill(1)"      # Linux: minor / major page faults
# measured (macOS, Apple silicon, 16 KB pages; macOS reports minor faults as "page reclaims"):
#   touched with .fill(1):   35,962 page reclaims   (node alone: ~3,100 → ~32,800 for 512 MB ≈ 512 MB / 16 KB)
#   allocated, untouched:     3,200 page reclaims   (lazy allocation: no pages until first touch)
VIRTUAL MEMORY, PAGES AND THE TLB
every address a program uses is translated, page by page, by hardware the OS sets up
swipe the figure sideways, or tap expand for full screen
1/6
pages
Pages: memory is managed in fixed-size pages, 4 KB on x86 and most Linux ARM, 16 KB on Apple silicon. A virtual address splits into a page number and an offset within the page; translation replaces the page number and keeps the offset.