V8's Algorithms
Under every mechanism the JS course drew there is a named algorithm: a trie for shapes, a tiny dispatch table for property access, a copying collector for the young, a tri-colour marker with a write barrier for the old. This part names each, gives its complexity and the data structure that makes it work, and ends with every engine algorithm in one table with its frontend-visible consequence.
The transition tree as a trie
"Hidden classes: the JS course said construction order decides identity. What is the data structure that makes that true?"
A trie. The root is the empty object's map; each edge is a (property name, attributes) pair; each node is the map an object has after adding that sequence of properties. Constructing an object by assignment is a walk down the trie, creating nodes on first use and following existing edges after. Two objects that take the same path share the leaf; two that take different paths do not, whatever properties they end up with. The JS course (part 3) is what the nodes contain; this chapter is the structure over them and its costs.
- Add property (the walk step): look up (name, attributes) in the current map's transition array. Hit: move to the child. Miss: allocate a child map (extend the descriptor array; assign the next slot), insert the edge, move. The transition array is a small sorted array searched by binary search, promoted to a hash table when a map accumulates many children.
- Object literal: no walk at runtime; the parser precomputed the leaf map for the literal's shape (its "boilerplate"), so creation is an allocation with that map plus field copies.
- Delete: no edge for removal; the object leaves the trie for dictionary mode.
- Reconfigure (defineProperty on an existing property, freeze): special transitions keyed by the operation, also in the trie.
- Field generalisation: when a field's representation widens (Smi to Double), the map and all its descendants are deprecated and a parallel subtree with the wider representation is created; live objects migrate lazily on their next access (the JS course part 4's "deprecated map" deopt).
- Per property addition: O(log k) for k siblings in the transition array (one or two compares typically), plus the allocation on a miss.
- Memory: O(number of distinct construction paths × depth). A class constructed one way is depth nodes total; constructed N ways is N × depth. Shape explosion is a trie with many branches.
- Sharing: descriptor arrays are shared along a chain (a child extends its parent's), so the trie's memory is less than nodes × properties.
- Pruning: maps unreachable from live objects and from their parents' transition arrays (held weakly) are collected; an identical construction later rebuilds the branch with new identities, and inline caches that remembered the old maps miss once.
Inline cache lookup
// inline cache lookup, as the algorithm it is: a tiny dispatch table keyed by map, specialised per call site
// state: uninitialised | monomorphic (1 entry) | polymorphic (2..4 entries) | megamorphic (global table)
function loadIC(site, obj) { // what the LoadIC handler does, in JS terms
const map = obj.map
for (const [m, handler] of site.entries) // 1 compare when monomorphic; up to 4 when polymorphic
if (m === map) return handler(obj) // handler: "load slot 2", "load from prototype P", "call getter"
// miss:
const handler = fullLookup(obj, site.name) // walk descriptors and the prototype chain: O(depth × properties)
if (site.entries.length < 4) site.entries.push([map, handler])
else site.state = 'megamorphic' // from now on: stubCache[hash(map, name)], a global 2-way set-associative table
return handler(obj)
}
// complexity per access: mono O(1), poly O(k ≤ 4), mega O(1) expected via the hash + a full lookup on a stub cache miss
// the data structure choice: a per-site array (tiny, inlined into code) vs a global hash (shared, never inlined). the per-site array wins
// while it stays small; the global table is the fallback. the JS course part 3 has the machine code each state compiles to- Per call site, a tiny table: up to four (map, handler) pairs, stored in the feedback vector slot (the JS course part 2). The handler is a small encoded recipe: "in-object slot 2", "properties array index 5", "on prototype P at depth 1, constant F", "call this getter".
- Lookup: compare the receiver's map against each entry. Monomorphic: one compare. Polymorphic: a linear scan of two to four. The table is small enough to be inlined into generated code as a compare chain (part 3 of the JS course has the machine code).
- Miss: a full lookup (descriptor array search on the map, then the prototype chain), then insert the result into the site's table. The fifth distinct map tips the site to megamorphic.
- Megamorphic: the site stops tracking maps; lookups go to the global stub cache, a fixed-size two-way set-associative hash table keyed by (map, name) → handler, shared by all megamorphic sites. A hit is a hash and a compare; a miss is a full lookup and an insertion that may evict another entry.
- Four entries: a linear scan of four compares is cheaper than a hash; beyond four, a hash wins, and the global table amortises its memory across sites. The number is a tuned constant, not a law.
- Per-site tables beat a global one while they fit because the compare chain is inlined (no call, no hash) and the compiler can hoist it; the global table is a call and can be evicted by other sites' entries.
- Handlers are data, not code, so a polymorphic site does not need four code paths; the optimiser, when it compiles the site, turns the table into code.
- Dynamic dispatch caches in every dynamic language runtime (Smalltalk invented inline caches; Self made them polymorphic; V8 inherited both).
- Branch prediction in CPUs: a per-site history that predicts the next outcome; wrong predictions are the deopt.
- Memoising a dispatch in your own code: a small array of (key, result) for the last few keys at a hot call site is often faster than a Map lookup, for exactly the reasons above.
Generational collection: Cheney's copying scavenger
The young generation is collected by copying: live objects are moved to a fresh space and everything left behind is garbage. Cheney's algorithm does this breadth-first with no recursion and no separate worklist, using the destination space itself as the queue. Its cost is proportional to the live data, which under the generational hypothesis is a small fraction of the space.
- Two pointers into to-space:
scan(the next copied object to process) andfree(where the next copy goes). Both start at the beginning. - Copy the roots' targets: for each root pointer into from-space: copy the object to
free, advancefree, overwrite the old object's header with a forwarding pointer to the copy, update the root. - Scan loop: while
scan < free: take the object atscan; for each pointer field referencing from-space: if the target is already forwarded, update the field to the forwarding address; else copy it (as in step 2) and update. Advancescanpast the object. - Flip: when
scan == free, everything reachable has been copied and every pointer updated. The from-space is now entirely garbage; swap the roles of the spaces. No sweeping, no free lists.
- Time O(live): dead objects are never visited. Allocation-heavy, short-lived workloads pay almost nothing per collection.
- Compacting by construction: survivors are packed contiguously in to-space, in breadth-first order, which also improves locality for objects that reference each other.
- Space: two semi-spaces, so half the young generation is idle at any time. The price of simplicity; the young generation is small, so it is cheap.
- Breadth-first order is a side effect of using to-space as the queue; depth-first would need a stack. Some collectors (including V8's, in parts) use hybrid orders to keep parents near children.
- Promotion: objects surviving a second scavenge are copied to old space rather than to-space (a different
freepointer, in old pages). The "age" is a mark bit per object or a per-page boundary. - Parallelism: multiple threads with their own to-space chunks and worklists; the forwarding pointer is installed with a compare-and-swap so that two threads reaching the same object copy it once.
- Roots from old space: the remembered set (the JS course part 5's write barrier) supplies old-to-young pointers; those slots are scanned and updated like roots. Without it the scavenger would have to scan all of old space.
- Pretenuring: an allocation site whose objects keep surviving is switched to allocate in old space directly, skipping the copies.
Mark-sweep and the tri-colour invariant
The old generation is too big to copy, so it is marked and swept: find everything reachable, then free everything else. Doing the marking while the program runs requires an invariant that the program can break and a barrier that repairs it. Three colours express the invariant; the write barrier enforces it; the result is a collector that runs concurrently and never frees a live object.
- Colours: white (not yet reached; presumed dead), grey (reached, children not yet examined), black (reached, children examined). Initially everything is white; the roots are greyed.
- Loop: pop a grey object; for each child that is white, grey it (push to the worklist); blacken the object. Terminate when no grey remains. White objects are unreachable.
- Complexity: O(reachable objects + reachable edges). The worklist is a stack per marking thread, with a bag for overflow.
- The hazard: the mutator stores a pointer to a white object into a black object and deletes the other paths to the white object. The marker never revisits black objects; the white object is never greyed; it is swept while still referenced.
- The invariant: no black object points to a white object ("strong tri-colour invariant"). If it holds at the end of marking, every white object is unreachable from black ones, and since all reachable objects are black, white objects are unreachable.
- The barrier (Dijkstra-style): on storing pointer
pinto objectoduring marking: ifpis white, grey it. That restores the invariant for any store, whatevero's colour. Alternatives: Yuasa's barrier greys the overwritten old value instead (preserving a snapshot of the graph at marking start); Steele's rechecks the black object. - Termination: helpers drain their worklists; the barrier may keep adding grey objects; a final stop-the-world phase drains the remaining grey set with the mutator paused, processes weak references, and flips the meaning of the colour bits so no clearing pass is needed.
// incremental marking as an algorithm: the same tri-colour marker, run in bounded steps, with the barrier keeping it honest
// state between steps: the grey worklist and the colour bits, which persist across mutator execution
function markStep(budgetBytes) {
let done = 0
while (grey.length && done < budgetBytes) { const o = grey.pop(); for (const c of children(o)) if (isWhite(c)) { setGrey(c); grey.push(c) } setBlack(o); done += size(o) }
return grey.length === 0
}
// the scheduler: after each task (or on allocation when the heap grows), run markStep with a budget proportional to allocation since the last step
// ("allocation-driven marking": the faster the program allocates, the more marking it must do, so marking keeps pace and the heap does not run away)
// concurrent marking: helper threads run markStep on their own worklists with work stealing; the barrier pushes to a per-thread buffer the marker drains
// finalisation: stop the world; drain all worklists; process weak references (ephemerons: a fixpoint over WeakMap entries); flip the colour sense
// the cost model: total marking work ∝ live heap; main-thread share ∝ what helpers could not do; pause ∝ what was left for finalisation- Sweep: walk each page's mark bitmap; runs of unmarked words become free-list entries. O(heap) in the worst case, done lazily per page on helper threads or when an allocator needs that page.
- Compact: for fragmented pages, move live objects to fresh pages and update every pointer to them (using the remembered set and a pass over the pages that reference them). A bounded number of pages per cycle, in parallel, with a pause.
- Weak references (ephemerons): a WeakMap entry's value is marked only if its key is; since marking a value can make another key reachable, the marker iterates to a fixpoint over the ephemeron set at finalisation. Many weak entries make that fixpoint measurable.
The engine's algorithms, in one table
| Problem | Algorithm / structure | Complexity | The frontend-visible consequence | JS course |
|---|---|---|---|---|
| Which shape is this object? | Trie of maps keyed by property additions | O(1) per added property; O(paths × depth) memory | Construction order is identity; conditional properties split shapes | P3 |
| Where is property x on this object? | Per-site inline cache (≤ 4 entries) → global stub cache (hash) | O(1) mono; O(k) poly; O(1) expected mega + full lookups on miss | Mono fast, poly fine, mega 10× slower; feedback belongs to the callee | P2, P3 |
| What types has this operation seen? | A lattice per feedback slot (None → SignedSmall → Number → … → Any) | O(1) join | Types only widen; a string once makes an add generic forever | P2 |
| When to optimise? | Interrupt budget counters (calls and back-edges); tiering thresholds | O(1) per event | Warm-up; OSR for loops; cold code stays interpreted | P0, P4 |
| Is this guess still valid? | Guards in code + dependency lists on maps, prototypes, cells (lazy deopt) | O(1) per guard; O(dependents) per invalidation | One prototype edit deopts every dependent function | P4 |
| How to go back to the interpreter? | Translation tables per deopt point; frame reconstruction | O(frame size) | Deopts are microseconds; deopt loops are the cost | P4 |
| Collect the young generation | Cheney copying, parallel, with promotion | O(live) | Allocation is cheap when objects die young | P5 |
| Collect the old generation | Tri-colour mark (incremental + concurrent, write barrier), lazy sweep, partial compaction | O(live) mark; O(heap) sweep; bounded compaction | Pauses are short and spread; mutation during marking costs a barrier | P5 |
| Find old → young pointers | Remembered set via the write barrier (card marking / slot sets) | O(1) per store; O(set) per scavenge | Mutating long-lived structures with young pointers costs something | P5 |
| Which allocations to pretenure? | Per-allocation-site survival statistics | O(1) per allocation | Sites that always survive skip the copies | P5 |
| Concatenate strings | Cons strings (rope) with lazy flattening | O(1) concat; O(n) first read | += in a loop is fine until the first read | P6 |
| Compare / hash strings | Internalisation (string table) + cached hashes | O(1) for internalised; O(n) otherwise | Dynamic keys cost a hash + table probe per access | P6 |
| Map / Set | Ordered hash table (buckets + entries + chains) | O(1) expected | Insertion order free; holes until rehash | P12 |
| Sort | TimSort | O(n log n); O(n) on runs | Stable; near-sorted input is nearly free | P14 here, P9 in this course |
| Regular expressions | Irregexp: backtracking compiled to native code (or bytecode), with fast paths; an experimental linear-time engine for simple patterns | Exponential worst case (ReDoS); linear for the non-backtracking subset | Catastrophic backtracking on crafted input; the l flag opts into linear | P9 of this course |
| Parse | Hand-written recursive descent + precedence climbing; pre-parser | O(n) | Startup is parse-bound; lazy parsing halves it | P1 |
| Optimise | Sea-of-nodes / Turboshaft graph rewrites; escape analysis; linear-scan register allocation | Roughly O(n log n) per function with budgets | Inlining budgets; large functions never optimised | P4; part 8 here |