Part 6 · 5 chapters · ~40 min

V8's Algorithms

Under every mechanism the JS course drew there is a named algorithm: a trie for shapes, a tiny dispatch table for property access, a copying collector for the young, a tri-colour marker with a write barrier for the old. This part names each, gives its complexity and the data structure that makes it work, and ends with every engine algorithm in one table with its frontend-visible consequence.

35

The transition tree as a trie

the question

"Hidden classes: the JS course said construction order decides identity. What is the data structure that makes that true?"

A trie. The root is the empty object's map; each edge is a (property name, attributes) pair; each node is the map an object has after adding that sequence of properties. Constructing an object by assignment is a walk down the trie, creating nodes on first use and following existing edges after. Two objects that take the same path share the leaf; two that take different paths do not, whatever properties they end up with. The JS course (part 3) is what the nodes contain; this chapter is the structure over them and its costs.

operations
  1. Add property (the walk step): look up (name, attributes) in the current map's transition array. Hit: move to the child. Miss: allocate a child map (extend the descriptor array; assign the next slot), insert the edge, move. The transition array is a small sorted array searched by binary search, promoted to a hash table when a map accumulates many children.
  2. Object literal: no walk at runtime; the parser precomputed the leaf map for the literal's shape (its "boilerplate"), so creation is an allocation with that map plus field copies.
  3. Delete: no edge for removal; the object leaves the trie for dictionary mode.
  4. Reconfigure (defineProperty on an existing property, freeze): special transitions keyed by the operation, also in the trie.
  5. Field generalisation: when a field's representation widens (Smi to Double), the map and all its descendants are deprecated and a parallel subtree with the wider representation is created; live objects migrate lazily on their next access (the JS course part 4's "deprecated map" deopt).
costs
  1. Per property addition: O(log k) for k siblings in the transition array (one or two compares typically), plus the allocation on a miss.
  2. Memory: O(number of distinct construction paths × depth). A class constructed one way is depth nodes total; constructed N ways is N × depth. Shape explosion is a trie with many branches.
  3. Sharing: descriptor arrays are shared along a chain (a child extends its parent's), so the trie's memory is less than nodes × properties.
  4. Pruning: maps unreachable from live objects and from their parents' transition arrays (held weakly) are collected; an identical construction later rebuilds the branch with new identities, and inline caches that remembered the old maps miss once.
the algorithmic lesson
A trie turns "has this sequence been seen before" into a walk of O(sequence length) with O(1) per step. The same structure answers autocomplete (prefixes of words), routing (prefixes of paths), and token tables; V8 uses it for prefixes of property sequences.
THE TRANSITION TREE AS A TRIE
lookup by property-addition path
swipe the figure sideways, or tap expand for full screen
1/6
root
The root: the initial map for objects with a given prototype and in-object capacity. Its transition array is empty until the first property is added anywhere in the program.
36

Inline cache lookup

code
// inline cache lookup, as the algorithm it is: a tiny dispatch table keyed by map, specialised per call site
// state: uninitialised | monomorphic (1 entry) | polymorphic (2..4 entries) | megamorphic (global table)
function loadIC(site, obj) {                      // what the LoadIC handler does, in JS terms
  const map = obj.map
  for (const [m, handler] of site.entries)        // 1 compare when monomorphic; up to 4 when polymorphic
    if (m === map) return handler(obj)            // handler: "load slot 2", "load from prototype P", "call getter"
  // miss:
  const handler = fullLookup(obj, site.name)      // walk descriptors and the prototype chain: O(depth × properties)
  if (site.entries.length < 4) site.entries.push([map, handler])
  else site.state = 'megamorphic'                 // from now on: stubCache[hash(map, name)], a global 2-way set-associative table
  return handler(obj)
}
// complexity per access: mono O(1), poly O(k ≤ 4), mega O(1) expected via the hash + a full lookup on a stub cache miss
// the data structure choice: a per-site array (tiny, inlined into code) vs a global hash (shared, never inlined). the per-site array wins
// while it stays small; the global table is the fallback. the JS course part 3 has the machine code each state compiles to
the structure
  1. Per call site, a tiny table: up to four (map, handler) pairs, stored in the feedback vector slot (the JS course part 2). The handler is a small encoded recipe: "in-object slot 2", "properties array index 5", "on prototype P at depth 1, constant F", "call this getter".
  2. Lookup: compare the receiver's map against each entry. Monomorphic: one compare. Polymorphic: a linear scan of two to four. The table is small enough to be inlined into generated code as a compare chain (part 3 of the JS course has the machine code).
  3. Miss: a full lookup (descriptor array search on the map, then the prototype chain), then insert the result into the site's table. The fifth distinct map tips the site to megamorphic.
  4. Megamorphic: the site stops tracking maps; lookups go to the global stub cache, a fixed-size two-way set-associative hash table keyed by (map, name) → handler, shared by all megamorphic sites. A hit is a hash and a compare; a miss is a full lookup and an insertion that may evict another entry.
why these sizes
  1. Four entries: a linear scan of four compares is cheaper than a hash; beyond four, a hash wins, and the global table amortises its memory across sites. The number is a tuned constant, not a law.
  2. Per-site tables beat a global one while they fit because the compare chain is inlined (no call, no hash) and the compiler can hoist it; the global table is a call and can be evicted by other sites' entries.
  3. Handlers are data, not code, so a polymorphic site does not need four code paths; the optimiser, when it compiles the site, turns the table into code.
the same pattern elsewhere
  1. Dynamic dispatch caches in every dynamic language runtime (Smalltalk invented inline caches; Self made them polymorphic; V8 inherited both).
  2. Branch prediction in CPUs: a per-site history that predicts the next outcome; wrong predictions are the deopt.
  3. Memoising a dispatch in your own code: a small array of (key, result) for the last few keys at a hot call site is often faster than a Map lookup, for exactly the reasons above.
37

Generational collection: Cheney's copying scavenger

The young generation is collected by copying: live objects are moved to a fresh space and everything left behind is garbage. Cheney's algorithm does this breadth-first with no recursion and no separate worklist, using the destination space itself as the queue. Its cost is proportional to the live data, which under the generational hypothesis is a small fraction of the space.

the algorithm
  1. Two pointers into to-space: scan (the next copied object to process) and free (where the next copy goes). Both start at the beginning.
  2. Copy the roots' targets: for each root pointer into from-space: copy the object to free, advance free, overwrite the old object's header with a forwarding pointer to the copy, update the root.
  3. Scan loop: while scan < free: take the object at scan; for each pointer field referencing from-space: if the target is already forwarded, update the field to the forwarding address; else copy it (as in step 2) and update. Advance scan past the object.
  4. Flip: when scan == free, everything reachable has been copied and every pointer updated. The from-space is now entirely garbage; swap the roles of the spaces. No sweeping, no free lists.
properties
  1. Time O(live): dead objects are never visited. Allocation-heavy, short-lived workloads pay almost nothing per collection.
  2. Compacting by construction: survivors are packed contiguously in to-space, in breadth-first order, which also improves locality for objects that reference each other.
  3. Space: two semi-spaces, so half the young generation is idle at any time. The price of simplicity; the young generation is small, so it is cheap.
  4. Breadth-first order is a side effect of using to-space as the queue; depth-first would need a stack. Some collectors (including V8's, in parts) use hybrid orders to keep parents near children.
V8's additions
  1. Promotion: objects surviving a second scavenge are copied to old space rather than to-space (a different free pointer, in old pages). The "age" is a mark bit per object or a per-page boundary.
  2. Parallelism: multiple threads with their own to-space chunks and worklists; the forwarding pointer is installed with a compare-and-swap so that two threads reaching the same object copy it once.
  3. Roots from old space: the remembered set (the JS course part 5's write barrier) supplies old-to-young pointers; those slots are scanned and updated like roots. Without it the scavenger would have to scan all of old space.
  4. Pretenuring: an allocation site whose objects keep surviving is switched to allocate in old space directly, skipping the copies.
CHENEY'S COPYING COLLECTOR
the scavenger as a breadth-first copy
swipe the figure sideways, or tap expand for full screen
1/6
two spaces
From-space holds 8 objects; the roots reference A and C. To-space is empty; two pointers into it: scan (next object to process) and free (next allocation address), both at the start.
38

Mark-sweep and the tri-colour invariant

The old generation is too big to copy, so it is marked and swept: find everything reachable, then free everything else. Doing the marking while the program runs requires an invariant that the program can break and a barrier that repairs it. Three colours express the invariant; the write barrier enforces it; the result is a collector that runs concurrently and never frees a live object.

marking, sequentially
  1. Colours: white (not yet reached; presumed dead), grey (reached, children not yet examined), black (reached, children examined). Initially everything is white; the roots are greyed.
  2. Loop: pop a grey object; for each child that is white, grey it (push to the worklist); blacken the object. Terminate when no grey remains. White objects are unreachable.
  3. Complexity: O(reachable objects + reachable edges). The worklist is a stack per marking thread, with a bag for overflow.
concurrently
  1. The hazard: the mutator stores a pointer to a white object into a black object and deletes the other paths to the white object. The marker never revisits black objects; the white object is never greyed; it is swept while still referenced.
  2. The invariant: no black object points to a white object ("strong tri-colour invariant"). If it holds at the end of marking, every white object is unreachable from black ones, and since all reachable objects are black, white objects are unreachable.
  3. The barrier (Dijkstra-style): on storing pointer p into object o during marking: if p is white, grey it. That restores the invariant for any store, whatever o's colour. Alternatives: Yuasa's barrier greys the overwritten old value instead (preserving a snapshot of the graph at marking start); Steele's rechecks the black object.
  4. Termination: helpers drain their worklists; the barrier may keep adding grey objects; a final stop-the-world phase drains the remaining grey set with the mutator paused, processes weak references, and flips the meaning of the colour bits so no clearing pass is needed.
code
// incremental marking as an algorithm: the same tri-colour marker, run in bounded steps, with the barrier keeping it honest
// state between steps: the grey worklist and the colour bits, which persist across mutator execution
function markStep(budgetBytes) {
  let done = 0
  while (grey.length && done < budgetBytes) { const o = grey.pop(); for (const c of children(o)) if (isWhite(c)) { setGrey(c); grey.push(c) } setBlack(o); done += size(o) }
  return grey.length === 0
}
// the scheduler: after each task (or on allocation when the heap grows), run markStep with a budget proportional to allocation since the last step
//   ("allocation-driven marking": the faster the program allocates, the more marking it must do, so marking keeps pace and the heap does not run away)
// concurrent marking: helper threads run markStep on their own worklists with work stealing; the barrier pushes to a per-thread buffer the marker drains
// finalisation: stop the world; drain all worklists; process weak references (ephemerons: a fixpoint over WeakMap entries); flip the colour sense
// the cost model: total marking work ∝ live heap; main-thread share ∝ what helpers could not do; pause ∝ what was left for finalisation
sweeping and compaction, briefly
  1. Sweep: walk each page's mark bitmap; runs of unmarked words become free-list entries. O(heap) in the worst case, done lazily per page on helper threads or when an allocator needs that page.
  2. Compact: for fragmented pages, move live objects to fresh pages and update every pointer to them (using the remembered set and a pass over the pages that reference them). A bounded number of pages per cycle, in parallel, with a pause.
  3. Weak references (ephemerons): a WeakMap entry's value is marked only if its key is; since marking a value can make another key reachable, the marker iterates to a fixpoint over the ephemeron set at finalisation. Many weak entries make that fixpoint measurable.
TRI-COLOUR MARKING WITH A WRITE BARRIER
why concurrent marking never loses a live object
swipe the figure sideways, or tap expand for full screen
1/6
colours
Start: roots are grey; everything else is white. The marker pops a grey object, marks its children grey, and colours it black. Worklist: the grey set (a stack or a bag per thread).
39

The engine's algorithms, in one table

ProblemAlgorithm / structureComplexityThe frontend-visible consequenceJS course
Which shape is this object?Trie of maps keyed by property additionsO(1) per added property; O(paths × depth) memoryConstruction order is identity; conditional properties split shapesP3
Where is property x on this object?Per-site inline cache (≤ 4 entries) → global stub cache (hash)O(1) mono; O(k) poly; O(1) expected mega + full lookups on missMono fast, poly fine, mega 10× slower; feedback belongs to the calleeP2, P3
What types has this operation seen?A lattice per feedback slot (None → SignedSmall → Number → … → Any)O(1) joinTypes only widen; a string once makes an add generic foreverP2
When to optimise?Interrupt budget counters (calls and back-edges); tiering thresholdsO(1) per eventWarm-up; OSR for loops; cold code stays interpretedP0, P4
Is this guess still valid?Guards in code + dependency lists on maps, prototypes, cells (lazy deopt)O(1) per guard; O(dependents) per invalidationOne prototype edit deopts every dependent functionP4
How to go back to the interpreter?Translation tables per deopt point; frame reconstructionO(frame size)Deopts are microseconds; deopt loops are the costP4
Collect the young generationCheney copying, parallel, with promotionO(live)Allocation is cheap when objects die youngP5
Collect the old generationTri-colour mark (incremental + concurrent, write barrier), lazy sweep, partial compactionO(live) mark; O(heap) sweep; bounded compactionPauses are short and spread; mutation during marking costs a barrierP5
Find old → young pointersRemembered set via the write barrier (card marking / slot sets)O(1) per store; O(set) per scavengeMutating long-lived structures with young pointers costs somethingP5
Which allocations to pretenure?Per-allocation-site survival statisticsO(1) per allocationSites that always survive skip the copiesP5
Concatenate stringsCons strings (rope) with lazy flatteningO(1) concat; O(n) first read+= in a loop is fine until the first readP6
Compare / hash stringsInternalisation (string table) + cached hashesO(1) for internalised; O(n) otherwiseDynamic keys cost a hash + table probe per accessP6
Map / SetOrdered hash table (buckets + entries + chains)O(1) expectedInsertion order free; holes until rehashP12
SortTimSortO(n log n); O(n) on runsStable; near-sorted input is nearly freeP14 here, P9 in this course
Regular expressionsIrregexp: backtracking compiled to native code (or bytecode), with fast paths; an experimental linear-time engine for simple patternsExponential worst case (ReDoS); linear for the non-backtracking subsetCatastrophic backtracking on crafted input; the l flag opts into linearP9 of this course
ParseHand-written recursive descent + precedence climbing; pre-parserO(n)Startup is parse-bound; lazy parsing halves itP1
OptimiseSea-of-nodes / Turboshaft graph rewrites; escape analysis; linear-scan register allocationRoughly O(n log n) per function with budgetsInlining budgets; large functions never optimisedP4; part 8 here
the pointer
Part 7 is reconciliation: React's diff as an O(n) heuristic over a tree, step by step, with the keyed algorithm from part 1 inside it, and what Vue, Svelte and Solid do instead. The engine's algorithms are below your code; the reconciler's are the first layer above it.