Part 5 · 8 chapters · ~55 min

Memory And GC

Every object literal, closure, array and string is an allocation, and the program never frees anything. This part is the generational heap and why most allocations are nearly free, major collection done mostly off the main thread, the write barrier that makes mutation cost something, reading a heap snapshot in three moves, what allocation really costs and what escape analysis removes, the shapes of leaks and how to find each, the limits and what happens at them, and the tools.

The generational heap

the question

"If every object literal is a heap allocation, how is JavaScript not spending all its time in the garbage collector?"

Because most objects die almost immediately, and V8's heap is designed around that fact. New objects go to a small nursery where allocation is a pointer bump and collection copies only the survivors and discards the rest in bulk. Objects that live long enough are promoted to a large old generation collected rarely by a marking collector that runs mostly on other threads. The collector's name is Orinoco; the design is generational, incremental, concurrent and parallel.

the spaces

New space (young generation): two semi-spaces ("from" and "to") of 1 to 16 MB each depending on heap size and platform, plus the nursery/intermediate distinction (objects that have survived once are in the intermediate generation). Allocation: bump the top pointer. Collection: the scavenger (Cheney-style copying), parallel across threads.
Old space: survivors promoted after two scavenges, plus pretenured allocations. Pages of 256 KB. Collection: mark-sweep-compact, incremental and concurrent. The --max-old-space-size limit is this space.
Code space: compiled machine code, on executable pages (with write protection flipped during codegen).
Large object space: objects over the page-size threshold (~128 KB for regular, smaller for code) each get their own pages and are never moved; they are marked and freed in place.
Read-only space: the deserialised snapshot: built-in maps, canonical strings, root objects. Shared between isolates in some configurations; never collected.
Map space (merged into old space in recent versions): hidden classes.

the scavenge, step by step

Roots: the stack of every frame, global handles, the remembered set (old-to-young pointers recorded by the write barrier), and other roots.
Copy: each live object reachable from a root is copied to "to" space (first survival) or promoted to old space (second survival); a forwarding pointer is left in the old location so other references can be updated. The copied objects' own pointers are then scanned (Cheney's algorithm: the "to" space is itself the work queue).
Flip: "from" and "to" swap roles. The old "from" space, including every dead object, is reused without touching it.
Cost: proportional to live bytes in the nursery, which is usually a small fraction. Typical pauses: 0.1 to 2 ms. Parallel: several threads copy simultaneously.

run it

node --trace-gc -e "let a=[]; for(let i=0;i<5e6;i++){ a.push({i}); if(a.length>1e5) a=[] }". Each line is a collection: Scavenge with before → after sizes and the pause. Change the retention (keep 1e6 instead of 1e5) and watch the pauses grow, because more is live when each scavenge runs.

THE GENERATIONAL HEAP

young, old, and the spaces in between

swipe the figure sideways, or tap expand for full screen

1/6

young

The young generation: a nursery (semi-space "from") of a few megabytes (1 to 16 MB, sized by heap size and device) plus an intermediate space. Allocation is a pointer bump: the next object goes at the allocation top, no search. When the nursery fills, a scavenge runs.

Major GC: marking, sweeping, compacting, concurrently

The old generation cannot be copied wholesale; it is too big. It is collected by marking everything reachable, sweeping what is not, and occasionally compacting fragmented pages. The engineering is in doing nearly all of that while JavaScript keeps running.

the phases

Start (pause). Triggered by old-space growth past a limit computed from the live size after the last major GC and a growth factor that depends on allocation speed and available memory. Marks the roots.
Incremental and concurrent marking. Tri-colour: white (unvisited), grey (visited, children not yet), black (done). The main thread does marking slices between tasks; helper threads mark concurrently. The write barrier, during marking, catches stores of a white object into a black one and greys the target so the invariant holds and nothing live is missed.
Finalisation (pause). Drain the remaining grey work including barrier-flagged objects; process weak references: clear WeakMap entries whose keys are white, clear WeakRefs, schedule FinalizationRegistry cleanup jobs; process ephemerons (WeakMap values reachable only through their keys, handled with a fixpoint iteration).
Sweeping (concurrent). Walk pages, build free lists from unmarked regions. Lazy: a page may be swept on demand when the allocator needs it.
Compaction (parallel, with a pause). Choose pages with the most fragmentation; evacuate their live objects to free pages; update all pointers to the moved objects (using the remembered set and a full pointer-updating pass for the affected pages). Bounded by choosing few pages.

what the pauses are

Young: scavenge pauses, 0.1 to a few ms, frequent.
Old, start and finalise: short, a few ms each.
Old, compaction: bounded; a few ms typically.
Marking slices: not pauses exactly, but main-thread time interleaved with yours, visible in a profile as "GC" segments between tasks.
The bad case: a huge live heap (GBs) with heavy allocation during marking makes marking long and the finalise pause longer; a heap near its limit triggers back-to-back major GCs ("GC thrashing") and, past the limit, the fatal "heap out of memory".

run it

node --trace-gc --trace-gc-verbose prints each phase with timings. node --trace-gc-nvp gives name=value pairs per GC for scripts. In Chrome, Performance → the Main lane shows "Major GC" and "Minor GC" blocks with their durations, and the Memory checkbox adds the heap size line. A sawtooth is healthy; a staircase is a leak; a flat line at the limit with constant GC is a process about to die.

MARK, SWEEP, COMPACT, CONCURRENTLY

how a major GC avoids stopping the world for long

swipe the figure sideways, or tap expand for full screen

1/7

trigger: roots

Trigger: the old generation has grown past its limit (set relative to the live size after the last GC, times a growth factor). The collector begins: a short pause to mark the roots (stack, globals, handles, remembered sets).

Write barriers and the cost of mutation

Generational and concurrent collection both depend on the program telling the collector about pointer changes. The mechanism is the write barrier: extra instructions on every pointer store. It is cheap per store and invisible in most code, and it is the reason mutating old objects is not free.

what the barrier does

Generational part: on storing a pointer into object A, if A is in old space and the stored value is in young space, record A's slot in the remembered set. The scavenger scans the remembered set as roots, so it finds young objects referenced only from old ones without scanning old space.
Marking part: during an active marking phase, on storing a pointer, if the target is white (unmarked), mark it grey (or record it for the marker). This preserves the tri-colour invariant while the program mutates the graph under the marker.
Code shape: store; check "is A in old space and value in new space" (a page-flag test on both); if so, call the barrier stub that inserts into the remembered set. The fast path is a few instructions and two memory reads of page headers. Optimised code elides the barrier when it can prove the value is a Smi (not a pointer) or the object is young.

where it costs

Writing young pointers into old objects at a high rate: a long-lived array that is constantly appended with fresh objects, a cache that is refilled, a long-lived component tree that is re-rendered with new props objects. Each write is a barrier hit and a remembered-set entry; the next scavenge scans them all.
Building large structures: objects in a big graph being built get promoted (because the build takes longer than two scavenges), and then every further pointer store into them crosses generations.
Typed arrays, strings and numbers do not involve the barrier (no pointers stored). Numeric-heavy code is barrier-free, which is one reason struct-of-arrays layouts beat array-of-structs for hot numeric data.

worked numbers

the pattern to avoid, and the one to prefer:

  // old-space array, young objects, barrier on every push, remembered set grows
  const log = [];                                // promoted long ago
  onEvent(e => log.push({ t: Date.now(), e }))   // each push: barrier + the object is kept alive → promoted later

  // the same information in a typed ring buffer: no pointers, no barrier, no promotion
  const times = new Float64Array(N), codes = new Uint16Array(N); let head = 0;
  onEvent(e => { times[head] = Date.now(); codes[head] = code(e); head = (head + 1) % N })

not a rule for everything; a rule for the hot, long-lived, frequently mutated structure in a profile.

Reading a heap snapshot

A heap snapshot is a serialised copy of the object graph: every object, its size, its references. The DevTools Memory panel computes retained sizes, a dominator tree and retainer paths from it. Reading one is a skill with three moves: sort by retained size, compare two snapshots, read retainers.

the columns

Constructor: objects grouped by their constructor name (or type for internals: (array), (string), (closure), (system), (compiled code)). Object and Array are the big generic buckets; named classes are easier to attribute.
Distance: shortest path length from a root. Small numbers are near globals; "Detached" DOM trees have a distance but no path through the document.
Shallow size: the object's own bytes. Arrays and strings carry their contents here (as a separate (array) or (string) entry retained by the JS object, in some views).
Retained size: what would be freed if this object died: itself plus its exclusive dominees. The column to sort by.

the three moves

Sort by retained size in the Summary view. The top few rows are where the memory is. Expand one: its instances, each with retained size.
Compare. Snapshot before; perform the suspected leaking action N times; snapshot after; in the second, select "Objects allocated between Snapshot 1 and Snapshot 2". What is listed survived the interval. Constructor counts that equal N (or a multiple) are the leak.
Retainers. Select an instance; the bottom pane shows the chain of references from a root to it. Read upward until you find the edge that should not exist: a Map entry, a closure's context variable, an event listener's handler, a global, a detached node's parent pointer.

what you will find

Caches without eviction: a Map or object on a module scope with a growing count.
Listeners on long-lived targets: the retainer path goes through "(closure)" → a context → a component or node.
Closures over big scopes: a small handler whose context (shared with the scope's other closures) holds a large array.
Detached DOM: in a browser, nodes with "Detached" in the constructor column, retained by JS references.
Timers and subscriptions: intervals (the retainer shows the timer list), observers, store subscriptions.
Node-specific: native handles, Buffers pooled, streams not closed, promises never settled holding their reaction closures.

run it

Node: node --inspect app.js, open chrome://inspect, Memory → Take snapshot; or node --heapsnapshot-signal=SIGUSR2 then kill -USR2 <pid> to write a .heapsnapshot file to load in DevTools; or require('v8').writeHeapSnapshot() from code. Snapshots pause the process and can take seconds on a large heap; take them on a replica, not the only instance.

A HEAP SNAPSHOT, READ

shallow size, retained size, and the retainer path

swipe the figure sideways, or tap expand for full screen

1/6

reachability

The graph: GC roots (window or global, the stack, handles) at the top; objects below; edges are references. An object is alive if there is a path from a root to it. Nothing else matters: not "is it used", not "did I null the variable I thought held it".

Allocation: what it costs and what the compiler removes

An allocation in optimised code is a few instructions. The cost is elsewhere: in the scavenges a high allocation rate causes, in the copying of objects that live too long, and in the cache misses of touching fresh memory. The compiler removes allocations it can prove are temporary; the rest are yours to budget.

the fast path

Inline bump allocation: compare top + size against the limit; store the new top; write the map and initial fields. Four to ten instructions for a small object.
The slow path: on limit, call into the runtime, which runs a scavenge and retries, or routes large objects elsewhere.
Allocation folding: several allocations in a row in optimised code are folded into one bump of the combined size.
Pretenuring: an allocation site whose objects keep surviving scavenges (tracked via the AllocationSite in the feedback) is switched to allocate directly in old space, skipping two copies. Automatic; visible in --trace-pretenuring.

the real costs

Scavenge frequency = allocation rate / nursery size. 100 MB/s with an 8 MB nursery is 12 scavenges a second. Each costs the copying of whatever is live at that moment.
Promotion of medium-lived objects: copied twice, then subject to the write barrier and the major collector. The worst lifetime is "longer than two scavenges, shorter than the process".
Cache and memory bandwidth: every allocated byte is a byte written to fresh memory, evicting cache lines of data you were using.
Large objects: arrays and strings over the threshold go straight to large object space: no copy, but a page each and a cost to free.

escape analysis

What it removes: allocations in a function (including inlined callees) whose result does not escape: not stored into a heap object, not returned to a non-inlined caller, not passed to a non-inlined call, not captured by a closure. The object becomes a set of local values; its fields are registers.
What qualifies in practice: small result objects from inlined helpers ({x, y}, [a, b] returned and destructured), iterator result objects from inlined iterators ({value, done} in a for-of over an array), temporary arrays in inlined builtins, arguments objects, closures created and called within the function.
What does not: anything pushed, stored, returned to the outside, or captured. And anything in a function that is not optimised, which is most functions.
The lesson: write the natural code with small helper objects; in hot optimised paths they are usually free. Measure before contorting code to avoid them. When the profile shows allocation in a hot loop, the fix is usually a structural one (reuse a buffer, a struct-of-arrays layout, a typed array), not micro-avoidance of literals.

run it

DevTools Memory → "Allocation sampling" (low overhead, by function) or "Allocation instrumentation on timeline" (every allocation, with stacks). Node: --heap-prof writes a sampling profile. The function at the top of an allocation profile in a steady-state service is the one to look at with --trace-turbo-escape.

WHAT ALLOCATION COSTS

bump pointer, scavenge pressure, and escape analysis

swipe the figure sideways, or tap expand for full screen

1/6

fast path

The fast path: the allocation top pointer is in a register or a known memory slot. new object = check (top + size <= limit), store, top += size. Inline, three or four instructions, no call. Then initialise the fields (the map, the empty properties and elements, each field).

Leaks: shapes, detection, fixes

A leak in a garbage-collected language is a reference that should have been dropped. The collector cannot tell the difference between "still needed" and "forgotten"; only you can. The shapes are few and the detection method is the same for all of them.

the shapes

Unbounded collections. A Map, Set, array or object used as a cache, registry, log or index, with no eviction. Correct references, infinite growth. Fix: a size cap with LRU eviction, a TTL, WeakMap where the key is an object whose lifetime should govern the entry.
Listeners and subscriptions. emitter.on, addEventListener on a long-lived target, store subscriptions, observers. The target keeps the handler; the handler's closure keeps everything it references. Fix: unsubscribe on teardown; AbortSignal for listeners; weak listener patterns.
Timers. setInterval never cleared, recursive setTimeout chains. Fix: clear on teardown; use an abortable loop.
Closures over large scopes. A small, long-lived closure sharing a context with a large, short-lived one. Fix: narrow the scope (create the small closure in its own function so its context does not include the big variable), or null the big variable when done.
Detached DOM (browser). Nodes removed from the document but referenced from JS. Fix: drop the references; WeakRef for optional handles.
Pending promises. A promise that never settles keeps its reaction closures and everything they capture. A request that never times out, an await on a stream that never ends. Fix: timeouts; AbortController.
Native resources (Node). Open file descriptors, sockets, streams not destroyed, native addon handles. Not GC-managed; a leak of the resource plus of the JS wrapper. Fix: explicit close; using declarations and Symbol.dispose where available.
Module-level state in long-running processes. Anything at module scope lives forever. Fine for constants; a trap for "temporary" accumulators.

detection, in order

Confirm growth. Memory over time, after forced GCs (DevTools' trash-can button, or --expose-gc and global.gc() in Node). Growth that survives GC is retention.
Find the constructor. Three snapshots; compare; the constructor whose count tracks the repetitions.
Read retainers. The path from a root names the collection, the listener, the closure, or the timer.
Cut the edge. One line, usually: a delete, an off, a clearInterval, a scope change.
Verify with the same three-snapshot procedure.

what is not a leak

A sawtooth that returns to the same baseline after each GC. A heap that grows for the first minutes of a process while caches and JIT code warm up, then plateaus. A big heap that is big because the data is big. Retention that does not grow with time or with repetitions of an action is a design choice, not a leak.

Limits, tuning, and out of memory

the limits

Old space: --max-old-space-size=N in MB. Default depends on the Node version, platform and system memory: roughly 2 GB on 64-bit historically, now scaled with available memory up to 4 GB or more in recent versions. v8.getHeapStatistics().heap_size_limit reports the effective value.
Young space: --max-semi-space-size=N in MB (per semi-space). Default 16 MB on 64-bit. Raising it reduces scavenge frequency and lets more objects die young, at the cost of memory and longer individual scavenges; a common tuning for allocation-heavy servers (32 or 64 MB).
Per isolate: every worker has its own limits, settable via resourceLimits in new Worker().
Containers: V8 reads system memory, not the cgroup limit, in some versions; set the flags explicitly in containers or the default may exceed the container's RAM and the process is OOM-killed by the kernel before V8's own limit triggers.

out of memory

What happens: a major GC fails to free enough; V8 retries with more aggressive settings (a "last resort" GC); if still over the limit, it aborts the process with "FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory" and a trace of the last GCs. Not catchable.
Before it: the symptom is GC thrashing: back-to-back major GCs each freeing little, CPU pinned on GC threads, throughput collapsing. --trace-gc shows mark-compacts a few hundred ms apart with "last resort" annotations.
Diagnosis: a heap snapshot before the limit (--heapsnapshot-near-heap-limit=N writes snapshots automatically as the heap approaches the limit), then the leak procedure.
Mitigation: raise the limit only if the data is legitimately that large; otherwise fix the retention. Raising the limit on a leak buys time, not a fix.

observing in production

process.memoryUsage(): rss, heapTotal, heapUsed, external, arrayBuffers. heapUsed over time is the leak signal; external and arrayBuffers are off-heap memory (Buffers, typed array backing stores) that the JS heap limit does not cover.
v8.getHeapStatistics(), v8.getHeapSpaceStatistics(): per-space sizes; used to see which space is growing.
perf_hooks GC observer: PerformanceObserver with entryType 'gc' gives each collection's kind and duration; export as metrics.
In the browser: performance.measureUserAgentSpecificMemory() (cross-origin isolated pages) for the real number; performance.memory (Chrome, imprecise) otherwise.

the pointer

Part 6 is how values are represented in that heap: why a small integer is not an allocation, why a string concatenation is, and what a "heap number" costs. Half of allocation-rate problems are numbers that became boxed and strings that became ropes.

Reading memory: tools

Tool	Shows	Use it for
`--trace-gc` / `--trace-gc-verbose` / `--trace-gc-nvp`	Each collection: kind, sizes before and after, pause, phases	Frequency and cost of GC; thrashing; scavenge pressure
`--expose-gc` + `global.gc()`	Forces a full GC	Measuring retention without waiting; before snapshots
DevTools Memory → Heap snapshot (Summary, Comparison, Containment, Statistics views)	The graph with shallow and retained sizes, dominators, retainers	What is retained and by whom; the leak procedure
DevTools Memory → Allocation instrumentation on timeline	Allocations over time with stacks; blue bars for still-live	Which code path allocates what survives
DevTools Memory → Allocation sampling / `node --heap-prof`	Sampled allocations by function, low overhead	Allocation hot spots in a steady-state workload
DevTools Performance monitor / Performance panel with Memory checked	Heap size over time; GC events in the Main lane	Sawtooth versus staircase; GC pauses next to jank
`node --heapsnapshot-signal=SIG`, `v8.writeHeapSnapshot()`, `--heapsnapshot-near-heap-limit`	Snapshot files from a running process	Production diagnosis without an attached debugger
`process.memoryUsage()`, `v8.getHeapSpaceStatistics()`, perf_hooks `gc` entries	Numbers to export as metrics	Dashboards; alerts on heapUsed slope and GC time share
`--trace-pretenuring`, `--trace-turbo-escape`	Pretenuring decisions; escape analysis results per allocation	Why an allocation was or was not eliminated
`%DebugPrint(obj)`, `%GetHeapUsage()` (natives syntax)	Object internals; current heap usage	Checking representation and size of one object
Chrome `chrome://memory-internals`, Task Manager (Shift+Esc) with JavaScript memory column	Per-renderer and per-site memory, including GPU and external	The whole tab's memory, not only the JS heap

run it, the whole procedure once

Write a server with a deliberate leak (an array at module scope pushed on every request). Load it; watch --trace-gc show scavenges promoting more each time; take two snapshots 1,000 requests apart; compare; read the retainer path to the module-scope array. Then fix it and watch the sawtooth return. Twenty minutes, and you have done the thing most engineers only read about.