Part 6 · 2 chapters · ~12 min

Concurrency and Multi-Core

The single-thread myth corrected, worker threads and the cost of structured clone, SharedArrayBuffer and Atomics for real shared state, cluster and port sharing, child processes and zombies, a decision matrix between workers, cluster, processes and containers, and thread-safe native addons.

Workers, messages and shared memory

code

// a small worker pool for CPU-bound work (password hashing)
import { Worker } from 'node:worker_threads';
import os from 'node:os';

class Pool {
  #idle = []; #queue = [];
  constructor(file, n = os.availableParallelism() - 1) {
    for (let i = 0; i < n; i++) this.#idle.push(new Worker(file));
  }
  run(task) {
    return new Promise((resolve, reject) => { this.#queue.push({ task, resolve, reject }); this.#next(); });
  }
  #next() {
    if (!this.#idle.length || !this.#queue.length) return;
    const w = this.#idle.pop(), { task, resolve, reject } = this.#queue.shift();
    const done = (fn, v) => { w.off('message', ok); w.off('error', bad); this.#idle.push(w); fn(v); this.#next(); };
    const ok = v => done(resolve, v), bad = e => done(reject, e);
    w.once('message', ok); w.once('error', bad); w.postMessage(task);
  }
}
// hash-worker.js: parentPort.on('message', p => parentPort.postMessage(scryptSync(p, salt, 64).toString('hex')))

code

// shared state between workers: a counter in a SharedArrayBuffer
const sab = new SharedArrayBuffer(4); const counter = new Int32Array(sab);
new Worker(new URL('./w.js', import.meta.url), { workerData: { sab } });
Atomics.add(counter, 0, 1);                      // atomic read-modify-write
Atomics.wait(counter, 0, 0, 100);                // sleep until it changes (not on the main thread in browsers)
Atomics.notify(counter, 0, 1);

The Computers course part 8 measured the costs on this machine: an atomic add is about 5.5 times a plain increment, and two workers writing adjacent slots run about 4.7 times slower than the same work 128 bytes apart. Message passing first; share memory only with a measured reason.

FOUR WAYS TO USE MORE CORES

worker threads, cluster, child processes and more containers

swipe the figure sideways, or tap expand for full screen

1/5

worker threads

A Worker is a new V8 isolate and a new libuv loop on a new OS thread inside the same process. It has its own heap; communication is by message passing (structured clone) or shared memory (SharedArrayBuffer).

new isolate + new loop + new thread, same processfor CPU-bound work: hashing, parsing, image work

cluster, child processes and the decision matrix

code

// cluster: one worker per core, restart on crash
import cluster from 'node:cluster'; import os from 'node:os'; import http from 'node:http';
if (cluster.isPrimary) {
  for (let i = 0; i < os.availableParallelism(); i++) cluster.fork();
  cluster.on('exit', (w, code) => { console.error(`worker ${w.process.pid} exited ${code}`); cluster.fork(); });
} else {
  http.createServer((req, res) => res.end(`pid ${process.pid}`)).listen(3000);
}

// child process: stream, time out, never leave zombies
import { spawn } from 'node:child_process';
const p = spawn('ffmpeg', ['-i', 'in.mp4', 'out.webm'], { stdio: ['ignore', 'pipe', 'pipe'], timeout: 60_000 });
p.on('exit', (code, signal) => console.log({ code, signal }));
process.on('SIGTERM', () => p.kill('SIGTERM'));

need	choose	why
CPU-heavy work inside a request (hashing, PDF, image resize)	worker pool (Piscina)	keeps the loop free; shares the process memory budget
more requests per second on a VM	cluster or PM2 cluster mode	one process per core, shared port
more requests per second on Kubernetes	more replicas, one process each	the orchestrator scales, restarts and limits memory per pod
run another program, or isolate untrusted work	child process	separate address space, any language
a native library that calls back from its own threads	Node-API ThreadSafeFunction	the only safe way to enter JS from another thread

SO_REUSEPORT: on Linux, several processes can bind the same port and let the kernel balance connections. Node's cluster does not use it by default (it uses round-robin in the primary); reusePort on server.listen exists in recent versions for platforms that support it.