Part 1 · 2 chapters · ~20 min

M1: Redis

A Redis that redis-cli can talk to: RESP and an incremental parser for split and pipelined reads, a single-threaded loop that makes commands atomic, lazy and sampled expiry, pub/sub, and an append-only file with absolute deadlines and fsync policies.

How Redis works

an event loop over a map, a protocol, and a log

RESP: arrays of length-prefixed bulk strings, which makes the protocol binary-safe.
An incremental parser handles split and pipelined reads.
Single-threaded execution makes every command atomic, so slow commands hurt everyone.
Expiry is lazy on read plus sampled active cleanup.
The AOF stores absolute deadlines, has an fsync policy, and is replayed on start.
Pub/sub pushes on the same connection and is at-most-once.

real Redis has	tiny-redis	why it matters
an ae event loop over epoll/kqueue	Node's event loop	the same single-threaded model
RESP2 and RESP3	RESP2 plus inline commands	redis-cli and every client library talk to it
dozens of data types and encodings (listpack, quicklist, skiplist for sorted sets)	strings, lists, hashes on JS structures	encodings are a memory optimisation layered on the same commands
RDB snapshots via fork() and copy-on-write, plus the AOF	the AOF with rewrite	fork-based snapshots are an exercise
replication, Sentinel, Cluster with 16,384 hash slots	none	the Distributed Systems course parts 3 and 4
I/O threads (6.0+) for socket reads and writes	none	execution stays single-threaded even then

TINY-REDIS: BYTES TO A REPLY

a command arriving in pieces over TCP, parsed incrementally, executed atomically, persisted, and answered

swipe the figure sideways, or tap expand for full screen

1/6

RESP request

The client sends SET balance:acct_7 1500 EX 60 as a RESP array of bulk strings: *5\r\n$3\r\nSET\r\n$14\r\nbalance:acct_7\r\n... Each bulk string is length-prefixed, so values can contain any bytes, including \r\n.

Building it: tiny-redis

Repo: repos/redis. Five source files, about 400 lines. Start with the parser, because everything depends on it handling partial input.

code

// src/resp.ts: feed() returns every complete value; incomplete data waits for the next chunk
feed(chunk: Buffer): RespValue[] {
  this.buf = Buffer.concat([this.buf, chunk]);
  const out: RespValue[] = [];
  for (;;) {
    const r = this.parse(0);
    if (!r) break;                    // incomplete: keep the bytes, wait for more
    out.push(r.value);
    this.buf = this.buf.subarray(r.next);
  }
  return out;
}
// …
case '$': {
  const len = Number(l.text);
  if (len === -1) return { value: null, next: l.next };
  if (this.buf.length < l.next + len + 2) return null;          // the bulk string is not all here yet
  return { value: this.buf.toString('utf8', l.next, l.next + len), next: l.next + len + 2 };
}

code

// src/store.ts: active expiry, bounded work per tick
activeExpireCycle(sample = 20): number {
  let removed = 0;
  for (;;) {
    const keys = [...this.expires.keys()];
    if (keys.length === 0) return removed;
    let expired = 0;
    for (let k = 0; k < Math.min(sample, keys.length); k++) {
      const key = keys[Math.floor(Math.random() * keys.length)];
      if (!this.alive(key)) expired++;               // alive() deletes it if expired
    }
    removed += expired;
    if (expired <= sample / 4) return removed;      // few expired in the sample: stop for this tick
  }
}

code

// src/aof.ts: a relative TTL is stored as an absolute deadline, so a replay later is correct
if (cmd === 'SET') {
  const i = args.findIndex(x => /^(EX|PX)$/i.test(x));
  if (i > 0) {
    const ms = /^EX$/i.test(args[i]) ? Number(args[i + 1]) * 1000 : Number(args[i + 1]);
    rec = [...args.slice(0, i), 'PXAT', String(now + ms), ...args.slice(i + 2)];
  }
}

Run it. In repos/redis: npm test, then npm start and in another terminal redis-cli -p 6380. Try SET balance:acct_7 1500 EX 30, TTL balance:acct_7, SUBSCRIBE prices in one client and PUBLISH prices "USDNGN 1530" in another. Stop the server, start it again, and GET the key: the AOF replayed it with its remaining TTL.

exercises

1. MULTI/EXEC: queue commands per connection and run them back to back (atomic, because the loop is single-threaded). 2. BLPOP with a list of waiting clients, woken by LPUSH. 3. RDB snapshots: serialise a frozen copy of the keyspace while serving writes. 4. Replication: stream the AOF to a replica and have it apply each command. Each is a page of code once you see where it belongs.