Part 3 · 1 chapters · ~8 min

Shadow Traffic, Dark Launches and Flag Releases

Request mirroring and response comparison, Scientist-style experiments in code, the side-effect rule for shadows, dark launches as production load tests, and separating deploy from release with feature flags.

4

Running new code where nobody can see it

code
# Istio: mirror 100% of traffic to v2, serve from v1
http:
- route: [ { destination: { host: ledger, subset: v1 }, weight: 100 } ]
  mirror: { host: ledger, subset: v2 }
  mirrorPercentage: { value: 100.0 }

# in code: compare an old and new implementation on the same input (Scientist pattern)
const result = await experiment('fee-calc-v2', {
  control: () => feeV1(transfer),          // returned to the caller
  candidate: () => feeV2(transfer),        // run, compared, discarded
  compare: (a, b) => a.kobo === b.kobo,    // mismatches logged with inputs
});

Deploy is not release. With flags (Config course part 6), code reaches production switched off, then turns on for staff, then 1%, then everyone, with a kill switch. Rollback of a feature becomes a flag flip instead of a deploy.

SHADOW TRAFFIC AND DARK LAUNCHES
run the new path on real traffic without anyone seeing its result
real requestproduction pathv1, response returnedmirrorcopy of the requestnew pathv2, response discardedcomparatordiff results, latency
swipe the figure sideways, or tap expand for full screen
1/5
mirroring
A proxy (Envoy request mirroring, Istio mirror) copies each request to the new version asynchronously. The user only ever gets the production response.
copy real requests to the new versionusers never see the shadow response