Part 3 · 2 chapters · ~16 min

Configuration and Secrets

One artefact with configuration injected; deploy config, app config, flags and secrets in their own homes; layering by environment, region and tenant; promotion as a reviewed diff; GitOps and drift; then secrets at scale: distribution, dual-credential rotation, third-party keys, inventory and leak response.

Environments, promotion, GitOps and drift

one artefact, layered configuration, reviewed promotion

Build once and inject configuration. Frontends use runtime config instead of baked-in environment variables.
Each kind of configuration has a home: deploy config, app config, feature flags, secrets.
Layering: defaults, then environment, region and tenant, validated by a schema.
Promotion is a reviewed diff that moves through each environment the same way.
GitOps: git holds the desired state and a controller keeps clusters matching it.
Drift is detected, and a /version endpoint answers "what changed?" during an incident.

code

// config.ts: typed, validated, layered; a typo fails CI, not prod
import { z } from 'zod';

const Config = z.object({
  env: z.enum(['dev', 'staging', 'prod']),
  pspTimeoutMs: z.number().int().min(500).max(10_000),
  dailyCapMinor: z.record(z.enum(['NG', 'KE', 'GH']), z.number().int().positive()),
  ledgerUrl: z.string().url(),
});

export const config = Config.parse(deepMerge(
  defaults,                                   // in code
  loadYaml(`config/${process.env.APP_ENV}.yaml`),
  loadYaml(`config/${process.env.APP_ENV}.${process.env.REGION}.yaml`, { optional: true }),
));

the frontend version

A SPA built with VITE_API_URL baked in needs one build per environment, which means what passed staging is not what ships to prod. Serve /config.json from the deployment (or render config into the HTML on the server) and build once.

CONFIGURATION ACROSS ENVIRONMENTS

what changes between dev, staging and prod, where each kind of value lives, and how a change is promoted

swipe the figure sideways, or tap expand for full screen

1/6

one artefact

One artefact, many environments: the same image digest runs in dev, staging and prod; only configuration differs. Building per environment (a bundle with the prod API URL baked in) means what was tested is not what ships. For frontends, inject runtime config (a /config.json fetched at boot, or server-rendered values) instead of build-time env vars where environments differ.

Secrets at scale: distribution, rotation and response

every credential changes, nothing breaks

Distribution: the store is the source of truth and an operator syncs per namespace. Encrypt etcd.
Dual-credential rotation overlaps the old and new credentials, so no deploy has to be coordinated.
Consumers re-read on an interval or after an auth failure.
Third-party keys rotate with two active keys at once.
Keep an inventory with owners, policies and last-rotated dates.
Leak response should be a routine operation, practised every quarter.

code

# external-secrets: sync one secret into the payments namespace
apiVersion: external-secrets.io/v1
kind: ExternalSecret
metadata: { name: payments-db, namespace: prod }
spec:
  refreshInterval: 1h
  secretStoreRef: { name: aws-secrets, kind: ClusterSecretStore }
  target: { name: payments-db }
  data:
    - secretKey: password
      remoteRef: { key: prod/payments/db, property: password }

SECRETS AT SCALE: ROTATION WITHOUT A DEPLOY

syncing secrets into clusters, rotating them on a schedule, and responding when one leaks

swipe the figure sideways, or tap expand for full screen

1/6

distribution

Distribution: the External Secrets Operator (or the Secrets Store CSI driver) syncs secrets from the cloud store into Kubernetes Secrets or mounted files, per namespace, using the workload's identity. The secret store remains the source of truth; the cluster holds a cached copy with a refresh interval. Kubernetes Secrets are base64, not encrypted: enable envelope encryption of etcd with KMS.