Part 0 · 2 chapters · ~12 min
Background Jobs and Why They Exist
Moving slow, flaky and retryable work out of requests, 202 Accepted with status resources, what to enqueue (ids not objects), job priorities and separate queues, concurrency limits per job type, scheduled and delayed jobs, and choosing a job system per language.
1
Requests stay fast; work happens later
code
// enqueue ids, not objects: workers load fresh state and survive schema changes
await queue.add('submit-payout', { payoutId: p.id }, {
jobId: `submit-payout:${p.id}`, // dedupe: the same payout is never queued twice
attempts: 8, backoff: { type: 'exponential', delay: 2_000 },
priority: p.amountKobo > 100_000_000 ? 1 : 5,
});
res.status(202).location(`/v1/payouts/${p.id}`).json({ id: p.id, state: 'reserved' });WHY BACKGROUND JOBS
respond fast, do slow and retryable work later
swipe the figure sideways, or tap expand for full screen
1/4
the problem
Calling a bank rail inside the request takes seconds, sometimes times out, and ties up a server thread or connection. The user waits, and a retry may double-submit.
slow, flaky work inside requeststimeouts and double submits
2
Queues, priorities and limits
| practice | why |
|---|---|
| separate queues by urgency (critical, default, bulk) | a marketing blast must never delay OTP or payout jobs |
| concurrency limits per job type | protect partners and databases (no more than 20 concurrent rail calls) |
| job ids for deduplication | enqueueing twice does not run twice |
| small payloads (ids) | fresh data, small queues, no stale snapshots |
| timeouts per job | a hung job releases its slot and is retried |