Part 8 · 2 chapters · ~18 min

Transactions Across Services

Two-phase commit with prepare, commit and the blocking problem; sagas as local transactions with compensations, orchestration versus choreography, isolation and pivot points; then the transactional outbox and inbox for exactly-once effect, and drawing service boundaries so you need none of it.

Two-phase commit and sagas

atomic and blocking, or compensating and visible

2PC prepare: every participant prepares durably and votes. A yes is a promise.
2PC commit: the decision is logged first, then every participant applies it.
Blocking: if the coordinator is lost, participants sit in doubt with locks held.
Sagas are local transactions with compensations that run in reverse.
Orchestrate sagas with more than three steps.
The costs: no isolation (so use semantic locks), and pivot points where you can only retry forward.

code

// a transfer saga as a Temporal workflow: durable state, retries, compensations
export async function transferSaga(t: Transfer) {
  const compensations: (() => Promise<void>)[] = [];
  try {
    await wallet.debit(t.from, t.amount, { idem: `${t.id}:debit` });
    compensations.unshift(() => wallet.refund(t.from, t.amount, { idem: `${t.id}:refund` }));

    await payouts.reserve(t.to, t.amount, { idem: `${t.id}:reserve` });
    compensations.unshift(() => payouts.release(t.to, { idem: `${t.id}:release` }));

    await fees.record(t, { idem: `${t.id}:fee` });
    await payouts.send(t.to, { idem: `${t.id}:send` });     // pivot: from here, retry forward only
  } catch (e) {
    if (isAfterPivot(e)) throw e;                            // Temporal retries the workflow forward
    for (const c of compensations) await c();                // each idempotent, retried until done
    throw e;
  }
}

TWO-PHASE COMMIT AND SAGAS

atomic commit across participants with a coordinator, versus a sequence of local transactions with compensations

swipe the figure sideways, or tap expand for full screen

1/6

2PC prepare

Two-phase commit, phase one (prepare): the coordinator asks every participant "can you commit?". Each participant does the work, writes it durably in a prepared state (locks held), and votes yes or no. A yes is a promise: it can commit later no matter what.

The outbox, and when to avoid all of this

events if and only if the change committed

The dual write loses or invents events, whichever order you choose.
The outbox writes the event in the same local transaction as the change.
A relay publishes the events at least once.
Ordering: key by aggregate id. CDC preserves commit order.
The inbox on the consumer side completes exactly-once effect.
Avoid it where you can, by keeping data that must change atomically in one service and one database.

code

-- the outbox table and the transaction that writes it
CREATE TABLE outbox (
  id           bigserial PRIMARY KEY,
  event_id     uuid NOT NULL UNIQUE,
  aggregate_id text NOT NULL,
  type         text NOT NULL,
  payload      jsonb NOT NULL,
  created_at   timestamptz NOT NULL DEFAULT now(),
  sent_at      timestamptz
);

BEGIN;
UPDATE wallets SET balance_kobo = balance_kobo - 5000000 WHERE id = 'w_7' AND balance_kobo >= 5000000;
INSERT INTO outbox (event_id, aggregate_id, type, payload)
VALUES (gen_random_uuid(), 'w_7', 'WalletDebited', '{"amount_kobo": 5000000, "transfer_id": "tr_91a"}');
COMMIT;

the first question

Before choosing between 2PC, a saga or an outbox, ask whether the data really needs to live in separate services. A wallet, its holds and its ledger entries that must change together belong in one service and one database, where an ordinary transaction does the job. Draw service boundaries around transactional consistency, not around nouns.

THE TRANSACTIONAL OUTBOX

publishing an event if and only if the database change committed, without a distributed transaction

swipe the figure sideways, or tap expand for full screen

1/6

dual write

The dual write: UPDATE wallets, COMMIT, then producer.send(event). A crash after COMMIT and before send loses the event: the ledger is debited and the notification service, the analytics pipeline and the payout service never find out.