---
title: "Transactional Outbox"
description: "The optional producer-side outbox helper every SDK ships (ADR-0029): a caller-owned DB transaction writes the business row and the encoded envelope together, then a separate relay publishes the stored bytes verbatim — removing the producer dual write with no envelope change and no DB driver in the core. The OutboxStore save / fetchUnpublished / markPublished / markFailed contract, the Outbox writer, the OutboxRelay flush/drain at-least-once handoff, and the in-memory reference store. The consumer-side mirror of idempotency. The wire envelope stays frozen at schema_version 1."
source: https://babelqueue.com/docs/spec/1.x/outbox/
updated: 2026-06-21T00:00:00.000Z
---

# Transactional Outbox

[Idempotency](/docs/spec/1.x/idempotency/) deduplicates on the **consumer** side. The
outbox is its mirror on the **producer** side: it removes the **dual write** — the
crash window between committing your business row and publishing the message — so the
message commits *atomically with the business data*, then a separate relay hands it to
the broker (ADR-0029).

> **Status:** Authoritative · Helper layer · envelope frozen at `schema_version: 1`

Nothing here touches the wire. The outbox **stores the existing encoded envelope verbatim**
and the relay **publishes those exact bytes** — `trace_id` preserved, byte-compatible. The
outbox's own columns (id, status, attempts) are bookkeeping *around* the envelope, never
*on* it. The envelope stays frozen at `schema_version: 1`.

## The dual write, and why a transaction can't span it

A plain producer does two things that must both happen or neither: commit the business row
(`INSERT INTO orders …`) and publish the message to the broker. They are two independent
systems. A crash *between* them leaves the order saved with no message, or a message sent
for an order that rolled back. There is no distributed transaction to lean on —
BabelQueue is broker-agnostic and deliberately avoids one (GR-7), and no broker offers a
commit the database can see.

The outbox pattern removes the dual write by making the publish a **local** write first:

1. In the **same database transaction** as the business row, persist the encoded envelope
   into an **outbox table**. Commit both together — they succeed or roll back atomically.
2. A separate **relay** reads the durable outbox rows afterwards, publishes each to the
   broker, and marks it published. The handoff to the broker is now a local problem (read a
   row, publish, mark it), not a cross-system one.

The honest scope is **exactly-once handoff into the broker, then at-least-once on the
wire** as always: a crash between *publish* and *mark-published* re-publishes the row on the
next pass. That redelivery is deduped by the consumer on `meta.id` —
[idempotency](/docs/spec/1.x/idempotency/) is the consumer-side mirror that closes the loop.

## The `OutboxStore`

Every SDK exposes the same four-method persistence contract. The **core defines it and binds
to no DB driver** (GR-7); a concrete adapter over your database is the caller's:

| Operation | Meaning |
| :--- | :--- |
| `save(encodedEnvelope, queue)` | Persist the encoded envelope bytes + its target queue; return the new row id. **Called inside the caller's open transaction.** |
| `fetchUnpublished(limit)` | Read up to `limit` pending rows, **oldest-first**. A production adapter SHOULD claim/lock the rows it returns (e.g. `SELECT … FOR UPDATE SKIP LOCKED`) so two relays don't double-publish. |
| `markPublished(ids)` | Mark these rows published — called only **after** the transport accepts them. |
| `markFailed(id, error)` | Record a publish failure and leave the row **pending** for a later pass. |

The transaction boundary is **the caller's**. `save()` runs on the transaction you already
opened around your business write; you commit both together. The store never begins or
commits anything — that is the whole point, and the reason no DB driver enters the core.

## The `Outbox` writer

The writer is tiny: `Outbox.write(envelope)` encodes the envelope via the **frozen
`EnvelopeCodec`** (bytes unchanged), captures `meta.queue` (else `"default"`), and delegates
to `OutboxStore.save()`. **It does not begin or commit anything.** You call it inside your
own transaction, right beside your business write:

```text
begin transaction
    INSERT INTO orders (…)            -- business row
    outbox.write(envelope)           -- the message, same transaction
commit                                -- both, or neither
```

## The `OutboxRelay`

The relay drives the broker handoff through the existing publish-only `Transport` seam:

- **`flush()`** publishes **one batch** of pending rows. Each row is marked **published only
  after** the transport accepts it; a failing publish is caught (`markFailed`, row left
  pending) with a **bounded linear backoff**, and the batch continues — one poison row never
  blocks the others. The sleeper is injectable, so tests run instantly.
- **`drain()`** loops `flush` while a pass makes progress, with a safety ceiling.

The relay publishes the **stored bytes verbatim** — it never decodes, rebuilds, or
re-encodes the envelope. So `trace_id` is preserved end-to-end (GR-4) and the published
bytes are byte-identical to what the producer encoded (GR-1/GR-5).

## The in-memory reference store

Every core ships an `InMemoryOutboxStore` for tests and single-process demos. It honours the
four-method contract over a process-local, insertion-ordered map (oldest-first) but has **no
real transaction** and does **not** claim/lock rows. Production needs a **DB-backed adapter**
— the [InitORM outbox example](https://github.com/BabelQueue/babelqueue-examples/tree/main/outbox-initorm)
is the reference for one (the outbox-table DDL, a `produce.php` that does the business write
and `outbox->write` in one `$db->transaction`, and a `relay.php`), keeping the core itself
DB-free.

## Go

The helper is the `…/outbox` subpackage of the core module (stdlib only, like
`idempotency`/`schema`). `outbox.New(store)` is the writer; `outbox.NewRelay(transport,
store, opts)` is the relay:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/outbox"
)

store := outbox.NewInMemoryStore() // tests / single process
box := outbox.New(store)

// inside the caller's own DB transaction, beside the business write:
env, _ := babelqueue.Make("urn:babel:orders:created",
	map[string]any{"order_id": 1042}, babelqueue.WithQueue("orders"))
id, err := box.Write(env) // encodes via the frozen codec, calls Store.Save — no commit

// later, a relay drains the durable rows to the broker:
relay := outbox.NewRelay(transport, store, outbox.Options{})
res, err := relay.Drain(ctx, 0) // res.Published / res.Failed
```

`Store` is the interface (`Save` / `FetchUnpublished` / `MarkPublished` / `MarkFailed`); a
production fleet implements it over Postgres/MySQL — binding `Save` to the caller's open
transaction — and passes it to `New` and `NewRelay` unchanged.

## PHP

`BabelQueue\Outbox\Outbox::write()` mirrors the Go writer; `OutboxRelay` drains through the
publish-only `Transport`:

```php
use BabelQueue\Outbox\Outbox;
use BabelQueue\Outbox\OutboxRelay;
use BabelQueue\Outbox\InMemoryOutboxStore;

$store = new InMemoryOutboxStore(); // tests / single process
$outbox = new Outbox($store);

// inside your own $db->transaction(...), beside the business write:
$id = $outbox->write($envelope); // encodes via EnvelopeCodec, calls $store->save()

// later: a relay hands the durable rows to the broker
$result = (new OutboxRelay($transport, $store))->drain();
// $result->published / $result->failed
```

A production store implements `BabelQueue\Outbox\OutboxStore` (`save` / `fetchUnpublished` /
`markPublished` / `markFailed`) over a PDO/ORM connection, with `save()` running on the
caller's open transaction — see the InitORM example's `InitOrmOutboxStore`.

## Per-SDK

The outbox is optional and identical in spirit everywhere — same four-method `OutboxStore`,
same caller-owned transaction, same `write` / `flush` / `drain`, same verbatim-bytes relay.
It shipped across all six SDK cores (per-SDK MINORs; the envelope stayed frozen).

| SDK | Writer · Relay | Store · reference |
| :--- | :--- | :--- |
| Go | `outbox.New(store)` `.Write` · `outbox.NewRelay(t, store)` `.Flush`/`.Drain` | `outbox.Store` · `outbox.NewInMemoryStore()` |
| PHP | `new Outbox($store)` `->write` · `new OutboxRelay($t, $store)` `->flush`/`->drain` | `BabelQueue\Outbox\OutboxStore` · `InMemoryOutboxStore` |
| Python | `Outbox(store)` `.write` · `OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`babelqueue.outbox`) |
| Node | `new Outbox(store)` `.write` · `new OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`@babelqueue/core`) |
| Java | `new Outbox(store)` `.write` · `new OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`com.babelqueue.outbox`) |
| .NET | `new Outbox(store)` `.WriteAsync` · `new OutboxRelay(publisher, store)` `.FlushAsync`/`.DrainAsync` | `IOutboxStore` · `InMemoryOutboxStore` (`BabelQueue.Outbox`) |

Two idiomatic differences to expect: **.NET** is async throughout (`WriteAsync` /
`FetchUnpublishedAsync` / `FlushAsync` / `DrainAsync`, `CancellationToken`-threaded) and its
store interface is `IOutboxStore`; **Python**'s method names are snake-case
(`fetch_unpublished` / `mark_published` / `mark_failed`) and its relay backoff is in seconds
where the others are in milliseconds. The broker seam the relay forwards through is the
SDK's existing publish surface — a `Transport` (Go/PHP/Python), an `OutboxTransport`
(Node/Java), or an `OutboxPublisher` delegate (.NET).

The DB-backed adapter stays the caller's in every SDK — the cores ship only the in-memory
reference, exactly as on the PHP side. Relay concurrency (claim/lock so two relays don't
double-publish), and retention/archival of published rows, are the adapter's concern.

See each SDK's reference for wiring a store:
[Go](/docs/babelqueue-go/1.x/introduction/), [PHP](/docs/php-sdk/1.x/introduction/),
[Python](/docs/babelqueue-python/1.x/introduction/), [Node](/docs/babelqueue-node/1.x/introduction/),
[Java](/docs/babelqueue-java/1.x/introduction/), [.NET](/docs/babelqueue-dotnet/1.x/introduction/).

The outbox guarantees the message is *sent* once you commit; [idempotency](/docs/spec/1.x/idempotency/)
guarantees its *effect* fires once on the consumer despite the at-least-once redelivery the
relay's re-publish can cause. Use them together. To keep one trace across the produce→consume
hop, see [Observability (OpenTelemetry)](/docs/spec/1.x/observability/).