# BabelQueue

> BabelQueue is an open-source, broker-agnostic standard and set of SDKs that let PHP/Laravel, Symfony, Python, Go, Node.js, Java and .NET exchange queue messages over a single frozen strict-JSON envelope (`schema_version: 1`) whose job identity is a URN — no PHP `serialize()` anywhere on the wire.

BabelQueue is a specification first and a set of libraries second. Six SDK cores implement it — PHP (with Laravel and Symfony adapters), Python, Go, Node.js, Java and .NET — and every one emits byte-compatible envelopes, so a job produced in one language is consumed natively in another. Seven brokers are GA and locked by a shared cross-SDK conformance suite: Redis, RabbitMQ, Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka and Apache ActiveMQ/Artemis. The envelope is frozen at `schema_version: 1`; identity is the URN in the `job` field, never a class name; `data` is pure JSON; and `trace_id` is preserved and forwarded unchanged across every hop, so one correlation id ties a whole chain of work together across languages. BabelQueue is MIT licensed and developed in the open at https://github.com/BabelQueue.

Every page listed below also exists as raw markdown at its own URL with `.md` appended — for example `/docs/spec/1.x/envelope.md`.

---

# Wire Contract

Source: https://babelqueue.com/docs/spec/1.x/introduction/

The frozen, language-neutral contract every BabelQueue SDK speaks — the envelope, the URN identity, and the rules that keep all six languages byte-compatible.

BabelQueue is, at its core, **one contract**: a strict, language-neutral definition
of the bytes that travel on the queue. Every SDK — PHP, Python, Go, Node.js, Java
and .NET — produces and consumes **exactly** this shape, so a message written in
one language is read natively in another, over the broker you already run.

This section is the source of truth for that contract. If an SDK, an example, or a
marketing page ever disagrees with these pages, **the contract wins**.

## What the contract covers

- **[The envelope](/docs/spec/1.x/envelope/)** — the JSON document on the wire:
  `job`, `trace_id`, `data`, `meta`, `attempts`, plus the cross-language data rules
  (encoding, numbers, time) and the producer/consumer rules every SDK obeys.
- **[URN naming](/docs/spec/1.x/urn/)** — how a message is identified by a stable
  `urn:babel:<context>:<event>` string instead of a class name, so any language can
  route on it.

## The three things every message carries

1. **Identity** — *what* the message is: the `job` field, a **URN**, never a class name.
2. **Correlation** — *how to trace it* across services: a required `trace_id`.
3. **Payload + bookkeeping** — the business `data`, descriptive `meta`, and the
   transport `attempts` counter.

## Frozen at `schema_version: 1`

The envelope is **frozen**: within `schema_version: 1`, the existing envelope fields,
their types and their meanings do not change — no field is renamed, removed or retyped,
and the required fields keep their meaning. A new field may be added **only** as an
optional, ignorable field and only through an architecture decision record and an
explicit version decision. A consumer that does not understand an unknown field MUST
still process the message correctly and SHOULD preserve that field when it re-emits the
message. Renaming, removing or retyping a field requires a bump of
`meta.schema_version`. Consumers MUST reject or quarantine any `schema_version` they do
not understand.

This stability is the whole point — a `v1` producer stays readable by every `v1`
consumer, in any language, indefinitely.

## Cross-SDK parity

Every SDK ships against a shared suite of **golden envelope fixtures**: two SDKs
producing the same logical message emit byte-comparable envelopes (modulo the
intrinsically per-message/per-language fields `meta.id`, `trace_id`, `meta.lang` and
`meta.created_at`). Parity is enforced in every SDK's CI, not just asserted here.

Continue to [The envelope](/docs/spec/1.x/envelope/).

---

# The Envelope

Source: https://babelqueue.com/docs/spec/1.x/envelope/

The frozen BabelQueue message envelope (schema_version 1): every field, the meta block, cross-language data rules, producer/consumer rules, and forbidden fields.

The single, frozen definition of the bytes that travel on the queue. Every SDK, in
every language, MUST produce and consume exactly this shape.

> **Status:** Authoritative · Frozen · `schema_version: 1`

## Canonical envelope

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": {
    "order_id": 1042,
    "amount": 99.90
  },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "default",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 0
}
```

Top-level key order is not significant (JSON objects are unordered), but producers
SHOULD emit keys in the order above for readable logs.

## Field reference

| Field | Type | Required | Mutable | Description |
| :--- | :--- | :---: | :---: | :--- |
| `job` | string | ✅ | no | The message **URN** — its language-agnostic identity. See [URN naming](/docs/spec/1.x/urn/). The canonical field name is `job`; consumers MUST also accept `urn` as an inbound alias. **Never** a PHP/Java/… class name. |
| `trace_id` | string (UUID) | ✅ | no | Cross-service correlation id. Generated by the original producer; **preserved and forwarded unchanged** by every SDK across every hop. Enables end-to-end distributed tracing. |
| `data` | object | ✅ | no | The business payload — a **pure**, JSON-encodable key/value object. No language-specific types (no PHP objects, closures, resources; no binary without base64). Numbers follow the [data rules](#cross-language-data-rules). |
| `meta` | object | ✅ | no¹ | Producer-set descriptive metadata. Immutable after production. See [the meta block](#the-meta-block). |
| `attempts` | integer | ✅ | **yes** | Transport-level delivery/retry counter, starting at `0`. Mutated by the broker/worker on each (re)delivery. Deliberately **top-level**, kept out of the immutable `meta` block. |

¹ `meta` is conceptually immutable end-to-end. The only counter that legitimately
changes during a message's life is top-level `attempts` — it is *transport* state,
not producer metadata.

## The meta block

| Field | Type | Required | Description |
| :--- | :--- | :---: | :--- |
| `id` | string (UUID) | ✅ | Unique id for **this specific message**. Distinct from `trace_id`. |
| `queue` | string | ✅ | The logical queue name the message was produced onto (not the broker key). |
| `lang` | string (enum) | ✅ | Producer language tag — one of `php`, `go`, `python`, `java`, `dotnet`, `node`. For observability and debugging. |
| `schema_version` | integer | ✅ | Envelope schema version. Currently `1`. Consumers MUST reject (or quarantine) versions they do not understand. |
| `created_at` | integer | ✅ | Production time as **Unix epoch milliseconds, UTC** (milliseconds, not seconds). |

Producers MAY add extra `meta.*` keys for their own observability, but consumers
MUST ignore unknown `meta` keys (forward-compatibility), and those keys MUST NOT be
required for correct routing or processing.

## `id` vs `trace_id` — do not conflate

| | `meta.id` | `trace_id` |
| :--- | :--- | :--- |
| Scope | one message | one causal chain (may span many messages/services) |
| Generated by | the producing SDK, per message | the **first** producer in the chain; reused thereafter |
| Lifetime | dies with the message | propagates across every hop |
| Use | dedupe, ack/nack, logs for one message | end-to-end tracing/observability |

When a consumer produces a downstream message while handling one, it SHOULD copy the
inbound `trace_id` onto the new envelope and mint a **new** `meta.id`.

## Cross-language data rules

Because the same `data` is decoded by six languages, these are part of the contract,
not implementation details:

- **Encoding.** UTF-8 JSON only — no trailing commas, no comments, no `NaN`/`Infinity`.
  The reference encoders emit unescaped unicode and slashes so output is byte-identical
  across SDKs.
- **Numbers.** Integers must fit signed 64-bit; do not exceed `2^53 − 1` if any
  consumer is JavaScript/Node unless carried as a string. Floats are IEEE-754 doubles
  — **never** put currency or exact-decimal values in floats; use integer minor units
  (e.g. cents) or strings. (The example uses `99.90` for readability only.)
- **Time.** `meta.created_at` is Unix epoch **milliseconds** (UTC). Time values inside
  `data` SHOULD be Unix ms (integer) or RFC 3339 / ISO 8601 UTC strings — agreed per
  URN, never a locale-formatted string.
- **Binary.** Raw bytes are not allowed in JSON. Base64-encode and document the field
  per URN.
- **Booleans / null.** Use JSON `true`/`false`/`null` — never `0/1` or `"true"`.

## Producer rules

A conformant producer MUST:

1. Set `job` to a **non-empty** URN. An empty URN is a programming error and MUST raise.
2. Set `trace_id` to a UUID — reuse an inbound one if continuing a trace, otherwise
   generate a new v4 UUID.
3. Emit `data` as a pure JSON object (see the data rules above).
4. Populate every required `meta` field — `meta.id` unique per message, `meta.lang`
   matching the producing language, `meta.schema_version = 1`, `meta.created_at` in Unix ms.
5. Initialize top-level `attempts` to `0`.
6. Encode as UTF-8 JSON and publish via the broker binding.

## Consumer rules

A conformant consumer MUST:

1. Read the URN from `job`, accepting `urn` as an alias if `job` is absent.
2. Reject/quarantine any envelope whose `meta.schema_version` it does not support.
3. Route to the handler mapped to the URN. If none is mapped, apply the configured
   unknown-URN strategy (`fail` · `delete` · `release` · `dead-letter`).
4. Preserve `trace_id` for logging and for any downstream messages.
5. Treat `meta` and `data` as read-only; only the broker/worker mutates `attempts`.
6. Tolerate unknown extra keys (forward-compatibility) — never hard-fail on them.

## Forbidden fields

These appeared in early drafts and are **not** part of the contract. Do not emit or
rely on them:

| Forbidden field | Use instead |
| :--- | :--- |
| `timestamp` (top-level) | `meta.created_at` (Unix **ms**) |
| `meta.max_retries` | consumer-side config / per-URN policy |
| `meta.attempts` | top-level `attempts` |
| `meta.source` | `meta.lang` |
| `meta.ts` | `meta.created_at` |
| `urn` as the canonical producer field | `job` (with `urn` accepted only as an inbound **alias**) |

## Optional `dead_letter` block

Messages sitting on a **dead-letter queue** carry one extra, optional top-level field,
`dead_letter`, describing why they failed. It appears *only* on DLQ messages — never
on a normal queue — and is additive, so the envelope stays at `schema_version: 1`.
Normal consumers ignore it.

```json
"dead_letter": {
  "reason": "failed",
  "error": "Payment gateway timeout",
  "exception": "App\\Exceptions\\GatewayTimeout",
  "failed_at": 1749132730000,
  "original_queue": "orders",
  "attempts": 3,
  "lang": "php"
}
```

## Change control

- Any change to this envelope is **architecturally significant**.
- Additive, optional, backward-compatible fields → keep `schema_version = 1`.
- Removing/renaming/retyping a field, or changing semantics → **bump
  `schema_version`** and provide a migration/compat path for older consumers.

Continue to [URN naming](/docs/spec/1.x/urn/).

---

# URN Naming

Source: https://babelqueue.com/docs/spec/1.x/urn/

How every BabelQueue message is identified — a stable urn:babel:<context>:<event> string instead of a class name, so any language can route on it.

A message's identity travels in the envelope's [`job`](/docs/spec/1.x/envelope/) field
as a **URN** (Uniform Resource Name). The URN is the *only* thing a consumer uses to
decide which handler runs. Because it is a plain, stable string under the
application's control — never a class name — a Go or Python consumer can route on it
without sharing any type with the producer, and the producing class can be renamed or
moved freely without breaking any consumer.

## Format

```
urn:babel:<bounded-context>:<event-or-command>
```

| Segment | Meaning | Rules |
| :--- | :--- | :--- |
| `urn` | Literal URN scheme prefix (RFC 8141). | Fixed. |
| `babel` | Namespace identifier for the BabelQueue ecosystem. | Fixed. |
| `<bounded-context>` | The domain/service area that owns the message. | lowercase, `[a-z0-9-]`. |
| `<event-or-command>` | What happened (event, past tense) or what to do (command, imperative). | lowercase, dot-separated sub-parts allowed. |

**Examples**

```
urn:babel:orders:created
urn:babel:orders:invoice.requested
urn:babel:billing:payment.captured
urn:babel:media:thumbnail.generate
urn:babel:notifications:email.welcome.send
```

> This convention is **recommended, not enforced** by the libraries (to preserve
> flexibility), but teams SHOULD treat it as mandatory in their own projects.

## Rules

1. **Non-empty.** An empty URN is a fatal producer error.
2. **Stable across deployments.** A URN is a public contract; once published and
   consumed by another service, treat it as immutable. To change identity, introduce a
   **new** URN and migrate.
3. **Unique per message type.** One URN ↔ one logical message/handler.
4. **Never a class name.** `App\Jobs\ProcessOrder` must never appear on the wire.
5. **Lowercase & ASCII.** Use `[a-z0-9:._-]`. No spaces, no uppercase, no unicode.
6. **No payload in the URN.** Identifiers like `order_id` belong in `data`.
   `urn:babel:orders:created` ✅ — `urn:babel:orders:1042` ❌.

## Events vs commands

| Kind | Tense/mood | Example | Semantics |
| :--- | :--- | :--- | :--- |
| **Event** | past tense | `urn:babel:orders:created` | Something *happened*; zero-to-many consumers may react. |
| **Command** | imperative | `urn:babel:media:thumbnail.generate` | A request to *do* something; typically one handler. |

Prefer events for decoupled, fan-out flows; use commands for directed work.

## Mapping URNs to handlers

The wire identity is decoupled from code: each consuming service maps URNs to its own
handler types, in whatever registry that language/framework uses. For example, in
Laravel this lives in `config/babelqueue.php`:

```php
'handlers' => [
    'urn:babel:orders:created'           => \App\Consumers\OnOrderCreated::class,
    'urn:babel:orders:invoice.requested' => \App\Consumers\GenerateInvoice::class,
],
```

The same URN routed in Go, Python, Node, Java or .NET maps to that language's handler
registry. See the [consumer rules](/docs/spec/1.x/envelope/#consumer-rules).

## Versioning a message identity

When a message's `data` shape changes incompatibly, do **not** silently reinterpret
the old URN. In order of preference:

1. **Additive change** (new optional field): keep the URN; no version change.
2. **Breaking change**: mint a new URN with a version suffix and run both during
   migration, e.g. `urn:babel:orders:created` → `urn:babel:orders:created.v2`.
3. Decommission the old URN only after all producers/consumers have migrated.

URN-level versioning is independent of the envelope's `meta.schema_version` (which
versions the *envelope*, not the *payload of a specific URN*).

## Registry (recommended practice)

Teams SHOULD keep a checked-in registry of every URN they own — its `data` shape,
owning context, producers, and consumers — so the cross-service contract is
discoverable. A per-URN JSON Schema for `data` is encouraged. This registry is
project-specific and lives in your own apps, not in the BabelQueue libraries.

---

# Broker bindings

Source: https://babelqueue.com/docs/spec/1.x/broker-bindings/

How each broker carries the canonical envelope natively — the Redis, RabbitMQ, Amazon SQS, Azure Service Bus, Pulsar, Kafka and Artemis bindings. The body is identical everywhere; a binding maps the contract onto each broker's native metadata so consumers route and trace without decoding the body.

The [envelope](/docs/spec/1.x/envelope/) is **broker-agnostic** — it is just the
message body. A **binding** says how a given broker carries that body natively, so a
consumer can route by URN and continue a trace **without decoding the body**, and so a
message one SDK produces is consumed byte-for-byte by another over the same broker.

Two rules hold for every binding:

- The **body is always the canonical envelope** — compact UTF-8 JSON, byte-identical
  across SDKs. Native metadata is a *redundant, routable projection* of the body, never
  a replacement for it.
- The envelope is **frozen at `schema_version: 1`**. A new broker is purely additive —
  it never changes the wire format.

## Redis

Reliable-queue pattern: `RPUSH` to produce, a blocking move to a `:processing` list to
reserve, then remove on ack. Redis lists carry no per-message metadata, so routing and
tracing read the envelope body directly (`job`/`urn`, `trace_id`). At-least-once via the
processing list.

### Redis compatibility

The reservation step is not the same command in every SDK, so the minimum Redis version
differs. None of the SDKs defines a hash-tag convention for Redis Cluster.

| SDK / transport | Reservation command | Minimum Redis | Redis Cluster |
| :--- | :--- | :--- | :--- |
| Go (`babelqueue-go/redis`) | `BLMOVE <queue> <queue>:processing LEFT RIGHT` | 6.2 | Only with hash-tagged queue names (see below) |
| Python (`babelqueue[redis]`) | `BLMOVE … LEFT RIGHT` | 6.2 | Only with hash-tagged queue names |
| Java (`babelqueue-redis`) | `BLMOVE … LEFT RIGHT` | 6.2 | Only with hash-tagged queue names |
| .NET (`BabelQueue.Redis`) | `LMOVE … LEFT RIGHT` (non-blocking) | 6.2 | Only with hash-tagged queue names |
| Node (`@babelqueue/redis`) | `BRPOPLPUSH` (deprecated by Redis since 6.2, still available) | Any (also below 6.2) | Only with hash-tagged queue names |
| Laravel driver | Laravel's own Lua-script reservation (`:reserved` sorted set) | Set by Laravel | Only with hash-tagged queue names (`queues:{name}`, `:reserved`, `:delayed`) |

- **Pop side.** `BRPOPLPUSH` is the equivalent of `BLMOVE … RIGHT LEFT`, so it pops
  from the opposite end of the list than the `LEFT RIGHT` form the other SDKs use.
  Producers append with `RPUSH`, so the Node transport pops the most recently pushed
  element (LIFO), while the other SDKs pop the oldest (FIFO). This holds even in a
  Node-only fleet: messages pushed as A, B, C are handled as C, B, A. The binding does
  not harmonise this; do not rely on FIFO order when a Node consumer reads the queue.
- **Cluster.** A move between `queues:<name>` and its processing/reserved list is a
  multi-key command, and it only works when both keys sit in one hash slot. Wrap the
  queue name in a hash tag (for example `queues:{orders}` and
  `queues:{orders}:processing`) and use the same names in every producer and consumer,
  in every language. Multi-key commands can be briefly unavailable during manual
  resharding ([Redis Cluster specification](https://redis.io/docs/latest/operate/oss_and_stack/reference/cluster-spec/)).
- **At-least-once.** No SDK re-queues `<queue>:processing` automatically. A consumer
  that dies after the move, or a handler that fails, leaves the element there until
  an operator moves it back, so handlers must be
  [idempotent](/docs/spec/1.x/idempotency/).

## RabbitMQ (AMQP 0-9-1)

The body is the envelope; the contract is also projected onto AMQP properties and
headers so a consumer can route on the broker's metadata:

| AMQP field | Carries |
| :--- | :--- |
| `type` | the URN (`job`) |
| `correlation_id` | `trace_id` |
| `message_id` | `meta.id` |
| header `x-attempts` | `attempts` |
| header `x-schema-version` | `meta.schema_version` |
| header `x-source-lang` | `meta.lang` |

Messages are `application/json`, persistent delivery, on durable queues; consumed with
`basic.get` + manual ack (at-least-once).

## Amazon SQS

The canonical envelope is the **`MessageBody`**. On produce, the transport projects the
envelope onto native **`MessageAttributes`** — a routable view of the body. Ids and
strings are `DataType` **String**; counters are `DataType` **Number**:

| MessageAttribute | DataType | Value |
| :--- | :--- | :--- |
| `bq-job` | String | the URN (`job`) |
| `bq-trace-id` | String | `trace_id` |
| `bq-message-id` | String | `meta.id` |
| `bq-schema-version` | Number | `meta.schema_version` (`1`) |
| `bq-source-lang` | String | `meta.lang` |
| `bq-created-at` | Number | `meta.created_at` as epoch milliseconds |

**Consuming** uses SQS's native delivery semantics:

- A receive **reserves** the message for the visibility timeout; a successfully handled
  message is removed with `DeleteMessage`. A failing handler is **released**, not
  deleted, while retries remain: every SDK consumer calls `ChangeMessageVisibility` on
  the message with the configured release delay, which **defaults to 0 s** (redeliver
  now), so retries are at-least-once and do not wait out the visibility timeout. An
  unknown URN under the `release` strategy uses the same call, also with a 0 s default.
  Runtimes with a max-attempts policy (the Go and Python `App`, default 3) stop
  releasing once attempts are exhausted: the message is written to the BabelQueue
  dead-letter queue (`<queue>.dlq`) when dead-lettering is enabled, otherwise dropped,
  and in both cases deleted from the SQS queue.
- **Poison messages.** Because a failing message comes back immediately, a message that
  always fails is redelivered until the runtime's attempt limit (where one exists) or
  the queue's own policy stops it. Configure a native `RedrivePolicy`
  (`maxReceiveCount` → a dead-letter queue) on every consumed queue, or set a non-zero
  release delay as backoff.
- **`attempts` is reconciled** to `max(body.attempts, ApproximateReceiveCount − 1)`. The
  first delivery reads `0`; a runtime-incremented count is **never lowered**; an absent
  or non-numeric `ApproximateReceiveCount` is ignored. A drop-in driver that instead
  surfaces the broker's native count (e.g. Laravel's `SqsJob::attempts()`) documents
  that divergence.

**FIFO queues** (`.fifo`) set `MessageGroupId` (the configured group, else the queue
name) and `MessageDeduplicationId` = `meta.id` — unless the queue uses content-based
deduplication. Delayed delivery uses `DelaySeconds`, capped at SQS's 900-second maximum.

This binding is implemented identically across every SDK that ships an SQS transport
(Go, Python, Node, Java, PHP, .NET) and is locked by the
[conformance suite](https://github.com/BabelQueue/conformance), so the projected
attributes and the reconciled `attempts` are guaranteed to match.

## Azure Service Bus

The canonical envelope is the **`Body`**. Azure Service Bus has first-class native slots
for almost every envelope concept, so the binding maps onto native message fields and
needs only two custom application properties:

| Field | Value |
| :--- | :--- |
| `Subject` (a.k.a. Label) | the URN (`job`) — **route on `Subject` without reading the body** |
| `CorrelationId` | `trace_id` |
| `MessageId` | `meta.id` (enables ASB duplicate detection) |
| `ContentType` | `application/json` |
| `DeliveryCount` | broker-maintained, **1-based** — the native attempts source |
| `ApplicationProperties["bq-schema-version"]` | `meta.schema_version` (`1`) |
| `ApplicationProperties["bq-source-lang"]` | `meta.lang` |
| `ApplicationProperties["bq-created-at"]` | `meta.created_at` (epoch ms, convenience mirror) |

Application properties are **native AMQP-typed values** (numbers stay numbers), not the
`DataType`-wrapped strings SQS uses.

**Consuming** uses ASB's PeekLock model:

- A receive **reserves** the message for the lock duration; a handled message is removed
  with `Complete`. A failing handler `Abandon`s it — the broker redelivers and
  **increments `DeliveryCount`** (at-least-once). At `MaxDeliveryCount` ASB auto-moves it
  to the native `$DeadLetterQueue` sub-queue.
- **`attempts` is reconciled** to `max(body.attempts, DeliveryCount − 1)`: `DeliveryCount`
  (1-based) is the native floor (first delivery reads `0`), and a runtime-incremented body
  count is never lowered. The rule is identical for the native-consumer SDKs (.NET, Java,
  Node) and the runtime-transport SDKs (Python, Go).

**Delayed delivery** is native — set `ScheduledEnqueueTime` to `now + delay`. **Auth** is a
connection string or Azure AD (`DefaultAzureCredential`); transport is AMQP 1.0 over TLS
(or WebSockets). An optional `SessionId` (FIFO) is opt-in and not a contract field.

This binding is implemented identically across every SDK that ships an ASB transport
(.NET, Java, Python, Node, Go) and is locked by the
[conformance suite](https://github.com/BabelQueue/conformance). PHP is deferred (no modern
official client).

## Apache Pulsar

The canonical envelope is the **message payload**. Pulsar message properties are
**string→string**, so the binding projects a redundant, routable view of the body onto
`bq-` properties (every value stringified) and keeps the body authoritative:

| Property | Value |
| :--- | :--- |
| `bq-job` | the URN (`job`) — **route on `bq-job` without reading the body** |
| `bq-trace-id` | `trace_id` |
| `bq-message-id` | `meta.id` |
| `bq-schema-version` | `meta.schema_version` (`"1"`) |
| `bq-source-lang` | `meta.lang` |
| `bq-attempts` | `attempts` — the **authoritative** count, carried in the body |
| `publishTime` | mirrors `meta.created_at` (broker-set; body authoritative) |

Properties are strings (Pulsar has no typed properties), so numbers are stringified —
unlike ASB's native AMQP-typed values.

**Consuming** receives one message at a time:

- A handled message is `acknowledge`d. A failing handler `negativeAcknowledge`s it — the
  broker redelivers it (at-least-once) and **increments `RedeliveryCount`**. With a native
  `DeadLetterPolicy` it eventually moves to the cross-language `<queue>.dlq` topic.
- **`attempts` is reconciled** to `max(body.attempts, RedeliveryCount)`. `RedeliveryCount`
  is **0-based** (0 on first delivery), so it maps directly with **no −1** — and a
  runtime-incremented body count is never lowered. The rule is identical for the
  native-consumer SDKs (.NET, Java, Node) and the runtime-transport SDKs (Python, Go).

**Delayed delivery** is native — `deliverAfter` (relative) or `deliverAt` (absolute), also
mirrored on a `bq-delay` property. The default subscription is **`Shared`**, named
`babelqueue`; topics default to `persistent://public/default/<queue>`. **Auth** is via the
service URL (`pulsar://` or `pulsar+ssl://`) plus any client-configured TLS/token.

This binding is implemented identically across every SDK that ships a Pulsar transport
(.NET, Java, Python, Node, Go) and is locked by the
[conformance suite](https://github.com/BabelQueue/conformance). **PHP** reaches Pulsar over
Pulsar's native **WebSocket API** (`PulsarTransport` to produce, `PulsarConsumer` to consume)
— the same envelope and `bq-` properties, just a different access path; it round-trips live
against the native SDKs.

## Apache Kafka

Kafka is a partitioned, append-only **log** with consumer-group **offset commits** — not a
queue with per-message ack. It has **no native** per-message ack, delayed delivery,
dead-letter queue, or delivery counter, so this binding absorbs all four in the transport
layer (the envelope stays `schema_version: 1`). The canonical envelope is the record
**value**; the contract fields move to `bq-` **record headers** (UTF-8 byte strings, so
integers are stringified):

| Field | Value |
| :--- | :--- |
| `value` | the canonical envelope JSON |
| `timestamp` | mirrors `meta.created_at` (Unix ms) |
| header `bq-job` | the URN (`job`) — **route on `bq-job` without reading the body** |
| header `bq-trace-id` | `trace_id` |
| header `bq-message-id` | `meta.id` |
| header `bq-schema-version` | `meta.schema_version` (`"1"`) |
| header `bq-source-lang` | `meta.lang` |
| header `bq-attempts` | `attempts` — the **authoritative** retry counter (Kafka has no native one) |

**Consuming** is **process-then-commit** (manual commit, `enable.auto.commit = false`): a
record is reserved by being polled, the handler runs, and only then is the offset committed
(`Commit`). A crash before commit redelivers it on the next poll — at-least-once; handlers
MUST be idempotent (dedupe on `meta.id`).

- **`attempts` is reconciled** to the **`bq-attempts` header when present** (authoritative),
  else the body's own `attempts` (the fallback for a non-BabelQueue producer). This is **not
  a max** — the header overrides the body.
- **Retry / delay** is SDK-owned, since Kafka has neither: a failing handler republishes the
  envelope to a tiered **retry topic** `<topic>.retry.<n>` with `bq-attempts + 1` (and
  `bq-delay` / `bq-original-topic`), then commits; a retry-topic worker re-injects it after
  the tier delay. A delay or release with **no retry topics configured raises** rather than
  silently dropping.
- **Terminal failure → DLQ:** at max-tries the envelope goes to `<topic>.dlq` with the
  additive `dead_letter` block (opt-in; if disabled, terminal failures degrade to
  commit-and-drop).

**Connection** is `bootstrap.servers` + a `group.id`; **auth** is `SASL_SSL` / `SSL` via the
native client. This binding is implemented identically across Java, Go, Node, Python and .NET
and is locked by the [conformance suite](https://github.com/BabelQueue/conformance). **PHP**
ships both halves over **`ext-rdkafka`** (opt-in — the one binding that relaxes the
zero-extension rule): `KafkaTransport` to produce, `KafkaConsumer` (process-then-commit) to
consume, plus the SDK-owned retry-topic machinery (`KafkaRetryRouter` + `KafkaRetryConsumer`);
proven live Java→PHP.

## Apache ActiveMQ Artemis

Artemis speaks **AMQP 1.0** (not RabbitMQ's 0-9-1), and gives the binding native primitives —
per-message settlement, scheduled delivery, a delivery counter and a dead-letter address — so
it maps onto them rather than re-implementing. The canonical envelope is the **message body**;
the contract fields ride the slots a JMS peer reads, plus the string-valued `bq_` application
properties:

| Field | Value |
| :--- | :--- |
| body | the canonical envelope JSON |
| annotation `x-opt-jms-type` | the URN (`job`) → JMSType — **route on it without reading the body** |
| `correlation-id` | `trace_id` → JMSCorrelationID |
| `creation-time` | `meta.created_at` (Unix ms) → JMSTimestamp |
| property `bq_schema_version` | `meta.schema_version` (`"1"`) |
| property `bq_source_lang` | `meta.lang` |
| property `bq_attempts` | `attempts` — a 0-based mirror (the body stays authoritative) |
| property `bq_app_id` | `"babelqueue"` |

These property names use **underscores**, not the hyphens of the Kafka/Pulsar bindings: a JMS
property name must be a valid Java identifier (no `-`), and the Java binding produces/consumes
over JMS, so every Artemis SDK uses the same JMS-legal `bq_` form for cross-protocol parity.

Unlike Kafka/Pulsar, the URN / `trace_id` / message-id are **not** `bq_` properties — they ride
the JMS-native slots (`x-opt-jms-type`, `correlation-id`, the broker-set message-id), so a plain
JMS consumer routes and correlates with no BabelQueue awareness.

**Consuming** reserves one message at a time and settles it per message:

- A handled message is **`accept`ed** (acknowledged). A failing handler **`release`s** it — the
  broker redelivers it (at-least-once) and **increments the AMQP `delivery-count`**. At max-tries
  the envelope goes to the cross-language `<queue>.dlq` with the additive `dead_letter` block
  (alongside Artemis's own dead-letter address).
- **`attempts` is reconciled** to `max(body.attempts, delivery-count)`. The AMQP `delivery-count`
  is **0-based** (0 on first delivery), so it maps directly with **no −1** — and a
  runtime-incremented body count is never lowered. The Java binding consumes over **JMS** and
  reads the 1-based `JMSXDeliveryCount`, subtracting 1 to arrive at the **same** 0-based
  `attempts`. The rule is identical for the native-consumer SDKs (.NET, Java, Node) and the
  runtime-transport SDKs (Python, Go).

**Delayed delivery** is native — JMS 2.0 `setDeliveryDelay` (Java) or the `x-opt-delivery-time`
annotation (AMQP), also mirrored on a `bq_delay` property. **Connection** is the broker's AMQP
acceptor (`amqp://` / `amqps://`, default port 5672); Java connects over JMS via the Artemis
client. This binding is implemented identically across Java (JMS), .NET, Python, Node and Go (all
AMQP 1.0) and is locked by the
[conformance suite](https://github.com/BabelQueue/conformance). **PHP** reaches Artemis over
**STOMP** (`StompTransport` to produce; Laravel ships the `babelqueue-artemis` drop-in consume
driver) — Artemis bridges STOMP ↔ AMQP 1.0 ↔ JMS on the same address, so a STOMP-produced
message is consumed natively by the JMS and AMQP-1.0 SDKs, and vice versa.

## Other brokers

With Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka and Apache ActiveMQ Artemis all
shipped, the binding catalogue covers every broker on the expansion roadmap. New brokers follow
the same shape — body identical, contract fields projected onto native metadata — and ship as
additive MINOR releases. The envelope stays `schema_version: 1`.

---

# Broker setup recipes

Source: https://babelqueue.com/docs/spec/1.x/broker-recipes/

Quick-start recipes for pointing each BabelQueue SDK at a broker — worked examples for Redis and RabbitMQ, plus where to find the other five (SQS, Azure SB, Pulsar, Kafka, Artemis). The exact connection, runtime, transport or DSN per language, all producing and consuming the one canonical envelope.

A copy-paste recipe per SDK for the two most common brokers — **Redis** and
**RabbitMQ** — so a message produced in one language is consumed in another over the
broker you already run. Every recipe emits and reads the exact
[canonical envelope](/docs/spec/1.x/envelope/) (`schema_version: 1`); only the wiring
differs. The body is identical everywhere, so any pairing interoperates — see the
[broker bindings](/docs/spec/1.x/broker-bindings/) for how each broker carries it
natively.

> **Seven brokers are GA — these recipes cover two of them.** Redis and RabbitMQ are
> the worked examples below; the same shape extends to **Amazon SQS**, **Azure Service
> Bus**, **Apache Pulsar**, **Apache Kafka** and **Apache ActiveMQ/Artemis**, all GA and
> conformance-locked. For those five, swap in the matching transport/extra — Go
> `…/sqs · …/pulsar · …/kafka · …/artemis · …/azureservicebus`, Python `[sqs] [pulsar]
> [kafka] [artemis]` (+ Azure SB), Node `@babelqueue/sqs · …/pulsar · …/kafka · …/artemis
> · …/azure-service-bus`, Java `babelqueue-sqs/-pulsar/-kafka/-artemis/-azureservicebus`,
> .NET `BabelQueue.Sqs/.Pulsar/.Kafka/.Artemis/.AzureServiceBus`, and PHP `php-sdk`
> `SqsTransport` / `KafkaTransport`+`KafkaConsumer` / `PulsarTransport`+`PulsarConsumer` /
> `StompTransport` (Artemis) plus the Laravel `babelqueue-sqs` and `babelqueue-artemis`
> drivers. The one documented gap is **PHP × Azure Service Bus**. The per-broker
> projection is in the [broker bindings](/docs/spec/1.x/broker-bindings/); the keys are in
> the [configuration reference](/docs/spec/1.x/broker-bindings/).

> **Runtime vs core-only.** The shapes differ by language: a *runtime* SDK (Laravel,
> Symfony, Python, Go's `App`, Node adapters) gives you `publish`/worker out of the box,
> while a *core-only* SDK (.NET, Java core) is a codec — you move the bytes with your own
> broker client, or a dedicated transport where one ships (e.g. `BabelQueue.Redis`,
> `com.babelqueue:babelqueue-redis`). Where a language has no first-party RabbitMQ
> transport on the core, the recipe uses its framework adapter (MassTransit, Spring AMQP),
> which is the supported path.

All recipes use the same queue (`orders`) and URN (`urn:babel:orders:created`) so you
can mix any producer with any consumer.

## PHP / Laravel

Laravel plugs in as a custom queue connection. Pick the driver per broker in
`config/queue.php`:

```php
// config/queue.php — Redis
'connections' => [
    'babelqueue' => [
        'driver'      => 'babelqueue-redis',
        'connection'  => 'default',
        'queue'       => env('BABELQUEUE_QUEUE', 'orders'),
        'retry_after' => 90,
    ],
],
```

```php
// config/queue.php — RabbitMQ
'connections' => [
    'babelqueue' => [
        'driver' => 'babelqueue-rabbitmq',
        'host'   => env('RABBITMQ_HOST', '127.0.0.1'),
        'port'   => env('RABBITMQ_PORT', 5672),
        'queue'  => env('BABELQUEUE_QUEUE', 'orders'),
    ],
],
```

Publish with the facade and run the standard worker against the connection:

```php
use BabelQueue\Facades\BabelQueue;

BabelQueue::publish('urn:babel:orders:created', ['order_id' => 1042]);
```

```bash
php artisan queue:work babelqueue
```

URN→handler mapping and the dead-letter policy live in `config/babelqueue.php` — see
[Laravel configuration](/docs/laravel/1.x/configuration/).

## PHP / Symfony

Symfony Messenger keeps its own transport DSN — `redis://` or `amqp://` — and you swap
in the BabelQueue serializer so the wire format is the canonical envelope:

```yaml
# config/packages/messenger.yaml
framework:
    messenger:
        transports:
            babel:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'   # redis://localhost:6379/messages
                                                        # or amqp://guest:guest@localhost:5672/%2f/messages
                serializer: 'babelqueue.messenger.serializer'
        routing:
            'App\Message\OrderCreated': babel
```

Map inbound URNs back to message classes in `config/packages/babelqueue.yaml`, then
dispatch and consume the Messenger way (`messenger:consume babel`). See
[Symfony configuration](/docs/symfony/1.x/configuration/).

## PHP (framework-less core)

The `babelqueue/php-sdk` core is a codec; its optional reference transports move the
bytes. `RedisTransport` (over `predis/predis`) and `AmqpTransport` (over
`php-amqplib`) both publish the encoded envelope:

```php
use BabelQueue\Codec\EnvelopeCodec;
use BabelQueue\Transport\RedisTransport;

$transport = new RedisTransport(new Predis\Client('redis://localhost:6379'));
$env = EnvelopeCodec::make('urn:babel:orders:created', ['order_id' => 1042], 'orders');
$transport->publish(EnvelopeCodec::encode($env), 'orders');
```

Swap `RedisTransport` for `AmqpTransport` to publish to RabbitMQ — see
[PHP transports](/docs/php-sdk/1.x/transports/).

## Python

The Python runtime chooses its transport from the broker URL scheme — `redis://` or
`amqp://`. Install the matching extra:

```python
# pip install "babelqueue[redis]"   →  Redis
# pip install "babelqueue[amqp]"     →  RabbitMQ (via pika)
from babelqueue import BabelQueue

app = BabelQueue("redis://localhost:6379/0", queue="orders")
# app = BabelQueue("amqp://guest:guest@localhost:5672/", queue="orders")

app.publish("urn:babel:orders:created", {"order_id": 1042})
```

Register handlers and run the worker as in
[Python configuration](/docs/babelqueue-python/1.x/configuration/).

## Go

The Go core is a codec; the optional zero-dependency `App` runtime plus a per-broker
transport module gives you publish/consume. Install the module you need:

```go
// go get github.com/babelqueue/babelqueue-go/redis   →  Redis
// go get github.com/babelqueue/babelqueue-go/amqp     →  RabbitMQ
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/redis"
)

tr, err := redis.New("redis://localhost:6379/0") // amqp.New("amqp://guest:guest@localhost:5672/")
if err != nil {
	return err
}
defer tr.Close()

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil // env.Data, env.TraceID
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

The Redis transport runs the reliable-queue pattern; AMQP uses durable queues with
manual ack. See [Runtime & transports](/docs/babelqueue-go/1.x/runtime-and-transports/).

## Node.js

Beyond `@babelqueue/core`, thin transport packages wire it to each broker — you bring
the client:

```ts
// npm install @babelqueue/redis ioredis        →  Redis
import Redis from "ioredis";
import { RedisPublisher, RedisConsumer } from "@babelqueue/redis";

const client = new Redis("redis://localhost:6379/0");

await RedisPublisher.create(client, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });

const consumer = new RedisConsumer(client, "orders", {
  "urn:babel:orders:created": (env) => console.log(env.data.order_id, env.trace_id),
}, { maxTries: 3 });
await consumer.run(() => true);
```

```ts
// npm install @babelqueue/rabbitmq amqplib      →  RabbitMQ
import amqp from "amqplib";
import { RabbitMQPublisher } from "@babelqueue/rabbitmq";

const channel = await (await amqp.connect("amqp://guest:guest@localhost:5672/")).createChannel();
await channel.assertQueue("orders", { durable: true });

await RabbitMQPublisher.create(channel, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });
```

For a BullMQ/NestJS stack on Redis, use `@babelqueue/bullmq` /
`@babelqueue/nestjs` — see [Adapters & transports](/docs/babelqueue-node/1.x/adapters/).

## Java

The Java core is a codec. For **Redis** it ships a first-party transport,
`com.babelqueue:babelqueue-redis` (built on Lettuce):

```java
import com.babelqueue.redis.RedisPublisher;
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.sync.RedisCommands;
import java.util.Map;

RedisClient client = RedisClient.create("redis://localhost:6379");
RedisCommands<String, String> redis = client.connect().sync();

RedisPublisher.create(redis, "orders")
    .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
```

For **RabbitMQ**, use the Spring Boot adapter `com.babelqueue:babelqueue-spring`: its
auto-configured `MessageConverter` wires the canonical envelope into `RabbitTemplate`
(producing) and `@RabbitListener` (consuming):

```java
@Service
class Orders {
    private final BabelQueuePublisher babelQueue;
    Orders(BabelQueuePublisher babelQueue) { this.babelQueue = babelQueue; }

    void create() {
        babelQueue.publish("urn:babel:orders:created", Map.of("order_id", 1042L), "orders");
    }
}
```

See the [Redis transport](/docs/babelqueue-java/1.x/redis/) and
[Spring Boot adapter](/docs/babelqueue-java/1.x/spring/).

## .NET

The .NET core is a codec. For **Redis** it ships `BabelQueue.Redis` (built on
StackExchange.Redis):

```csharp
using BabelQueue.Redis;
using StackExchange.Redis;

var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
var db = redis.GetDatabase();

await new RedisPublisher(db, "orders")
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

For **RabbitMQ**, use the MassTransit adapter `BabelQueue.MassTransit`: register its
`System.Text.Json` envelope converter so MassTransit's RabbitMQ transport carries the
canonical envelope. (You can also encode with `EnvelopeCodec.Encode(...)` and publish
with your own `RabbitMQ.Client` channel.) See the
[Redis transport](/docs/babelqueue-dotnet/1.x/redis/) and
[MassTransit adapter](/docs/babelqueue-dotnet/1.x/masstransit/).

## Cross-language sanity check

Because every recipe writes the same envelope to the same `orders` queue, you can
mix producers and consumers freely — e.g. a Python `redis://` producer and a Go
`…/redis` consumer, or a Laravel `babelqueue-rabbitmq` producer and a Node
`@babelqueue/rabbitmq` consumer. For a runnable end-to-end demo, see
[Cross-language: Python → Go over Redis](/docs/examples/1.x/cross-language-redis/).

Continue to [Dead-letter & tracing](/docs/spec/1.x/dead-letter-and-tracing/).

---

# Dead-letter & tracing

Source: https://babelqueue.com/docs/spec/1.x/dead-letter-and-tracing/

How a failed message lands on <queue>.dlq with the additive dead_letter block, and how trace_id is preserved and forwarded unchanged across every hop — both cross-language, both keeping the envelope frozen at schema_version 1.

Two cross-cutting behaviours every SDK shares, both designed so a failure or a trace
crosses language boundaries unchanged: **dead-lettering** (where a message goes when it
can't be processed) and **trace propagation** (how one correlation id ties a whole chain
of work together). Both are part of the [wire contract](/docs/spec/1.x/envelope/) and
neither touches the frozen `schema_version: 1`.

## Dead-letter queue: `<queue>.dlq`

When a message exhausts its retries, or arrives with a URN no consumer handles (and the
unknown-URN strategy is `dead_letter`), it is **republished to a dead-letter queue**
rather than dropped. The convention is the **source queue name plus a `.dlq` suffix** —
a message that failed on `orders` lands on `orders.dlq`. Because the dead-lettered
message is still the canonical envelope, **any** SDK can read the DLQ and triage it,
regardless of which language produced or failed the message.

The default suffix is `.dlq` across the ecosystem (e.g. Laravel's `dead_letter.suffix`,
Go's `WithDeadLetterSuffix`). Dead-lettering is **opt-in**: enable it per SDK
(`dead_letter: true` in Python, `WithDeadLetter(true)` in Go, `maxTries` + the DLQ path
in the Node transports, the `dead_letter` block in Laravel config). With it disabled, a
terminally failed message is dropped instead.

## The additive `dead_letter` block

A message sitting on a DLQ carries **one extra, optional top-level field**, `dead_letter`,
describing why it failed. It appears **only** on DLQ messages — never on a normal queue —
and is purely **additive**, so the envelope stays frozen at `schema_version: 1`. Normal
consumers ignore it; a triage tool reads it.

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 3,
  "dead_letter": {
    "reason": "failed",
    "error": "Payment gateway timeout",
    "exception": "App\\Exceptions\\GatewayTimeout",
    "failed_at": 1749132730000,
    "original_queue": "orders",
    "attempts": 3,
    "lang": "php"
  }
}
```

The original envelope is preserved **unchanged** inside the dead-lettered message — `job`,
`trace_id`, `data`, `meta` and `attempts` are exactly what they were, so the DLQ message
is a faithful record of what failed and where it came from (`original_queue`). Every SDK
exposes a helper that wraps an envelope in this block and returns a **copy**, leaving the
input untouched:

| SDK | Helper |
| :--- | :--- |
| PHP / Laravel | the `dead_letter` policy in `config/babelqueue.php` (driver applies it) |
| Python | `dead_letter=True` runtime option (runtime applies it) |
| Go | `babelqueue.Annotate(env, reason, queue, attempts, err)` |
| Node.js | `annotate(env, reason, queue, { attempts, error })` from `@babelqueue/core` |
| Java | `DeadLetters.annotate(env, reason, queue, attempts, error, exception)` |
| .NET | `DeadLetters.Annotate(env, reason, queue, attempts: …, error: …)` |

> **`dead_letter` keys are descriptive, not contract envelope fields.** This block is a
> diagnostic side-channel that exists only on the DLQ. Do not confuse it with the frozen
> envelope — and note the forbidden top-level/meta fields (`timestamp`,
> `meta.max_retries`, `meta.source`, `meta.ts`) still never appear, on the DLQ or
> anywhere else. See [forbidden fields](/docs/spec/1.x/envelope/#forbidden-fields).

## Broker-native dead-lettering

Some brokers have a native dead-letter mechanism, and the bindings use it *alongside* the
cross-language `<queue>.dlq` convention rather than instead of it:

- **Redis / RabbitMQ / Go's runtime** — the SDK republishes the wrapped envelope to
  `<queue>.dlq`.
- **Apache Kafka** — no native DLQ, so the SDK owns it: terminal failures go to
  `<topic>.dlq` with the `dead_letter` block (after the tiered `<topic>.retry.<n>`
  topics).
- **Apache Pulsar / ActiveMQ Artemis** — a native dead-letter policy/address exists, and
  the binding also writes the cross-language `<queue>.dlq` so other SDKs can triage.
- **Amazon SQS / Azure Service Bus** — these have first-class redrive / `$DeadLetterQueue`
  facilities you configure on the broker; the envelope on them is still canonical.

See the [broker bindings](/docs/spec/1.x/broker-bindings/) for the per-broker detail.

## Trace propagation (`trace_id`)

`trace_id` is a required, **immutable** UUID that the *first* producer in a chain mints,
and that every SDK then **preserves and forwards unchanged across every hop** — the fourth
golden rule (GR-4). It is how you follow one unit of work as it fans out across services
and languages.

The rule, straight from the [consumer rules](/docs/spec/1.x/envelope/#consumer-rules): when
a consumer produces a downstream message while handling one, it **copies the inbound
`trace_id` onto the new envelope** and mints a **new** `meta.id`. Do not conflate the two —
see [`id` vs `trace_id`](/docs/spec/1.x/envelope/#id-vs-trace_id--do-not-conflate):

| | `meta.id` | `trace_id` |
| :--- | :--- | :--- |
| Scope | one message | one causal chain (may span many messages/services) |
| Minted by | the producing SDK, per message | the **first** producer; reused thereafter |
| Lifetime | dies with the message | propagates across every hop |

### Continuing a trace, per SDK

Every producer API takes an optional inbound trace id; pass it to continue the chain, omit
it to start a fresh one:

```python
# Python — reuse the inbound trace_id on a downstream message
app.publish("urn:babel:shipping:requested", {"order_id": 1042}, trace_id=inbound["trace_id"])
```

```go
// Go — WithTraceID continues the trace; omit it to mint a new one
env, _ := babelqueue.Make(
    "urn:babel:shipping:requested",
    map[string]any{"order_id": 1042},
    babelqueue.WithTraceID(inbound.TraceID),
)
```

```ts
// Node — pass traceId in the options
const env = EnvelopeCodec.make(
  "urn:babel:shipping:requested",
  { order_id: 1042 },
  { traceId: inbound.trace_id },
);
```

```java
// Java — pass the inbound trace id (or null to mint a fresh one)
Envelope env = EnvelopeCodec.make(
    "urn:babel:shipping:requested", Map.of("order_id", 1042L), "orders", inbound.traceId());
```

```csharp
// .NET — pass traceId to continue the trace
var env = EnvelopeCodec.Make(
    "urn:babel:shipping:requested",
    new Dictionary<string, object?> { ["order_id"] = 1042L },
    queue: "orders", traceId: inbound.TraceId);
```

The framework adapters automate this so you never plumb it by hand: Laravel hands the
handler the `$traceId` and reuses it; Symfony's `babelqueue.messenger.trace_middleware`
attaches a `BabelTraceStamp` so any message a handler dispatches **inherits** the inbound
`trace_id`.

### Tracing through the DLQ

Because the dead-lettered message preserves the original envelope unchanged, **`trace_id`
survives dead-lettering too**. A message that failed three hops into a chain still carries
the chain's `trace_id` on `orders.dlq`, so you can correlate the failure back to the
original producer and every service that touched it — in any language.

## In one line

A failure becomes a canonical envelope on `<queue>.dlq` with an additive `dead_letter`
block any SDK can read, and one `trace_id` — minted once, forwarded unchanged — ties the
whole chain together, DLQ included. Neither mechanism changes the frozen
`schema_version: 1`.

Back to [Broker bindings](/docs/spec/1.x/broker-bindings/) · or the
[broker setup recipes](/docs/spec/1.x/broker-recipes/).

---

# Idempotency

Source: https://babelqueue.com/docs/spec/1.x/idempotency/

The meta.id consumption contract and the optional dedupe helper every SDK ships (ADR-0022) — a seen-set Store keyed on meta.id, wrapped around a handler so an already-processed message is skipped, not run twice. Single-process InMemory store for tests; a shared store for production fleets. The wire envelope is untouched at schema_version 1.

At-least-once delivery is the floor every broker gives you: a message **can** arrive
more than once — after a worker crash before ack, a redelivery, or a manual replay. The
[wire contract](/docs/spec/1.x/envelope/) already carries the one field that makes
de-duplication possible across every language: `meta.id`, the unique identity of **this
specific message** (distinct from `trace_id`, which spans a whole causal chain — see
[`id` vs `trace_id`](/docs/spec/1.x/envelope/#id-vs-trace_id--do-not-conflate)).

This page is the cross-SDK contract for **consuming `meta.id`** and the optional,
dependency-free **dedupe helper** each SDK ships to enforce it (ADR-0022).

> **Status:** Authoritative · Helper layer · envelope frozen at `schema_version: 1`

This is a **tooling layer**, not a wire change. The helper adds nothing to the envelope
and removes nothing — `meta.id` is already in `schema_version: 1`. Idempotency is a
choice the consumer makes about how it treats that field.

## The consumption contract

Handlers SHOULD be idempotent (error-handling §1): processing the same `meta.id` twice
MUST be safe. The helper gives you a mechanical way to honour that without rewriting each
handler — it is **seen-set** dedupe, keyed on `meta.id`:

1. **Read `meta.id`.** It is the per-message key. A message with no usable `meta.id`
   runs unchanged (**fail-open**) — the helper never drops work it cannot identify.
2. **If the id was already processed, skip and return.** The runtime acks it, so the
   broker stops redelivering.
3. **Otherwise run the handler, then — on success only — remember the id.** A handler
   that throws leaves the id **unmarked**, so retry / [dead-letter](/docs/spec/1.x/dead-letter-and-tracing/)
   still apply and a later delivery runs the handler again.

This is **post-success dedupe under at-least-once**, not exactly-once and not an in-flight
concurrency lock. The store answers *"was this id processed?"* — never *"what did it
return"*, because queue handlers have no response to replay (unlike an HTTP idempotency
key). The narrow at-least-once window — the handler succeeds but the mark fails — is
documented and bounded by the "handlers are idempotent" guarantee: a redelivery simply
reprocesses harmlessly.

## The `Store`

Every SDK exposes the same three-method record of processed ids, keyed on `meta.id`:

| Operation | Meaning |
| :--- | :--- |
| `seen(id)` | Has this id already been processed (remembered)? |
| `remember(id)` | Record this id as processed. |
| `forget(id)` | Drop an id (manual eviction; a backend may also expire on its own TTL). |

Each SDK ships a reference **in-memory** store implementing those three methods. It is
process-local and **not** persistent — fine for tests and a single-process consumer, but
**not** for a production fleet, where two workers each have their own map and neither sees
the other's "seen" set. Production needs a **shared** store (a Redis key, a database
table, a PSR-16 cache) behind the same interface; the interface is stable, so swapping the
backend is a one-line change.

For the full store contract — the in-memory reference, how to back it with a shared
persistent store, the at-least-once → exactly-once-effect guarantee, and the broker-free
conformance fixtures that lock it — see
[Idempotency Stores & the Effect Guarantee](/docs/spec/1.x/idempotency-stores/).

## Go

The helper lives in the **core** module (stdlib only), so it ships with every consumer.
`idempotency.Wrap(store, handler)` decorates a handler; register it like any other:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/idempotency"
)

store := idempotency.NewInMemoryStore() // tests / single process

app.Handle("urn:babel:orders:created", idempotency.Wrap(store,
	func(ctx context.Context, env babelqueue.Envelope) error {
		// runs at most once per env.Meta.ID, even on redelivery
		return chargeOnce(ctx, env)
	}))
```

`InMemoryStore` satisfies the `Store` interface (`Seen` / `Remember` / `Forget`). For a
production fleet, implement those three methods over Redis or a database and pass your
store to `Wrap` unchanged.

## PHP

`BabelQueue\Idempotency\Idempotent::wrap($store, $handler)` is the PHP mirror, composing
with the consume runtime's ack-on-return / redeliver-on-throw contract:

```php
use BabelQueue\Idempotency\Idempotent;
use BabelQueue\Idempotency\InMemoryStore;

$store = new InMemoryStore(); // tests / single process

$dispatch->on('urn:babel:orders:created', Idempotent::wrap($store,
    function (\BabelQueue\Contracts\ConsumedMessage $message): void {
        chargeOnce($message); // runs at most once per meta.id
    }));
```

A production store implements `BabelQueue\Idempotency\IdempotencyStore` (`seen` /
`remember` / `forget`) over Redis, a database table, or any PSR-16 cache.

## Python

`babelqueue.idempotency.wrap(store, handler)` keeps the handler's signature (via
`functools.wraps`), so the runtime still passes it the right positional args:

```python
from babelqueue import BabelQueue
from babelqueue.idempotency import InMemoryStore, wrap

app = BabelQueue("redis://localhost:6379/0", queue="orders")
store = InMemoryStore()  # tests / single process

app.register("urn:babel:orders:created", wrap(store, on_order_created))
```

A production store implements the `IdempotencyStore` protocol (`seen` / `remember` /
`forget`) over a shared backend.

## Per-SDK

The helper is optional and identical in spirit everywhere — same `Store` shape, same
seen-set semantics, same fail-open on a missing `meta.id`. The in-memory reference is for
tests and single-process consumers; bring a shared store for a fleet.

| SDK | Entry point | Reference store |
| :--- | :--- | :--- |
| Go | `idempotency.Wrap(store, handler)` | `idempotency.NewInMemoryStore()` |
| PHP | `Idempotent::wrap($store, $handler)` | `BabelQueue\Idempotency\InMemoryStore` |
| Python | `idempotency.wrap(store, handler)` | `babelqueue.idempotency.InMemoryStore` |
| Node | `Wrap(store, handler)` | `InMemoryStore` (`@babelqueue/core`) |
| Java | `Idempotent.wrap(store, handler)` | in-memory `Store` (`com.babelqueue.idempotency`) |

See each SDK's reference for store backends and registration:
[Go](/docs/babelqueue-go/1.x/introduction/), [PHP](/docs/php-sdk/1.x/introduction/),
[Python](/docs/babelqueue-python/1.x/introduction/), [Node](/docs/babelqueue-node/1.x/introduction/),
[Java](/docs/babelqueue-java/1.x/introduction/).

For the related guard that stops an **intended** replay from re-firing external
side-effects, see [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/). For the
producer-side mirror that makes the message *commit* atomically with your business write —
removing the dual write — see the [Transactional Outbox](/docs/spec/1.x/outbox/).

Continue to [Observability (OpenTelemetry)](/docs/spec/1.x/observability/).

---

# Idempotency Stores & the Effect Guarantee

Source: https://babelqueue.com/docs/spec/1.x/idempotency-stores/

How the seen-set Store turns at-least-once delivery into an exactly-once effect (ADR-0022): the three-method seen / remember / forget contract every SDK's idempotency helper keys on meta.id, the in-memory reference store that ships in every core, and how to back it with a shared persistent store (Redis, a database table, a cache) for a fleet. The dedupe-key contract and the broker-free cross-SDK conformance fixtures that lock it. The wire envelope is untouched at schema_version 1.

The [Idempotency](/docs/spec/1.x/idempotency/) page covers the helper and the consumption
contract. This page is the companion deep-dive: the **`Store`** behind the helper, the
**exactly-once-effect** guarantee it buys you under at-least-once delivery, the
**dedup-key** contract, and the **store backends** — the in-memory reference every SDK
ships, and how to back it with a shared persistent store for a production fleet.

> **Status:** Authoritative · Helper layer · envelope frozen at `schema_version: 1`

Nothing here touches the wire. The envelope is frozen at `schema_version: 1`; `meta.id`
already carries the per-message identity the dedupe is keyed on. The store is a **consumer
choice** about how it treats that field, not a protocol change.

## At-least-once delivery → exactly-once effect

Every broker BabelQueue speaks is **at-least-once**: a message *can* arrive more than once —
a worker crashes after running the handler but before the ack, a broker redelivers on a
visibility-timeout lapse, a proxy double-sends, an operator replays a queue. The wire
contract makes this explicit and tells handlers to be idempotent (error-handling §1).

The idempotency helper mechanises that: it gives you **exactly-once *effect*** without
exactly-once *delivery*. Delivery stays at-least-once — the broker may still hand you the
same message many times — but the **side-effect** (the charge, the email, the row insert)
fires **once**. The store is what makes the difference: it remembers which `meta.id`s have
already produced their effect, so a redelivery is recognised and skipped.

This is **not** exactly-once delivery (no broker offers that without a transaction the queue
cannot see) and **not** an in-flight concurrency lock — two deliveries of the same id that
race *before the first completes* may both run. It is **seen-set, post-success dedupe**: the
honest, dependency-free guarantee that composes with retry and [dead-letter](/docs/spec/1.x/dead-letter-and-tracing/).

## The dedup-key contract

The dedupe key is **`meta.id`, verbatim** — the canonical, unique identity of *this specific
message*, minted by the first producer and carried unchanged across every hop. It is **not**
`trace_id`: `trace_id` spans a whole causal chain, so many distinct messages share one (see
[`id` vs `trace_id`](/docs/spec/1.x/envelope/#id-vs-trace_id--do-not-conflate)). Keying on
`trace_id` would wrongly collapse unrelated messages; keying on `meta.id` dedupes exactly the
redeliveries of one message.

Because `meta.id` is part of the frozen envelope, the key is **the same in every language**:
a payment a PHP or Python producer sent is de-duplicated by a Go, Java or Node consumer —
same envelope, same identity, one effect. A message with **no usable `meta.id`** (empty or
absent) cannot be deduped, so the helper **fails open**: it runs the handler unchanged rather
than silently dropping work it cannot identify.

## The `Store`

Every SDK exposes the same three-method record of processed ids, keyed on `meta.id`:

| Operation | Meaning |
| :--- | :--- |
| `seen(id)` | Has this id already been processed (remembered)? |
| `remember(id)` | Record this id as processed. |
| `forget(id)` | Drop an id — manual eviction / replay; a backend may also expire ids on its own TTL. |

The helper drives those three methods around your handler:

1. Read `meta.id`. No usable id → run the handler (fail-open) and return.
2. `seen(id)` is true → **skip**: do not invoke the handler; return so the runtime acks it
   and the broker stops redelivering.
3. Otherwise run the handler; **on success only**, `remember(id)`. A handler that throws
   leaves the id **unmarked**, so a redelivery runs it again — retry / dead-letter still
   apply. `forget(id)` later un-remembers an id so a deliberate replay runs once more.

> The store answers *"was this id processed?"* — never *"what did it return"*. Queue handlers
> have no response to replay (unlike an HTTP idempotency key), so a seen-set is all the
> contract needs.

## Store backends

### The in-memory reference (ships in every core)

Each SDK ships a reference **in-memory** store implementing the three methods over a
process-local set or map. It is the right choice for **tests** and a **single-process
consumer**, and it is what the [conformance fixtures](#cross-sdk-conformance) run against.

| SDK | Reference store |
| :--- | :--- |
| Go | `idempotency.NewInMemoryStore()` |
| PHP | `BabelQueue\Idempotency\InMemoryStore` |
| Python | `babelqueue.idempotency.InMemoryStore` |
| Node | `InMemoryStore` (`@babelqueue/core`) |
| Java | in-memory `Store` (`com.babelqueue.idempotency`) |

It is **process-local and not persistent**: two workers each hold their own set, neither
sees the other's "seen" ids, and a restart forgets everything. That is fine for one process;
it is **not** enough for a fleet.

### A shared persistent store

A production fleet needs a **shared** store so every worker dedupes against the same record —
and one that **survives a restart**. The interface is the extension point: implement the
same three methods (`seen` / `remember` / `forget`) over a shared backend and pass your store
to the helper unchanged — no other code changes, because the helper only ever calls those
three methods. You can still bring your own; **Go and PHP now also ship ready-made
persistent stores** so you usually don't have to:

| SDK | Persistent stores |
| :--- | :--- |
| Go | `idempotency-postgres` (`postgres.New(ctx, dsn)` / `postgres.NewWithDB(db)`) and `idempotency-redis` (`redis.New(url)` / `redis.NewWithClient(client)`) — **separate submodules**, so the core stays driver-free. Both take a `WithTable`/`WithPrefix` and a `WithTTL` option; Postgres ships a `Migrate(ctx)` DDL helper. |
| PHP | `BabelQueue\Idempotency\PdoStore` (`new PdoStore($pdo, $table)`, with `PdoStore::ddl()` for the portable `CREATE TABLE`) and `RedisStore` (`new RedisStore($client, $prefix)`, sharing the Predis client with `RedisTransport`). |
| Python · Node · Java · .NET | In-memory reference + **bring your own** behind the same interface (no persistent store ships yet). |

The backends follow the obvious shapes behind the same three methods:

- **Redis** — one key per id with a TTL. `seen` is `EXISTS`, `remember` is `SET` (a TTL caps
  unbounded growth), `forget` is `DEL`. Fast, shared across workers, naturally expiring.
- **A database table** — a `(message_id PRIMARY KEY, …)` row. `remember` is an `INSERT`; a
  unique-constraint conflict *is* the "already seen" signal. Durable, queryable.

> **Pick a TTL deliberately.** A seen-set grows forever without one. Size the TTL to your
> broker's maximum redelivery window — long enough that no legitimate redelivery outlives the
> remembered id, short enough to bound storage. `forget` is the manual escape hatch for a
> planned replay. The shipped stores expose a TTL option (`WithTTL` in Go; a TTL on the
> claim in PHP).

### Closing the in-flight race: the claim contract

The base seen-set is **post-success dedupe**: two deliveries of the same id that race *before
the first completes* may both run (it is not an in-flight lock — see the
[Idempotency](/docs/spec/1.x/idempotency/) page). The shipped persistent stores add an
**atomic claim** to close that window for callers who want it:

- **Go** — `postgres.Store` and `redis.Store` expose `Claim(ctx, id) (bool, error)`: an atomic
  `INSERT … ON CONFLICT DO NOTHING` (Postgres) or `SET … NX PX` (Redis). The winner gets
  `true`; a concurrent duplicate gets `false` and parks.
- **PHP** — the `BabelQueue\Idempotency\ClaimingStore` interface **extends** `IdempotencyStore`
  with `claim($id, $ttlSeconds): bool` and `release($id): void`; `PdoStore` and `RedisStore`
  implement it. `claim` wins the race atomically and marks the id in-flight (TTL'd); `release`
  drops an *uncommitted* claim if the handler errors, while a committed `seen` mark is
  permanent.

This is an opt-in upgrade over the three-method seen-set, not a replacement: a store that
implements only `seen` / `remember` / `forget` still works with the helper unchanged.

### The narrow at-least-once window

Whatever the backend, there is one documented gap: the handler succeeds, then `remember`
fails (or the worker dies) **before** the id is recorded. A later redelivery then reprocesses.
This is bounded and acceptable under "handlers are idempotent" — a redelivery simply re-runs
harmlessly. Closing it entirely requires writing the dedupe record **inside the handler's own
database transaction** — the **transactional** mode (ADR-0022), the consumer-side sibling of
the [transactional outbox](/docs/spec/1.x/outbox/) on the produce side: the PHP `PdoStore` /
Go `postgres.Store` above are the stepping stone, since a DB-backed `remember` can run on the
same connection as the handler's write. The seen-set store is the dependency-free default; the
transactional store is the opt-in upgrade for callers who run their writes through a DB the
store can join.

## Cross-SDK conformance

The dedupe contract is locked by the **broker-free** cross-SDK
[conformance suite](https://github.com/BabelQueue/conformance) — the `idempotency` block in
`manifest.json`, vendored into every core SDK. It is transport-agnostic: pure envelope +
expected-outcome assertions over a single in-process store, so every SDK's helper is proven
to make the same decision.

- **`dedup_key`** — the key is `meta.id` verbatim; two deliveries with the same `meta.id`
  collapse to one effect, distinct ids never collapse.
- **`sequences`** — ordered delivery scenarios, each asserting whether a delivery **runs** the
  handler or is **skipped**, and the total side-effects after the sequence:
  - a duplicate delivery runs **once**;
  - an at-least-once redelivery storm is a **no-op**;
  - distinct ids each run;
  - a **throwing** handler leaves the id unmarked, so a later redelivery runs it (retry
    survives the guard);
  - a missing `meta.id` **fails open** (runs every time);
  - `forget` allows a re-run.

Because the assertions are the *decision* (run vs. skip) over the in-memory reference store,
the block is **store-agnostic** — a persistent backend that honours the same three-method
contract passes it unchanged.

## Per-SDK entry points

The helper is optional and identical in spirit everywhere — same `Store` shape, same seen-set
semantics, same fail-open on a missing `meta.id`.

| SDK | Entry point | Reference store |
| :--- | :--- | :--- |
| Go | `idempotency.Wrap(store, handler)` | `idempotency.NewInMemoryStore()` |
| PHP | `Idempotent::wrap($store, $handler)` | `BabelQueue\Idempotency\InMemoryStore` |
| Python | `idempotency.wrap(store, handler)` | `babelqueue.idempotency.InMemoryStore` |
| Node | `Wrap(store, handler)` | `InMemoryStore` (`@babelqueue/core`) |
| Java | `Idempotent.wrap(store, handler)` | in-memory `Store` (`com.babelqueue.idempotency`) |
| .NET | `Idempotency.Wrap(store, handler)` | `InMemoryStore` (`IIdempotencyStore`, `BabelQueue`) |

For a runnable, broker-free walkthrough (no Redis, no Docker) see the
[in-memory idempotency example](https://github.com/BabelQueue/babelqueue-examples/tree/main/idempotency-inmemory);
for the cross-language Python → Go version over Redis, the
[idempotency-payments example](https://github.com/BabelQueue/babelqueue-examples/tree/main/idempotency-payments).

This guard stops an **accidental** duplicate from re-running an effect. To stop an **intended**
replay (off a DLQ) from re-firing effects that already ran, see
[DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/) — the complementary primitive.
The producer-side sibling that makes the message *commit* once is the
[Transactional Outbox](/docs/spec/1.x/outbox/). To keep one trace across the deduped hop, see
[Observability (OpenTelemetry)](/docs/spec/1.x/observability/).

Back to the [Idempotency contract](/docs/spec/1.x/idempotency/).

---

# Transactional Outbox

Source: https://babelqueue.com/docs/spec/1.x/outbox/

The optional producer-side outbox helper every SDK ships (ADR-0029): a caller-owned DB transaction writes the business row and the encoded envelope together, then a separate relay publishes the stored bytes verbatim — removing the producer dual write with no envelope change and no DB driver in the core. The OutboxStore save / fetchUnpublished / markPublished / markFailed contract, the Outbox writer, the OutboxRelay flush/drain at-least-once handoff, and the in-memory reference store. The consumer-side mirror of idempotency. The wire envelope stays frozen at schema_version 1.

[Idempotency](/docs/spec/1.x/idempotency/) deduplicates on the **consumer** side. The
outbox is its mirror on the **producer** side: it removes the **dual write** — the
crash window between committing your business row and publishing the message — so the
message commits *atomically with the business data*, then a separate relay hands it to
the broker (ADR-0029).

> **Status:** Authoritative · Helper layer · envelope frozen at `schema_version: 1`

Nothing here touches the wire. The outbox **stores the existing encoded envelope verbatim**
and the relay **publishes those exact bytes** — `trace_id` preserved, byte-compatible. The
outbox's own columns (id, status, attempts) are bookkeeping *around* the envelope, never
*on* it. The envelope stays frozen at `schema_version: 1`.

## The dual write, and why a transaction can't span it

A plain producer does two things that must both happen or neither: commit the business row
(`INSERT INTO orders …`) and publish the message to the broker. They are two independent
systems. A crash *between* them leaves the order saved with no message, or a message sent
for an order that rolled back. There is no distributed transaction to lean on —
BabelQueue is broker-agnostic and deliberately avoids one (GR-7), and no broker offers a
commit the database can see.

The outbox pattern removes the dual write by making the publish a **local** write first:

1. In the **same database transaction** as the business row, persist the encoded envelope
   into an **outbox table**. Commit both together — they succeed or roll back atomically.
2. A separate **relay** reads the durable outbox rows afterwards, publishes each to the
   broker, and marks it published. The handoff to the broker is now a local problem (read a
   row, publish, mark it), not a cross-system one.

The honest scope is **exactly-once handoff into the broker, then at-least-once on the
wire** as always: a crash between *publish* and *mark-published* re-publishes the row on the
next pass. That redelivery is deduped by the consumer on `meta.id` —
[idempotency](/docs/spec/1.x/idempotency/) is the consumer-side mirror that closes the loop.

## The `OutboxStore`

Every SDK exposes the same four-method persistence contract. The **core defines it and binds
to no DB driver** (GR-7); a concrete adapter over your database is the caller's:

| Operation | Meaning |
| :--- | :--- |
| `save(encodedEnvelope, queue)` | Persist the encoded envelope bytes + its target queue; return the new row id. **Called inside the caller's open transaction.** |
| `fetchUnpublished(limit)` | Read up to `limit` pending rows, **oldest-first**. A production adapter SHOULD claim/lock the rows it returns (e.g. `SELECT … FOR UPDATE SKIP LOCKED`) so two relays don't double-publish. |
| `markPublished(ids)` | Mark these rows published — called only **after** the transport accepts them. |
| `markFailed(id, error)` | Record a publish failure and leave the row **pending** for a later pass. |

The transaction boundary is **the caller's**. `save()` runs on the transaction you already
opened around your business write; you commit both together. The store never begins or
commits anything — that is the whole point, and the reason no DB driver enters the core.

## The `Outbox` writer

The writer is tiny: `Outbox.write(envelope)` encodes the envelope via the **frozen
`EnvelopeCodec`** (bytes unchanged), captures `meta.queue` (else `"default"`), and delegates
to `OutboxStore.save()`. **It does not begin or commit anything.** You call it inside your
own transaction, right beside your business write:

```text
begin transaction
    INSERT INTO orders (…)            -- business row
    outbox.write(envelope)           -- the message, same transaction
commit                                -- both, or neither
```

## The `OutboxRelay`

The relay drives the broker handoff through the existing publish-only `Transport` seam:

- **`flush()`** publishes **one batch** of pending rows. Each row is marked **published only
  after** the transport accepts it; a failing publish is caught (`markFailed`, row left
  pending) with a **bounded linear backoff**, and the batch continues — one poison row never
  blocks the others. The sleeper is injectable, so tests run instantly.
- **`drain()`** loops `flush` while a pass makes progress, with a safety ceiling.

The relay publishes the **stored bytes verbatim** — it never decodes, rebuilds, or
re-encodes the envelope. So `trace_id` is preserved end-to-end (GR-4) and the published
bytes are byte-identical to what the producer encoded (GR-1/GR-5).

## The in-memory reference store

Every core ships an `InMemoryOutboxStore` for tests and single-process demos. It honours the
four-method contract over a process-local, insertion-ordered map (oldest-first) but has **no
real transaction** and does **not** claim/lock rows. Production needs a **DB-backed adapter**
— the [InitORM outbox example](https://github.com/BabelQueue/babelqueue-examples/tree/main/outbox-initorm)
is the reference for one (the outbox-table DDL, a `produce.php` that does the business write
and `outbox->write` in one `$db->transaction`, and a `relay.php`), keeping the core itself
DB-free.

## Go

The helper is the `…/outbox` subpackage of the core module (stdlib only, like
`idempotency`/`schema`). `outbox.New(store)` is the writer; `outbox.NewRelay(transport,
store, opts)` is the relay:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/outbox"
)

store := outbox.NewInMemoryStore() // tests / single process
box := outbox.New(store)

// inside the caller's own DB transaction, beside the business write:
env, _ := babelqueue.Make("urn:babel:orders:created",
	map[string]any{"order_id": 1042}, babelqueue.WithQueue("orders"))
id, err := box.Write(env) // encodes via the frozen codec, calls Store.Save — no commit

// later, a relay drains the durable rows to the broker:
relay := outbox.NewRelay(transport, store, outbox.Options{})
res, err := relay.Drain(ctx, 0) // res.Published / res.Failed
```

`Store` is the interface (`Save` / `FetchUnpublished` / `MarkPublished` / `MarkFailed`); a
production fleet implements it over Postgres/MySQL — binding `Save` to the caller's open
transaction — and passes it to `New` and `NewRelay` unchanged.

## PHP

`BabelQueue\Outbox\Outbox::write()` mirrors the Go writer; `OutboxRelay` drains through the
publish-only `Transport`:

```php
use BabelQueue\Outbox\Outbox;
use BabelQueue\Outbox\OutboxRelay;
use BabelQueue\Outbox\InMemoryOutboxStore;

$store = new InMemoryOutboxStore(); // tests / single process
$outbox = new Outbox($store);

// inside your own $db->transaction(...), beside the business write:
$id = $outbox->write($envelope); // encodes via EnvelopeCodec, calls $store->save()

// later: a relay hands the durable rows to the broker
$result = (new OutboxRelay($transport, $store))->drain();
// $result->published / $result->failed
```

A production store implements `BabelQueue\Outbox\OutboxStore` (`save` / `fetchUnpublished` /
`markPublished` / `markFailed`) over a PDO/ORM connection, with `save()` running on the
caller's open transaction — see the InitORM example's `InitOrmOutboxStore`.

## Per-SDK

The outbox is optional and identical in spirit everywhere — same four-method `OutboxStore`,
same caller-owned transaction, same `write` / `flush` / `drain`, same verbatim-bytes relay.
It shipped across all six SDK cores (per-SDK MINORs; the envelope stayed frozen).

| SDK | Writer · Relay | Store · reference |
| :--- | :--- | :--- |
| Go | `outbox.New(store)` `.Write` · `outbox.NewRelay(t, store)` `.Flush`/`.Drain` | `outbox.Store` · `outbox.NewInMemoryStore()` |
| PHP | `new Outbox($store)` `->write` · `new OutboxRelay($t, $store)` `->flush`/`->drain` | `BabelQueue\Outbox\OutboxStore` · `InMemoryOutboxStore` |
| Python | `Outbox(store)` `.write` · `OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`babelqueue.outbox`) |
| Node | `new Outbox(store)` `.write` · `new OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`@babelqueue/core`) |
| Java | `new Outbox(store)` `.write` · `new OutboxRelay(transport, store)` `.flush`/`.drain` | `OutboxStore` · `InMemoryOutboxStore` (`com.babelqueue.outbox`) |
| .NET | `new Outbox(store)` `.WriteAsync` · `new OutboxRelay(publisher, store)` `.FlushAsync`/`.DrainAsync` | `IOutboxStore` · `InMemoryOutboxStore` (`BabelQueue.Outbox`) |

Two idiomatic differences to expect: **.NET** is async throughout (`WriteAsync` /
`FetchUnpublishedAsync` / `FlushAsync` / `DrainAsync`, `CancellationToken`-threaded) and its
store interface is `IOutboxStore`; **Python**'s method names are snake-case
(`fetch_unpublished` / `mark_published` / `mark_failed`) and its relay backoff is in seconds
where the others are in milliseconds. The broker seam the relay forwards through is the
SDK's existing publish surface — a `Transport` (Go/PHP/Python), an `OutboxTransport`
(Node/Java), or an `OutboxPublisher` delegate (.NET).

The DB-backed adapter stays the caller's in every SDK — the cores ship only the in-memory
reference, exactly as on the PHP side. Relay concurrency (claim/lock so two relays don't
double-publish), and retention/archival of published rows, are the adapter's concern.

See each SDK's reference for wiring a store:
[Go](/docs/babelqueue-go/1.x/introduction/), [PHP](/docs/php-sdk/1.x/introduction/),
[Python](/docs/babelqueue-python/1.x/introduction/), [Node](/docs/babelqueue-node/1.x/introduction/),
[Java](/docs/babelqueue-java/1.x/introduction/), [.NET](/docs/babelqueue-dotnet/1.x/introduction/).

The outbox guarantees the message is *sent* once you commit; [idempotency](/docs/spec/1.x/idempotency/)
guarantees its *effect* fires once on the consumer despite the at-least-once redelivery the
relay's re-publish can cause. Use them together. To keep one trace across the produce→consume
hop, see [Observability (OpenTelemetry)](/docs/spec/1.x/observability/).

---

# Observability (OpenTelemetry)

Source: https://babelqueue.com/docs/spec/1.x/observability/

How every SDK maps the envelope's trace_id to an OpenTelemetry TraceID (ADR-0025) — UUID to 16 bytes, else SHA-256 — and emits a CONSUMER 'process <urn>' span and PRODUCER 'publish <urn>' span with messaging-semconv attributes, so a trace_id shared across hops becomes one OTel trace. The v0.2 W3C traceparent transport header (ADR-0028) adds true cross-hop span parent-child linkage, now in all six SDK cores. Retry and DLQ are visible via attempts and error status. An opt-in module; the wire envelope stays frozen at schema_version 1.

The [wire contract](/docs/spec/1.x/envelope/) already carries the one field a tracing
system needs: `trace_id`, a UUID **generated once by the first producer and forwarded
unchanged across every hop** (GR-4). That is what makes a Python producer and a Go
consumer show up on **one** trace. This page is the cross-SDK contract for projecting
that `trace_id` into [OpenTelemetry](https://opentelemetry.io/) (ADR-0025) and the
optional module each SDK ships to do it — plus the [W3C `traceparent`](#w3c-traceparent--true-cross-hop-span-linkage)
transport header (ADR-0028) that layers true cross-hop span linkage on top.

> **Status:** Authoritative · Optional module · envelope frozen at `schema_version: 1`

The OTel module is a **tooling layer**. The core never imports OpenTelemetry — wiring a
`TracerProvider` is opt-in (a Go submodule, a Python/Node extra, a PHP component). Nothing
is added to the envelope: correlation rides the `trace_id` that is already there.

## `trace_id` ↔ OTel TraceID — a deterministic bijection

OTel TraceIDs are 128-bit (16 bytes); BabelQueue `trace_id`s are UUID strings. The mapping
is deterministic and the same in every SDK, so any two hops that share a `trace_id` derive
the **same** TraceID:

- **A UUID `trace_id`** → its **16 raw bytes**, verbatim. The inverse formats those 16
  bytes back to a canonical UUID string, so a producer can stamp the active trace's id
  into `trace_id` and the consumer recovers exactly that TraceID.
- **Any other string** → **SHA-256, first 16 bytes** (and never the all-zero invalid
  TraceID). This keeps non-UUID `trace_id`s usable without breaking the
  [envelope rule](/docs/spec/1.x/envelope/#producer-rules) that producers emit a UUID.

Because the function is pure, no out-of-band header is needed for trace **identity** — the
`trace_id` field is the carrier. Exact cross-hop *span* parent-child linkage layers on top of
this via the W3C `traceparent` transport header ([below](#w3c-traceparent--true-cross-hop-span-linkage)):
the `trace_id` bijection is the v0.1 floor (every hop sharing a `trace_id` shares one trace),
and a delivered `traceparent` upgrades the consumer span to a true child of the producer span.

## Spans and attributes

Each SDK emits two span kinds, named to the [messaging semantic conventions](https://opentelemetry.io/docs/specs/semconv/messaging/):

| Span | Kind | Name | When |
| :--- | :--- | :--- | :--- |
| Consume | `CONSUMER` | `process <urn>` | per handled message |
| Publish | `PRODUCER` | `publish <urn>` | per published message |

The CONSUMER span carries the messaging-semconv attributes drawn from the envelope:

| Attribute | Source |
| :--- | :--- |
| `messaging.system` | `babelqueue` |
| `messaging.operation` | `process` |
| `messaging.destination.name` | `meta.queue` |
| `messaging.message.id` | `meta.id` |
| `messaging.message.conversation_id` | `trace_id` |
| `messaging.babelqueue.attempts` | top-level `attempts` |

**Retries and dead-letters are visible without extra plumbing.** `attempts` is on every
CONSUMER span, so a redelivery (`attempts: 2`) is distinguishable from a first attempt
(`attempts: 0`) on the trace. A handler that throws records the error on its span and sets
the span status to **Error**, so the failures that eventually
[dead-letter](/docs/spec/1.x/dead-letter-and-tracing/) a message are right there in the
trace timeline.

## W3C `traceparent` — true cross-hop span linkage

The `trace_id` bijection (v0.1) gives you **shared-trace correlation**: every hop that shares
a `trace_id` shows up on one trace. What it does **not** carry is the exact **span**
parent-child edge — the producer span's `span_id` as the consumer span's parent — so a trace
UI shows the spans together but can't draw the real produce→consume edge or attribute per-hop
latency to a specific parent span. The **v0.2** mechanism (ADR-0028) closes that, and it now
ships in **all six SDK cores**.

It rides the **out-of-band transport-header seam** (the same `HeaderPublisher` /
`ReceivedMessage.Headers` mechanism that carries the [replay-bypass marker](/docs/spec/1.x/redrive-and-replay/)) —
**not** an envelope field, so the wire stays frozen at `schema_version: 1`:

- **Producer** — injects the active span context as the standard W3C `traceparent` (and
  `tracestate`) header via OTel's `TraceContext` propagator, then publishes through the
  header-carrying transport. It **also still stamps `trace_id`** (the v0.1 path), so a
  header-blind consumer or a transport that drops headers recovers the same trace.
- **Consumer** — reads `traceparent` off the delivered message's headers, extracts the
  **remote parent**, and starts the `process <urn>` span as a **child of the producer span**.
  Absent the header, it falls back to the v0.1 `trace_id`-derived parent.

So enabling propagation is a **strict upgrade, never a regression**: a message produced
without a `traceparent` behaves exactly as v0.1. The public surface (`Publish` /
`WrapHandler` and their per-SDK equivalents) is unchanged in shape — the upgrade is internal —
and because the W3C format is the standard one, a babelqueue `traceparent` interoperates with
any OTel SDK or W3C-compliant peer.

**Per-transport coverage is the honest limit.** Propagation works on any transport that
implements the header capability and surfaces inbound headers; until a given broker binding
wires it, a publish on that broker degrades to v0.1 `trace_id` correlation with **no error**.
Today the in-tree reference transports carry it (in-memory, Redis, AMQP/RabbitMQ, SQS), and
the Node/.NET/Java adapter transports are wired across their native per-message metadata
channels (AMQP headers, SQS `MessageAttributes`, Kafka/Pulsar/Artemis/Azure properties,
BullMQ telemetry slot, a transport-owned Redis frame). The remaining gap is **PHP's
Kafka/Pulsar/STOMP producers**, a documented follow-up.

## Go

The OTel integration lives in its **own module** so the core never imports OpenTelemetry.
`otel.WrapHandler(tracer, handler)` emits the CONSUMER span; `otel.Publish(...)` emits the
PRODUCER span and stamps the active trace's id into `trace_id`:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	bqotel "github.com/babelqueue/babelqueue-go/otel"
)

// consumer: a "process <urn>" span per message, in the trace derived from trace_id
app.Handle("urn:babel:orders:created", bqotel.WrapHandler(tracer,
	func(ctx context.Context, env babelqueue.Envelope) error {
		return handle(ctx, env)
	}))

// producer: a "publish <urn>" span; the downstream consumer recovers the same trace
id, err := bqotel.Publish(ctx, tracer, app, "urn:babel:orders:created",
	map[string]any{"order_id": 1042})
```

`bqotel.TraceIDOf(traceID)` and `bqotel.UUIDOf(t)` expose the bijection directly if you
need to correlate by hand.

## Python

The module is reachable only with the `[otel]` extra (`pip install babelqueue[otel]`), so
importing the core stays dependency-free:

```python
from opentelemetry import trace
from babelqueue import BabelQueue, otel

tracer = trace.get_tracer("orders")
app = BabelQueue("redis://localhost:6379/0", queue="orders")

# consumer span: "process <urn>"
app.register("urn:babel:orders:created", otel.wrap_handler(tracer, on_order_created))

# producer span: "publish <urn>" — carries the active trace's id into trace_id
otel.publish(tracer, app, "urn:babel:orders:created", {"order_id": 1042})
```

`otel.trace_id_of(...)` / `otel.uuid_of(...)` expose the same bijection.

## Per-SDK

The OTel module is optional and identical in spirit everywhere — same two span names, same
messaging-semconv attributes, same `trace_id` ↔ TraceID bijection, and the same v0.2 W3C
`traceparent` span linkage, all driven off the frozen envelope.

| SDK | How it ships | Entry points |
| :--- | :--- | :--- |
| Go | `…/otel` submodule | `otel.WrapHandler`, `otel.Publish`, `TraceIDOf` / `UUIDOf` |
| Python | `babelqueue[otel]` extra | `otel.wrap_handler`, `otel.publish`, `trace_id_of` / `uuid_of` |
| Node | `@babelqueue/core/otel` subpath | `wrapHandler`, `publish`, `traceIdOf` / `uuidOf` |
| Java | `com.babelqueue.otel.Tracing` | `Tracing.wrapHandler`, `Tracing.publish`, `traceIdOf` / `uuidOf` |
| .NET | `BabelQueue.Core` `Telemetry` | `Telemetry.Wrap`, `Telemetry.PublishAsync` (header-aware overloads) |
| PHP | `BabelQueue\Otel\Tracing` | producer/consumer span helpers (`Tracing::wrap`, `Tracing::publish`) |

All six cores implement the v0.2 `traceparent` mechanism (`traceparent` injected on publish,
extracted on consume to parent the span); it activates on any transport that carries headers,
and degrades cleanly to v0.1 `trace_id` correlation otherwise.

Trace **identity** is bridged on every transport via the `trace_id` bijection (v0.1);
exact span **parent-child** linkage across hops, via the W3C `traceparent` transport header
([above](#w3c-traceparent--true-cross-hop-span-linkage)), now ships in all six cores and on
every transport that carries headers — the remaining gap is PHP's Kafka/Pulsar/STOMP
producers, where propagation cleanly degrades to v0.1 `trace_id` correlation.

See each SDK's reference for wiring a `TracerProvider`:
[Go](/docs/babelqueue-go/1.x/introduction/), [Python](/docs/babelqueue-python/1.x/introduction/),
[Node](/docs/babelqueue-node/1.x/introduction/), [Java](/docs/babelqueue-java/1.x/introduction/),
[PHP](/docs/php-sdk/1.x/introduction/).

Continue to [Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Per-URN Schema Validation

Source: https://babelqueue.com/docs/spec/1.x/schema-validation/

Optional, opt-in validation of a message's data block against a JSON Schema registered for its URN (ADR-0024), plus the broker-free babelqueue-registry / bqschema tooling that governs those schemas in git and CI. A Provider supplies the schema; Check validates producer-side (preferred) and Wrap validates consumer-side. The wire envelope stays frozen at schema_version 1.

The [wire contract](/docs/spec/1.x/envelope/) freezes the **envelope**, but deliberately
keeps `data` as *"pure JSON the caller validates"* — the business payload differs per URN,
so the contract cannot fix its shape. [URN naming §6](/docs/spec/1.x/urn/) **recommends**
that teams keep a checked-in, per-URN registry with a JSON Schema for each message's
`data`. This page is the cross-SDK contract for **enforcing** those schemas (ADR-0024) and
the `babelqueue-registry` / `bqschema` tooling that **governs** them.

> **Status:** Authoritative · Opt-in helper + tooling · envelope frozen at `schema_version: 1`

This is a **tooling layer over `data`**, not a wire change. The envelope is untouched; what
is validated is the `data` block, against a schema *you* register for a URN. It is
**opt-in**: a URN with no registered schema is never validated.

## The model: a Provider per URN

Every SDK exposes a `Provider` (interface / protocol) that answers one question — *what is
the JSON Schema for this URN's `data`?* — returning the schema, or "none registered" so
the caller skips validation. Two reference providers ship in each SDK:

- **`MapProvider`** — in-memory, for tests and for embedding schemas in code.
- **`DirProvider`** — the **registry bridge**: it reads a `babelqueue-registry`
  `registry.json` manifest (a list of `{urn, schema}` entries) and loads each URN's
  schema file lazily. This is what makes the registry's git-governed schemas enforceable
  at runtime.

The validator is an intentionally small **subset of JSON Schema draft-07** — `type`,
`required`, `properties`, `additionalProperties`, `items`, `enum`, `const`, `minLength`,
`minimum` — enough for real `data` shapes. Unknown keywords are ignored, and the verdicts
match across Go, PHP, Python and `bqschema`, so a schema validated in CI validates
identically at runtime. Zero dependencies (stdlib only).

## Producer-side `Check` is preferred

Validation can run on either side, but the sides are **not** equivalent:

- **Producer-side (`Check` / `assert` / `validate`) — preferred.** Call it before
  publishing so invalid `data` never enters the queue. A violation is caught at the source,
  where the caller can fix it, and the bad message is never written.
- **Consumer-side (`Wrap`) — a safety net.** Wrapping a handler validates each message
  before it runs. But invalid `data` will **not** become valid on retry, so a poison
  message exhausts its attempts and is
  [dead-lettered](/docs/spec/1.x/dead-letter-and-tracing/). That is correct fail-safe
  behaviour, but it spends retries on a message that can never succeed — hence
  producer-side `Check` is the recommended primary gate, with `Wrap` as defence in depth.

## Go

`schema.Check(provider, urn, data)` is the producer guard; `schema.Wrap(provider, handler)`
is the consumer safety net. `schema.Validate(provider, env)` is the envelope form of
`Check`. The package is in the **core** module (stdlib only):

```go
import "github.com/babelqueue/babelqueue-go/schema"

// DirProvider bridges a babelqueue-registry manifest to runtime validation
provider, err := schema.NewDirProvider("registry.json")
if err != nil {
	return err
}

// producer-side (preferred): keep invalid data out of the queue
if err := schema.Check(provider, "urn:babel:orders:created",
	map[string]any{"order_id": 1042, "amount": 99.90}); err != nil {
	return err // errors.Is(err, schema.ErrInvalidPayload)
}

// consumer-side safety net
app.Handle("urn:babel:orders:created", schema.Wrap(provider, handler))
```

`schema.NewMapProvider(map[string][]byte{...})` builds an in-memory provider for tests.

## PHP

`BabelQueue\Schema\SchemaValidated` mirrors the Go helper — `assert` (throwing producer
guard), `check` (non-throwing, for branching), and `wrap` (consumer safety net):

```php
use BabelQueue\Schema\DirProvider;
use BabelQueue\Schema\SchemaValidated;

$provider = new DirProvider('registry.json'); // registry bridge

// producer-side (preferred)
SchemaValidated::assert($provider, 'urn:babel:orders:created', [
    'order_id' => 1042,
    'amount'   => 99.90,
]); // throws InvalidPayloadException on mismatch

// consumer-side safety net
$dispatch->on('urn:babel:orders:created', SchemaValidated::wrap($provider, $handler));
```

`new BabelQueue\Schema\MapProvider([...])` is the in-memory provider.

## Python

`babelqueue.schema` mirrors the same surface — `validate` (raising producer guard),
`check` (non-raising), and `wrap` (consumer safety net, taking the URN explicitly since a
Python handler receives `data` positionally):

```python
from babelqueue.schema import DirProvider, validate, wrap

provider = DirProvider("registry.json")  # registry bridge

# producer-side (preferred)
validate(provider, "urn:babel:orders:created",
         {"order_id": 1042, "amount": 99.90})  # raises InvalidPayloadError on mismatch

# consumer-side safety net
app.register("urn:babel:orders:created", wrap(provider, "urn:babel:orders:created", handler))
```

`MapProvider.from_json({...})` builds the in-memory provider from raw schema strings.

## Governing schemas: `babelqueue-registry` and `bqschema`

The runtime helpers **enforce** schemas; the `babelqueue-registry` project **governs**
them. `bqschema` is a CLI you run in CI — **no Kafka, no service, no database**. Schemas
live in your git repo as files, referenced from a `registry.json` manifest:

```json
{
  "schemas": [
    { "urn": "urn:babel:orders:created", "schema": "schemas/orders-created.json", "owner": "orders" }
  ]
}
```

```sh
go install github.com/babelqueue/babelqueue-registry/cmd/bqschema@latest

# does this message's data match the schema registered for its URN?
bqschema validate --registry registry.json messages/order-created.json

# is this schema change backward-compatible, or does it break consumers?
bqschema compat schemas/orders-created.json schemas/orders-created.v2.json

# sanity-check the registry itself (every schema parses)
bqschema check --registry registry.json

# generate an AsyncAPI 3.0 event catalog from the registry
bqschema export-asyncapi --registry registry.json -o asyncapi.json

# inventory / audit / mask the GDPR-sensitive fields declared in the registry
bqschema gdpr --registry registry.json                 # inventory the x-gdpr-sensitive paths
bqschema gdpr --registry registry.json --require       # CI gate: fail on un-annotated PII
bqschema gdpr --registry registry.json --mask msg.json # redact PII for safe logging
```

`compat` enforces the [versioning policy](/docs/spec/1.x/envelope/#change-control) at the
`data` level, the same way the envelope rules work for the wire: adding an **optional**
field is compatible; removing / renaming / retyping a field, or making an optional field
required, is **breaking** — the signal to **mint a new URN** (`…:created.v2`) rather than
mutate the existing one, so consumers can upgrade before producers. Unlike a
broker-coupled schema registry, the schemas are plain git files, so there is no
cold-start circular dependency and the gate works identically across Redis, RabbitMQ, SQS,
Kafka, Pulsar and the rest.

### GDPR-sensitive fields: declare and audit

The same registry is where PII is **declared**. An `x-gdpr-sensitive` keyword on a property
(value `true` or a category string like `"email"`) marks a `data` field as personal data. It
is **validation-neutral** — parsed but ignored by `validate` and `compat`, so annotating an
existing schema is never a breaking change — and it flows through into the AsyncAPI catalog
automatically. `bqschema gdpr` inventories the marked paths, `--require` is a **CI gate** that
fails when a PII-named field is left un-annotated, and `--mask` redacts a message for safe
logging (one-way, registry-side — **not** a crypto primitive). The matching **runtime**
encryption that reads the same keyword to protect those fields on the wire is the SDK's job —
see [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/) (ADR-0030).

## Per-SDK

Validation is optional and opt-in everywhere — same `Provider` model, same draft-07
subset, same producer-preferred / consumer-safety-net split. The **registry bridge**
(`DirProvider`) ships in Go, PHP and Python; the in-memory **`MapProvider`** is the path
in Node, Java and .NET.

| SDK | Producer guard | Consumer wrap | Registry bridge |
| :--- | :--- | :--- | :--- |
| Go | `schema.Check` / `Validate` | `schema.Wrap` | `schema.DirProvider` |
| PHP | `SchemaValidated::assert` / `check` | `SchemaValidated::wrap` | `BabelQueue\Schema\DirProvider` |
| Python | `validate` / `check` | `wrap` | `babelqueue.schema.DirProvider` |
| Node | `validateSchema` | `wrap` | `MapProvider` |
| Java | `SchemaValidation` | `SchemaValidation` | `MapProvider` |
| .NET | core validator | core validator | `MapProvider` |

See each SDK's reference and the
[babelqueue-registry README](https://github.com/BabelQueue/babelqueue-registry) for
`bqschema` details:
[Go](/docs/babelqueue-go/1.x/introduction/), [PHP](/docs/php-sdk/1.x/introduction/),
[Python](/docs/babelqueue-python/1.x/introduction/), [Node](/docs/babelqueue-node/1.x/introduction/),
[Java](/docs/babelqueue-java/1.x/introduction/).

Continue to [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

---

# GDPR Field Encryption

Source: https://babelqueue.com/docs/spec/1.x/gdpr-field-encryption/

How every SDK encrypts the PII leaves of a message's data in place (ADR-0030): the validation-neutral x-gdpr-sensitive schema keyword declares which fields carry personal data; a caller-bound Cipher seam (so the core pulls no crypto dependency) plus opt-in protect / unprotect helpers walk the schema's sensitive paths and rewrite only the marked values, leaving data pure JSON and the envelope frozen. The registry's bqschema gdpr declares and audits the same keyword. Validate cleartext, before protect and after unprotect. The wire envelope stays frozen at schema_version 1.

A message's [`data`](/docs/spec/1.x/envelope/) is, by contract, **pure JSON the caller owns**.
When it carries personal data — an email, a national id, a postal address — GDPR/KVKK ask
you to encrypt those fields at rest and in transit. BabelQueue does this **per field**, not
per envelope: the SDK encrypts exactly the leaves you mark as sensitive, in place, leaving
the rest of `data` (and the whole envelope frame) untouched (ADR-0030).

> **Status:** Authoritative · Optional helper · envelope frozen at `schema_version: 1`

This is purely additive and validation-neutral. A protected value is still a JSON **string**
(the ciphertext), so `data` stays pure JSON (GR-3), the envelope is unchanged
(`schema_version: 1`, `trace_id` preserved — GR-1/GR-4), and an SDK without the key can
still carry the message; it just can't read the protected fields. The crypto is **opt-in**
and lives in a `gdpr` module the core never imports by default.

## Two halves: declare/audit, and enforce

GDPR support is split exactly along the line between the registry and the SDKs:

- **Declare + audit (the [registry](/docs/spec/1.x/schema-validation/), `bqschema gdpr`).**
  An `x-gdpr-sensitive` JSON-Schema keyword marks which `data` fields are PII. The registry
  inventories them, **gates CI** when a PII-named field is left un-annotated, and **masks**
  a message for safe logging. This is governance — it never sees runtime traffic.
- **Enforce at runtime (the SDKs).** Each SDK reads the **same** `x-gdpr-sensitive`
  annotation off the per-URN schema and encrypts/decrypts those leaves when it produces and
  consumes. This page is that runtime half.

The registry's `gdpr --mask` is the **one-way, registry-side logging** equivalent — it
redacts a value for a log line; it is explicitly **not** a crypto primitive and does not
protect data on the wire. That is the SDK's job.

## The `x-gdpr-sensitive` keyword

A property-level JSON-Schema extension on the per-URN `data` schema, accepting either `true`
or a non-empty **string category** (`"email"`, `"national_id"`, …):

```json
{
  "type": "object",
  "properties": {
    "order_id": { "type": "integer" },
    "email":    { "type": "string", "x-gdpr-sensitive": "email" },
    "profile": {
      "type": "object",
      "properties": {
        "full_name": { "type": "string", "x-gdpr-sensitive": true }
      }
    },
    "addresses": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": { "line": { "type": "string", "x-gdpr-sensitive": true } }
      }
    }
  }
}
```

It is **validation-neutral**: the keyword is *parsed* but **ignored by validation and by
`compat`**, so annotating an existing schema is **never** a breaking change that forces a new
URN. It nests — into nested objects (`profile.full_name`) and array items
(`addresses[].line`) — and the schema model exposes those marked leaves as **sensitive
paths**, the same walk the registry's audit uses.

## The `Cipher` seam

The actual encryption is a **caller-provided** interface bound to your KMS / Vault / HSM /
tokenisation service — so the **core pulls no crypto dependency** (GR-7):

| Method | Meaning |
| :--- | :--- |
| `encrypt(plaintext) → string` | Encrypt a value's bytes to a ciphertext **string** (so `data` stays JSON). |
| `decrypt(ciphertext) → bytes` | The inverse; a wrong-key/tampered input raises a **typed** decrypt error. |

Each SDK also ships a **reference cipher** for callers who hold their own AES-256-GCM key
(random nonce prepended, base64). It does **no key management** — it is the simplest correct
implementation, not a KMS. The reference is built on each platform's in-box crypto:

| SDK | Reference cipher | Built on |
| :--- | :--- | :--- |
| Go | `AESGCMCipher` (`NewAESGCMCipher(key)`) | stdlib `crypto/aes` + `crypto/cipher` |
| PHP | `OpenSslCipher` | `ext-openssl` (a `suggest`, not a `require`) |
| Node | `AesGcmCipher` | built-in `node:crypto` |
| Java | `AesGcmCipher` | JDK `javax.crypto` |
| .NET | `AesGcmCipher` | in-box `System.Security.Cryptography.AesGcm` |
| Python | *(none in core)* | the AES-256-GCM reference is the optional `babelqueue[gdpr]` extra (`cryptography`) |

Python is the one exception: its stdlib has no AES-GCM, so the core ships only the `Cipher`
protocol and the reference cipher rides the optional `[gdpr]` extra — never a core
dependency.

## `protect` / `unprotect`

Two standalone, opt-in helpers that walk the schema's sensitive paths and rewrite **only the
marked values** in `data`, in place:

- **`protect(data, schema, cipher)`** — producer-side, **after** you build `data` and
  **before** you encode. Each marked leaf's value is canonically JSON-encoded, then replaced
  by the cipher's ciphertext string. An absent marked field is **skipped** (not an error).
- **`unprotect(data, schema, cipher)`** — consumer-side, **after** decode and **before**
  handling. The byte-for-byte inverse — a wrong-key decrypt returns a **typed error**, so the
  message takes the retry / [dead-letter](/docs/spec/1.x/dead-letter-and-tracing/) path
  instead of being silently mishandled.

The round-trip is **exact**: because the value is canonically JSON-encoded before encryption,
`unprotect(protect(data)) == data`. Only values inside `data` change — no envelope field is
added, renamed or removed.

### Validate cleartext, not ciphertext

A schema that constrains a sensitive field (e.g. `"format": "email"`) would **reject the
ciphertext string**. So order the steps around the plaintext:

- **Producer:** [validate](/docs/spec/1.x/schema-validation/) → **`protect`** → encode →
  publish.
- **Consumer:** decode → **`unprotect`** → validate → handle.

Validation always sees cleartext; the wire always carries ciphertext.

## Go (the reference)

The `…/gdpr` subpackage of the core module (stdlib only). `Protect`/`Unprotect` are free
functions over the schema's `SensitivePaths()`:

```go
import (
	"github.com/babelqueue/babelqueue-go/gdpr"
	"github.com/babelqueue/babelqueue-go/schema"
)

cipher, _ := gdpr.NewAESGCMCipher(key) // your own 32-byte key; or bind a KMS to gdpr.Cipher
sch, _ := schema.Load("urn:babel:orders:created")

// producer: encrypt the marked leaves in place, before encode
if err := gdpr.Protect(data, sch, cipher); err != nil { /* … */ }

// consumer: the byte-for-byte inverse, after decode
if err := gdpr.Unprotect(data, sch, cipher); err != nil {
	// errors.Is(err, gdpr.ErrDecrypt) → wrong key/tampered → retry / dead-letter
}
```

`Cipher` is the interface (`Encrypt(plaintext []byte) (string, error)` / `Decrypt(ciphertext
string) ([]byte, error)`); bind it to a KMS/Vault and the core stays crypto-free.

## PHP

`BabelQueue\Gdpr\Gdpr::protect()` / `::unprotect()` are static helpers; `OpenSslCipher` is the
reference over `ext-openssl`:

```php
use BabelQueue\Gdpr\Gdpr;
use BabelQueue\Gdpr\OpenSslCipher;

$cipher = new OpenSslCipher($key); // or implement BabelQueue\Gdpr\Cipher over a KMS

// producer: encrypt the marked leaves in place (validate first)
Gdpr::protect($data, $schema, $cipher);

// consumer: the inverse (then validate, then handle)
try {
    Gdpr::unprotect($data, $schema, $cipher);
} catch (\BabelQueue\Gdpr\DecryptException $e) {
    // wrong key / tampered → retry / dead-letter
}
```

`ext-openssl` is a Composer `suggest`, not a `require`, so the core stays `ext-json`-only.

## Per-SDK

GDPR field encryption shipped across all six SDK cores with the same contract — a
caller-bound `Cipher` seam, opt-in `protect`/`unprotect` over the schema's sensitive paths,
a byte-for-byte round-trip, a typed wrong-key error feeding retry/DLQ, and the envelope frame
never touched. Each shipped as a per-SDK MINOR with the envelope frozen.

| SDK | Cipher interface | `protect` / `unprotect` | Sensitive paths |
| :--- | :--- | :--- | :--- |
| Go | `gdpr.Cipher` | `gdpr.Protect` / `gdpr.Unprotect` (free funcs) | `schema.Schema.SensitivePaths()` |
| PHP | `BabelQueue\Gdpr\Cipher` | `Gdpr::protect` / `Gdpr::unprotect` (static) | `BabelQueue\Schema\SensitivePaths::of()` |
| Node | `Cipher` (`@babelqueue/core`) | `protect` / `unprotect` (functions) | `sensitivePaths(schema)` |
| Java | `com.babelqueue.gdpr.Cipher` | `Gdpr.protect` / `Gdpr.unprotect` (static) | `SensitivePaths.of(schema)` |
| .NET | `BabelQueue.Gdpr.ICipher` | `Gdpr.Protect` / `Gdpr.Unprotect` (static) | `SchemaSensitivity.SensitivePaths(schema)` |
| Python | `Cipher` protocol (`babelqueue.gdpr`) | `protect` / `unprotect` (functions) | `sensitive_paths(schema)` |

Idiomatic differences to expect: **.NET** names the interface `ICipher`; **PHP** and **Java**
and **.NET** invoke through a static `Gdpr` class while **Go**, **Node** and **Python** use
free functions; the typed wrong-key error is `ErrDecrypt` (Go), `DecryptException`
(PHP/Java), `DecryptError` (Node/Python), `ProtectedFieldException` (.NET). Only **Python**
ships no concrete cipher in core — the AES-256-GCM reference is the `[gdpr]` extra.

See each SDK's reference for binding a cipher:
[Go](/docs/babelqueue-go/1.x/introduction/), [PHP](/docs/php-sdk/1.x/introduction/),
[Python](/docs/babelqueue-python/1.x/introduction/), [Node](/docs/babelqueue-node/1.x/introduction/),
[Java](/docs/babelqueue-java/1.x/introduction/), [.NET](/docs/babelqueue-dotnet/1.x/introduction/).

For the registry side — declaring the keyword, the `bqschema gdpr` audit gate, masking for
logs, and AsyncAPI carry-through — see [Per-URN schema validation](/docs/spec/1.x/schema-validation/).
Continue to [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

---

# DLQ Redrive & Replay-Bypass

Source: https://babelqueue.com/docs/spec/1.x/redrive-and-replay/

Operator tooling to move dead-lettered messages back onto a queue (ADR-0026) — Redrive resets the dead_letter block and attempts to 0 while preserving job, trace_id, data and meta verbatim, with dry-run, sandbox-routing and select filters — plus the Replay-Bypass guard (ADR-0027) that lets a deliberate replay skip external side-effects that already fired, via the out-of-band bq-replay-bypass transport header. The wire envelope stays frozen at schema_version 1.

A message that exhausts its retries lands on a **dead-letter queue** carrying the optional
[`dead_letter` block](/docs/spec/1.x/envelope/#optional-dead_letter-block). The
[wire contract](/docs/spec/1.x/dead-letter-and-tracing/) defines how a message *gets* there
but deliberately leaves getting it **back** to tooling. This page is that tooling: the
cross-SDK **Redrive** operation (ADR-0026) and the **Replay-Bypass** guard (ADR-0027) that
makes a replay safe to run.

> **Status:** Authoritative · Operator tooling · envelope frozen at `schema_version: 1`

Both are **tooling layers**. Redrive reads and re-publishes the existing frozen envelope;
Replay-Bypass rides an **out-of-band transport header**, never a new envelope field. The
wire stays at `schema_version: 1`.

## Redrive: move messages off the DLQ, reset

`Redrive` drains dead-lettered messages from a DLQ and re-publishes each, **reset for
reprocessing**:

- the `dead_letter` block is **removed** and top-level `attempts` reset to **`0`**;
- `job`, `trace_id`, `data` and `meta` are preserved **verbatim** — a redriven message is
  indistinguishable from a fresh one, and its `trace_id` keeps it on the original trace.

By default each message goes back to its own `dead_letter.original_queue`; `ToQueue`
overrides that to a **sandbox** queue so you can replay safely. Messages are drained from
the DLQ first and then processed, so restored messages are never re-encountered in the
same run; a DLQ message is acked only after its re-publish **succeeds**, and an undecodable
("poison") body is **restored, not lost**.

The options are the safe-replay primitives:

| Option | Effect |
| :--- | :--- |
| `DryRun` | Inspect and report the plan; every message is restored unchanged, nothing is re-published. |
| `Select` | A predicate picking which messages to redrive (e.g. by `reason` or URN); unselected are restored. |
| `ToQueue` | Re-publish to this queue instead of each message's source — route to a sandbox to replay safely. |
| `Max` | Cap how many messages are pulled from the DLQ (`0` = all currently available). |
| `Bypass` | Stamp the replay-bypass header on each redriven message (see below). |

The run returns a result summarizing what happened — counts of redriven vs skipped, and a
per-message item recording its `meta.id`, `trace_id`, URN, `dead_letter.reason`, target
queue, and whether it was actually redriven.

## Replay-Bypass: don't re-fire effects that already ran

Redrive solves *getting a message back*; it does not solve *what its handler does the
second time*. A deliberate replay re-runs the handler — and its external side-effects
re-fire: a second charge, a duplicate email.
[Idempotency](/docs/spec/1.x/idempotency/) stops an **accidental** duplicate (same
`meta.id` delivered twice); **Replay-Bypass** stops an **intended** reprocess from
re-firing effects that already happened. They are complementary.

The mechanism (ADR-0027): when `Redrive` runs with `Bypass`, it stamps the
**`bq-replay-bypass`** header on each redriven message. This is an **out-of-band transport
header**, not an envelope field — so the wire stays frozen, and it only propagates over a
broker whose transport implements the optional header capability. It rides the **same seam**
as the [`traceparent` propagation](/docs/spec/1.x/observability/#w3c-traceparent--true-cross-hop-span-linkage):
the in-memory transport carries it, header-capable bindings carry it, and a binding that does
not yet carry headers makes `Bypass` a no-op there. The runtime surfaces the marker to the
handler, which wraps its **external, non-idempotent** side so a replay re-runs the idempotent
core but skips the effects that already fired:

- `IsReplay(ctx)` — is the message currently being handled a deliberate replay?
- `BypassExternalEffects(ctx, fn)` — run `fn` **unless** this is a replay, in which case
  skip it.

## Go

`Redrive(ctx, transport, dlq, RedriveOptions{...})` and the replay guard
(`HeaderReplayBypass`, `IsReplay`, `BypassExternalEffects`) are in `babelqueue-go`:

```go
import babelqueue "github.com/babelqueue/babelqueue-go"

// dry run: report the plan, change nothing
plan, err := babelqueue.Redrive(ctx, transport, "orders.dlq",
	babelqueue.RedriveOptions{DryRun: true})

// redrive to a sandbox, stamping bq-replay-bypass, only messages that failed on timeout
res, err := babelqueue.Redrive(ctx, transport, "orders.dlq", babelqueue.RedriveOptions{
	ToQueue: "orders.sandbox",
	Bypass:  true,
	Select:  func(env babelqueue.Envelope) bool { return env.DeadLetter != nil && env.DeadLetter.Reason == "failed" },
})
```

The handler guards its external effects so a replay is safe:

```go
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	saveOrder(env) // idempotent core — always runs
	return babelqueue.BypassExternalEffects(ctx, func() error {
		return sendConfirmationEmail(env) // external effect — skipped on replay
	})
})
```

## Python

`babelqueue.redrive.redrive(transport, dlq, ...)` mirrors the Go API; the guard is
`is_replay` / `bypass_external_effects`:

```python
from babelqueue import BabelQueue, is_replay, bypass_external_effects
from babelqueue.redrive import redrive

app = BabelQueue("redis://localhost:6379/0")

# inspect, change nothing
plan = redrive(app.transport, "orders.dlq", dry_run=True)

# redrive to a sandbox, stamping bq-replay-bypass
res = redrive(app.transport, "orders.dlq", to_queue="sandbox", bypass=True)
```

```python
@app.handler("urn:babel:orders:created")
def on_order_created(data, meta):
    save_order(data)                                    # idempotent core — always runs
    bypass_external_effects(lambda: send_email(data))   # external effect — skipped on replay
```

## PHP

PHP's `Transport` is publish-only, so `Redrive::run($io, $dlq, $options)` drives a
`RedriveIO` (reserve / ack / publish) that you bind to your broker. The same reset
contract holds — `dead_letter` removed, `attempts` to `0`, everything else preserved:

```php
use BabelQueue\Redrive\Redrive;
use BabelQueue\Redrive\RedriveOptions;

$result = Redrive::run($io, 'orders.dlq', new RedriveOptions(
    toQueue: 'orders.sandbox', // safe sandbox replay
    dryRun: false,
    bypass: true,              // stamp bq-replay-bypass (needs a HeaderRedriveIO)
    select: fn (array $env) => ($env['dead_letter']['reason'] ?? '') === 'failed',
));
```

`Redrive::reset($envelope)` is exposed on its own for callers that re-publish through their
own machinery.

The replay-bypass guard reads the marker off the delivered message (a `HasHeaders`), so the
handler skips effects that already fired — the PHP mirror of Go's `BypassExternalEffects`:

```php
use BabelQueue\Redrive\ReplayBypass;

$handler = ReplayBypass::wrap(function (\BabelQueue\Contracts\ConsumedMessage $m): void {
    saveOrder($m);                                   // idempotent core — always runs
    ReplayBypass::bypassExternalEffects($m, fn () => sendEmail($m)); // skipped on replay
});
```

## Per-SDK

Redrive ships across the SDKs with the same reset contract and safe-replay options, and the
**Replay-Bypass core guard is now in all six** — the `bq-replay-bypass` marker is identical
across SDKs, so it is cross-SDK. The behaviour is the same everywhere; only the surfacing
differs by language: Go/Python carry the replay flag on the handler's context, Java on a
`ThreadLocal` scope, and Node/.NET/PHP pass the delivered headers explicitly (PHP's core
`Transport` is publish-only, so its guard is a header-reading helper like `Idempotent::wrap`,
not a context-based one).

| SDK | Redrive | Replay-Bypass guard |
| :--- | :--- | :--- |
| Go | `Redrive(ctx, t, dlq, RedriveOptions{…})` | `IsReplay` / `BypassExternalEffects` |
| Python | `redrive(transport, dlq, …)` | `is_replay` / `bypass_external_effects` |
| Java | `Redrive.redrive(t, dlq, opts)` | `Replay.isReplay` / `Replay.bypassExternalEffects` (`Replay.process`) |
| Node | `resetForRedrive` + `RedriveIO.publishWithHeaders` | `isReplay(headers)` / `bypassExternalEffects(headers, fn)` |
| .NET | `Redrive.RedriveAsync(t, dlq, opts)` (`IHeaderPublisher`) | `Replay.IsReplay(headers)` / `Replay.BypassExternalEffectsAsync(headers, fn)` |
| PHP | `Redrive::run($io, $dlq, $options)` (`bypass: true`) | `ReplayBypass::isReplay` / `bypassExternalEffects` / `wrap` |

The one honest limit is the **transport** layer, not the SDK: the marker only propagates over
a broker whose transport carries out-of-band headers (the in-memory transport does;
`HeaderPublisher`-capable bindings carry it, others fall back). Until a given broker binding
wires the header, `Bypass` is a best-effort no-op on it and sandbox `ToQueue` routing remains
the fallback — exactly the per-transport rollout shape the
[`traceparent` propagation](/docs/spec/1.x/observability/#w3c-traceparent--true-cross-hop-span-linkage)
follows, since both ride the same header seam.

For a runnable walkthrough, see the
[DLQ redrive example](/docs/examples/1.x/cross-language-redis/), and for the related
de-duplication primitive, [Idempotency](/docs/spec/1.x/idempotency/).

See each SDK's reference:
[Go](/docs/babelqueue-go/1.x/introduction/), [Python](/docs/babelqueue-python/1.x/introduction/),
[PHP](/docs/php-sdk/1.x/introduction/), [Java](/docs/babelqueue-java/1.x/introduction/),
[Node](/docs/babelqueue-node/1.x/introduction/), [.NET](/docs/babelqueue-dotnet/1.x/introduction/).

---

# Introduction

Source: https://babelqueue.com/docs/php-sdk/1.x/introduction/

Introduction — the framework-agnostic PHP core (babelqueue/php-sdk): the canonical envelope codec, contracts and dead-letter helpers.

`babelqueue/php-sdk` is the framework-agnostic **PHP core**: the wire-envelope
codec, contracts, validation and dead-letter helpers. It is the **single PHP
implementation of the wire format**, so the framework adapters reuse it and can
never drift.

Most PHP applications don't install this directly — they install an adapter, which
pulls the core in:

- **[Laravel](/docs/laravel/1.x/introduction/)** — a drop-in polyglot queue driver.
- **[Symfony](/docs/symfony/1.x/introduction/)** — a Messenger serializer.

Install the core directly only for a **framework-less PHP app**, or to build a new
adapter.

## What you get

- **`EnvelopeCodec`** — build (`make` / `fromJob`), `encode`, `decode`, `urn` and
  `accepts` for the canonical `{job, trace_id, data, meta, attempts}` envelope
  (`schema_version: 1`).
- **Contracts** — `PolyglotJob` (`getBabelUrn()` + `toPayload()`), `HasTraceId`,
  `InboundMessage`, `ConsumedMessage`, and a minimal `Transport` seam.
- **Validation** — `EnvelopeValidator` (consumer-side checks *with a reason*, so you can
  quarantine an unsupported `schema_version` instead of dropping it) and the offline
  `SchemaValidator` (validate any envelope against the bundled canonical JSON Schema).
- **Reference transports** — optional framework-less producers for **all seven brokers**:
  `RedisTransport`, `AmqpTransport`, `SqsTransport`, plus `KafkaTransport` (§6),
  `PulsarTransport` (§5) and `StompTransport` (§7, Artemis).
- **Framework-less consumers** — `KafkaConsumer` (§6 over `ext-rdkafka`, process-then-commit)
  and `PulsarConsumer` (§5 over Pulsar's WebSocket API), plus the §6.4/§6.5 Kafka retry-topic
  machinery (`KafkaRetryRouter` + `KafkaRetryConsumer`).
- **A consume runtime** — `Consume\Dispatcher` (URN → handler routing + the four
  `on_unknown_urn` strategies) and `Consume\DeadLetterPublisher` (route poison messages to
  `<queue>.dlq`).
- **Dead-letter** + **unknown-URN strategies**, and a dependency-free UUIDv4.

**Zero heavy dependencies** — PHP `^8.2` and `ext-json` only; each transport's broker client
is an opt-in suggestion you install only when you use it.

## Core, not a full framework worker

This core is the **contract runtime** plus optional building blocks — the codec, the
transports and the framework-less consume primitives above. It deliberately has no
long-running supervisor, backoff scheduler or process manager: on **Laravel** and **Symfony**
you bind to the framework's native queue (the drop-in driver, Messenger) and reuse that
framework's worker and retry; for a **framework-less** service, the `KafkaConsumer` /
`PulsarConsumer` `consume()` loops + the `Dispatcher` give you a complete consume path on
those brokers.

## The envelope

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

See the full [wire contract](/docs/spec/1.x/envelope/). Continue to
[Installation](/docs/php-sdk/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/php-sdk/1.x/installation/

Installation — the framework-agnostic PHP core babelqueue/php-sdk.

```bash
composer require babelqueue/php-sdk
```

## Requirements

- **PHP** `^8.2`
- **`ext-json`**

That's it — the core has **no heavy runtime dependencies**.

## Using a framework? Install the adapter instead

If you're on Laravel or Symfony, install the adapter — it pulls this core in for
you:

```bash
composer require babelqueue/laravel    # Laravel
composer require babelqueue/symfony    # Symfony (Messenger)
```

## Optional broker clients

The reference [transports](/docs/php-sdk/1.x/transports/) are framework-less and keep
the core dependency-free — install only the client you use:

```bash
composer require predis/predis              # for RedisTransport
composer require php-amqplib/php-amqplib    # for AmqpTransport
```

Next: [Producing messages](/docs/php-sdk/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/php-sdk/1.x/producing-messages/

Producing messages with the PHP core — build the canonical envelope with EnvelopeCodec, encode it, and publish through any transport.

Producing is two steps: **build** the canonical envelope, then **encode** it to JSON
and publish the bytes with your broker client.

## From a typed job

Implement `BabelQueue\Contracts\PolyglotJob` — a URN plus a pure-array payload:

```php
use BabelQueue\Contracts\PolyglotJob;

final class OrderCreated implements PolyglotJob
{
    public function __construct(public int $orderId) {}

    public function getBabelUrn(): string
    {
        return 'urn:babel:orders:created';
    }

    public function toPayload(): array
    {
        return ['order_id' => $this->orderId];
    }
}
```

```php
use BabelQueue\Codec\EnvelopeCodec;

$envelope = EnvelopeCodec::fromJob(new OrderCreated(1042), 'orders');
$json     = EnvelopeCodec::encode($envelope);
// publish $json onto the "orders" queue with your transport (below)
```

## From a raw URN + data

When you don't have a job object, build it directly:

```php
$envelope = EnvelopeCodec::make('urn:babel:orders:created', ['order_id' => 1042], 'orders');
$json     = EnvelopeCodec::encode($envelope);
```

`make(string $urn, array $data = [], string $queue = 'default', ?string $traceId = null)`
returns the envelope as a plain PHP array. A fresh `trace_id` (v4 UUID) is stamped
automatically; pass `$traceId` to continue an existing trace, or implement
`BabelQueue\Contracts\HasTraceId` on your job.

## Publish

`encode()` produces the exact UTF-8 JSON bytes every other SDK reads — see
[Transports](/docs/php-sdk/1.x/transports/) for the reference Redis/RabbitMQ
publishers, or move the string with any broker client you already use.

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "…",
  "data": { "order_id": 1042 },
  "meta": { "id": "…", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/php-sdk/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/php-sdk/1.x/consuming-messages/

Consuming with the PHP core — decode the envelope, validate it (quarantine unsupported versions), route by URN, and (for Kafka/Pulsar) drive the framework-less consumers and consume runtime.

The core gives you decode + validation + URN routing, and — for the brokers PHP can
consume framework-less (Kafka and Pulsar) — ready-made `consume()` loops plus a
[consume runtime](#the-consume-runtime). The loop that pulls bytes off a broker without a
core consumer is yours (or your framework adapter's).

## Decode

```php
use BabelQueue\Codec\EnvelopeCodec;

$envelope = EnvelopeCodec::decode($rawBody); // plain PHP array
```

## Validate before you dispatch

`EnvelopeValidator::check()` returns a **reason** (or `null` if valid), so you can
quarantine a message you don't understand instead of silently dropping it:

```php
use BabelQueue\Validation\EnvelopeValidator;

if ($reason = EnvelopeValidator::check($envelope)) {
    // e.g. EnvelopeValidator::REASON_UNSUPPORTED_SCHEMA_VERSION
    // → dead-letter / quarantine, don't drop.
    return;
}
```

Reasons include `REASON_MISSING_URN`, `REASON_MISSING_META`,
`REASON_UNSUPPORTED_SCHEMA_VERSION`, `REASON_INVALID_DATA`,
`REASON_MISSING_TRACE_ID`, `REASON_INVALID_ATTEMPTS`. There is also
`EnvelopeValidator::isValid($envelope): bool` and `validate($envelope): void` (which
throws `InvalidEnvelopeException` carrying the reason + envelope).

> `EnvelopeCodec::accepts($envelope)` is a quick boolean check for the same
> consumer-side rules when you don't need the reason.

For a **full structural check** against the bundled canonical JSON Schema — every field,
type and constraint, not just the consumer-side rules — use the offline `SchemaValidator`:

```php
use BabelQueue\Validation\SchemaValidator;

SchemaValidator::isValid($envelope);          // bool
$reason = SchemaValidator::check($envelope);  // "<json-pointer>: <reason>" | null
SchemaValidator::validate($envelope);         // throws on the first violation
```

It's offline and dependency-free (the schema ships in the package), so it's also handy in
tests and CI to assert an envelope you produce is conformant.

## Route by URN

```php
$urn = EnvelopeCodec::urn($envelope); // reads `job`, accepting `urn` as an alias

match ($urn) {
    'urn:babel:orders:created' => $handler->handle($envelope['data'], $envelope['meta']),
    default                    => /* apply your unknown-URN strategy */ null,
};
```

For unmapped URNs, the `BabelQueue\Routing\UnknownUrnStrategy` constants
(`FAIL` / `DELETE` / `RELEASE` / `DEAD_LETTER`) name the standard choices; see
[error handling in the wire contract](/docs/spec/1.x/envelope/#consumer-rules).

## Framework-less consumers (Kafka & Pulsar)

For Redis, RabbitMQ, SQS and Artemis, the broker loop is your framework worker's (the
Laravel drop-in drivers, Symfony Messenger) — the core stays a codec. For **Kafka** and
**Pulsar**, the core ships **complete framework-less consumers** with `receive` / ack /
release primitives and a `consume()` loop:

- **`KafkaConsumer`** (§6, over `ext-rdkafka`) — **process-then-commit** (at-least-once):
  `receive()` polls and decodes a record with `attempts` reconciled (the `bq-attempts`
  header wins, else the body), `commit()` advances the offset, and `consume($handler,
  $shouldStop)` runs the loop, committing only on a clean return.
- **`PulsarConsumer`** (§5, over Pulsar's WebSocket API) — `receive()`, `acknowledge()`
  (the §5 "delete"), `release()` (`negativeAcknowledge`, redeliver), and a `consume()`
  loop. `attempts` is reconciled to `max(body.attempts, redeliveryCount)`.

Both decouple from their broker client behind a one-method seam (`KafkaConsumerClient` /
`PulsarWebSocketConsumerClient`), so the consumer is dependency-free and testable:

```php
use BabelQueue\Transport\PulsarConsumer;

$consumer = new PulsarConsumer($pulsarWebSocketConsumerClient);

$consumer->consume(function ($message) {
    // $message->getUrn(), $message->getData(), $message->attempts()
    // return → acknowledge; throw → release (redeliver)
}, fn () => $shouldStop);
```

## The consume runtime

`consume()` takes any callable. Pass a `Consume\Dispatcher` and you get URN → handler
routing, the four `on_unknown_urn` strategies, and an optional max-attempts dead-letter
cap — composed from pieces the core already ships:

```php
use BabelQueue\Consume\Dispatcher;
use BabelQueue\Consume\DeadLetterPublisher;
use BabelQueue\Contracts\ConsumedMessage;
use BabelQueue\Routing\UnknownUrnStrategy;

$dispatch = (new Dispatcher(
    onUnknownUrn: UnknownUrnStrategy::DEAD_LETTER,
    maxAttempts: 5,
    deadLetters: new DeadLetterPublisher($pulsarProducer), // routes poison to <queue>.dlq
))->on('urn:babel:orders:created', fn (ConsumedMessage $m) => handle($m->getData(), $m->getTraceId()));

$consumer->consume($dispatch, fn () => $shouldStop);
```

A handler that returns acks the message; one that throws redelivers it (at-least-once).
On an unknown URN the strategy applies — `delete` drops it, `dead_letter` routes it to
`<queue>.dlq` (degrading to `delete` when no publisher is set), and `fail` / `release`
throw to redeliver.

### Kafka retry topics

Kafka has no native per-message retry or delay, so the core implements §6.4/§6.5 with the
tiered **retry-topic** pattern: `KafkaRetryRouter::route()` sends a failed record to
`<topic>.retry.<n>` (with `bq-attempts + 1`) or, past the cap, to `<topic>.dlq`; a
`KafkaRetryConsumer` then waits the tier delay and re-injects it into the work topic. See
the [Kafka binding](/docs/spec/1.x/broker-bindings/#apache-kafka).

## Cross-language

Because you decode the [canonical envelope](/docs/spec/1.x/envelope/), the message may
have been produced by **any** BabelQueue SDK — a Go service, a Python worker, a
Node app — and the URN is the only shared contract. See the
[cross-language example](/docs/examples/1.x/cross-language-redis/).

---

# Transports

Source: https://babelqueue.com/docs/php-sdk/1.x/transports/

Optional framework-less reference transports for the PHP core — RedisTransport, AmqpTransport, SqsTransport, KafkaTransport, PulsarTransport and StompTransport (Artemis) — plus the one-method Transport seam.

For a **framework-less** PHP producer, the core ships optional reference `Transport`
implementations covering **all seven brokers** — Redis, RabbitMQ, Amazon SQS, Apache
Kafka, Apache Pulsar and Apache ActiveMQ/Artemis. (Azure Service Bus is the one broker
without a PHP transport — [ADR-0021](/docs/spec/1.x/broker-bindings/#azure-service-bus).)
They keep the core dependency-free — install only the broker client you use.

## The Transport seam

`BabelQueue\Contracts\Transport` is a single method:

```php
interface Transport
{
    public function publish(string $payload, ?string $queue = null): ?string;
}
```

`$payload` is the `EnvelopeCodec::encode(...)` output. Any broker client can
implement it — phpredis (`ext-redis`) users, for example, can do it in one line
(`rpush`).

## RedisTransport

```bash
composer require predis/predis
```

```php
use BabelQueue\Codec\EnvelopeCodec;
use BabelQueue\Transport\RedisTransport;

$transport = new RedisTransport(new Predis\Client('redis://localhost:6379'));

$envelope = EnvelopeCodec::fromJob($job, 'orders');
$transport->publish(EnvelopeCodec::encode($envelope), 'orders');
// a Go / Python / Node consumer reads the identical envelope off "orders"
```

`RedisTransport` produces with `RPUSH`, matching the reliable-queue pattern the other
SDKs use.

## AmqpTransport

```bash
composer require php-amqplib/php-amqplib
```

```php
use BabelQueue\Transport\AmqpTransport;

$transport = new AmqpTransport(/* php-amqplib connection */);
$transport->publish(EnvelopeCodec::encode($envelope), 'orders');
```

`AmqpTransport` declares a durable queue, publishes persistent messages, and sets the
contract AMQP properties (`type` = URN, `correlation_id` = `trace_id`,
`message_id` = `meta.id`, plus `x-schema-version` / `x-source-lang` / `x-attempts`),
so consumers can route on properties without parsing the body.

## SqsTransport

```bash
composer require aws/aws-sdk-php
```

A produce-side Amazon SQS transport: it sends the canonical envelope as the `MessageBody`
with the [§3 `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs). It is
decoupled from the AWS SDK behind a one-method `BabelQueue\Transport\SqsClient` seam, so
wrap a real `Aws\Sqs\SqsClient` in a one-line adapter:

```php
use BabelQueue\Transport\SqsClient;
use BabelQueue\Transport\SqsTransport;

$adapter = new class (new Aws\Sqs\SqsClient([/* region, credentials */])) implements SqsClient {
    public function __construct(private Aws\Sqs\SqsClient $c) {}
    public function sendMessage(array $args): mixed { return $this->c->sendMessage($args); }
};

$transport = new SqsTransport($adapter, 'https://sqs.eu-central-1.amazonaws.com/123456789012/orders');
$transport->publish(EnvelopeCodec::encode($envelope));
// a Go / Python / Node / Java / .NET consumer reads the identical envelope + attributes
```

The projected attributes (`bq-job` = URN, `bq-trace-id`, `bq-message-id`, plus
`bq-schema-version` / `bq-source-lang` / `bq-created-at`) let a consumer route without
decoding the body; FIFO sets `MessageGroupId` / `MessageDeduplicationId`.

## KafkaTransport

Apache Kafka (§6) is PHP's one **opt-in** transport: its only viable client is the
`ext-rdkafka` C extension, so it deliberately relaxes the zero-extension rule
([ADR-0019](/docs/spec/1.x/broker-bindings/#apache-kafka)). The transport stays decoupled
behind the one-method `BabelQueue\Transport\KafkaProducer` seam, so you wrap your real
`RdKafka\Producer` in a one-line adapter:

```bash
pecl install rdkafka   # the ext-rdkafka extension
```

```php
use BabelQueue\Transport\KafkaProducer;
use BabelQueue\Transport\KafkaTransport;

$producer = new class (/* RdKafka\Producer */) implements KafkaProducer {
    public function produce(string $topic, string $payload, array $headers, ?int $timestampMs = null): void {
        // produce to $topic with the bq-* headers + record timestamp
    }
};

$transport = new KafkaTransport($producer, 'orders');
$transport->publish(EnvelopeCodec::encode($envelope), 'orders');
// a Java / Go / Node / Python / .NET consumer reads the identical envelope off the topic
```

The record **value** is the canonical envelope; the contract fields ride `bq-` **record
headers** (`bq-job` = URN, `bq-trace-id`, `bq-message-id`, `bq-schema-version`,
`bq-source-lang`, `bq-attempts` — hyphens, per §6), and the record timestamp mirrors
`meta.created_at`.

## PulsarTransport

Apache Pulsar (§5) has no mature native PHP client, so PHP produces over Pulsar's native
**WebSocket API** with a pure-PHP WebSocket client — **GR-7 stays intact** (no C extension,
[ADR-0020](/docs/spec/1.x/broker-bindings/#apache-pulsar)). It is decoupled behind the
one-method `BabelQueue\Transport\PulsarWebSocketClient` seam:

```bash
composer require textalk/websocket
```

```php
use BabelQueue\Transport\PulsarTransport;
use BabelQueue\Transport\PulsarWebSocketClient;

$client = new class (/* textalk/websocket client */) implements PulsarWebSocketClient {
    public function publish(string $topic, string $payload, array $properties): void {
        // base64 the payload into a WS producer frame with the bq-* properties; check the ack
    }
};

$transport = new PulsarTransport($client, 'orders'); // tenant/namespace default to public/default
$transport->publish(EnvelopeCodec::encode($envelope), 'orders');
// the BabelQueue "queue" maps to persistent://public/default/orders
```

The message **value** is the canonical envelope; `bq-` native **properties** (string→string,
hyphens) carry the contract fields. The body's `meta.created_at` stays authoritative — the
adapter must not set the frame's `eventTime`.

## StompTransport (Artemis)

Apache ActiveMQ **Artemis** (§7) speaks AMQP 1.0, which has no strong PHP client, so PHP
reaches it over **STOMP** (the mature pure-PHP `stomp-php` client; Artemis ships a STOMP
acceptor) — **GR-7 intact** ([ADR-0018](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis)).
Artemis bridges STOMP ↔ AMQP 1.0 ↔ JMS on the same address, so a STOMP-produced message is
consumed natively by the Java (JMS) and .NET / Node / Python / Go (AMQP 1.0) SDKs:

```bash
composer require stomp-php/stomp-php
```

```php
use BabelQueue\Transport\StompClient;
use BabelQueue\Transport\StompTransport;

$client = new class (/* Stomp\Client */) implements StompClient {
    public function send(string $destination, string $body, array $headers): void {
        // SEND the envelope JSON to the anycast address with the bq_ headers
    }
};

$transport = new StompTransport($client, 'orders');
$transport->publish(EnvelopeCodec::encode($envelope), 'orders');
```

Routing is **body-authoritative** (a STOMP header can't set the `x-opt-jms-type`
annotation, so consumers fall back to the body's `job` URN, which is always present). The
§7 fields ride `correlation-id` (= `trace_id`) and the **`bq_`** application properties —
note the **underscores** (JMS forbids hyphens in property names —
[ADR-0017](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis)).

> Consuming with the core. These transports are **producers**. PHP consumes Kafka and
> Pulsar with the framework-less [`KafkaConsumer` / `PulsarConsumer`](/docs/php-sdk/1.x/consuming-messages/),
> and Artemis with the Laravel [`babelqueue-artemis`](/docs/laravel/1.x/apache-activemq-artemis/)
> drop-in driver.

> On Laravel or Symfony you don't need these — the [Laravel driver](/docs/laravel/1.x/introduction/)
> and the [Symfony serializer](/docs/symfony/1.x/introduction/) move the bytes through
> the framework's own transport.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/php-sdk/1.x/reliability-and-governance/

The optional, dependency-free reliability and governance helpers in the PHP core — idempotency (seen-set + persistent PdoStore/RedisStore with the claim contract), the transactional outbox, DLQ redrive + replay-bypass, GDPR field encryption, and OpenTelemetry traceparent. Each composes with the core codec, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, the PHP core ships a set of **optional, opt-in helpers** that
mechanise the reliability and governance contracts the wire spec defines. None of them
touch the frozen envelope (`schema_version: 1`); each is a tooling layer over what the
codec already carries. They are the PHP face of the cross-SDK spec — follow the linked spec
page for the full contract.

## Idempotency

`BabelQueue\Idempotency\Idempotent::wrap($store, $handler)` makes a handler **run at most
once per `meta.id`**, even on at-least-once redelivery — skip if already seen, run then
remember on success, fail-open on a missing id.

```php
use BabelQueue\Idempotency\Idempotent;
use BabelQueue\Idempotency\InMemoryStore;

$store = new InMemoryStore(); // tests / single process
$dispatch->on('urn:babel:orders:created', Idempotent::wrap($store,
    fn (\BabelQueue\Contracts\ConsumedMessage $m) => chargeOnce($m)));
```

For a **fleet**, swap the in-memory store for a shipped persistent one — both implement the
`ClaimingStore` interface (which extends `IdempotencyStore` with an atomic `claim` /
`release` so two workers can't both run an in-flight id):

```php
use BabelQueue\Idempotency\PdoStore;
use BabelQueue\Idempotency\RedisStore;

$pdo->exec(PdoStore::ddl());          // portable CREATE TABLE (Postgres/MySQL/SQLite)
$store = new PdoStore($pdo);          // or: new RedisStore($predisClient)
```

See [Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

`BabelQueue\Outbox` removes the producer **dual write**: persist the encoded envelope in the
**same DB transaction** as your business row, then a relay publishes the durable rows.

```php
use BabelQueue\Outbox\Outbox;
use BabelQueue\Outbox\OutboxRelay;
use BabelQueue\Outbox\InMemoryOutboxStore;

$store  = new InMemoryOutboxStore();   // production: a PDO/ORM-backed OutboxStore
$outbox = new Outbox($store);

// inside YOUR $db->transaction(...), beside the business write — no commit of its own:
$id = $outbox->write($envelope);       // encodes via EnvelopeCodec, calls $store->save()

// later, a separate process drains the durable rows to the broker:
$result = (new OutboxRelay($transport, $store))->drain(); // ->published / ->failed
```

`OutboxStore` is the four-method contract (`save` / `fetchUnpublished` / `markPublished` /
`markFailed`); the transaction boundary is **yours**. The relay publishes the stored bytes
**verbatim**, so `trace_id` is preserved. See [Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`BabelQueue\Redrive\Redrive::run($io, $dlq, $options)` moves dead-lettered messages back onto
a queue — `dead_letter` removed, `attempts` reset to `0`, everything else preserved — with
`dryRun`, `select`, `toQueue` (sandbox) and `bypass` options. PHP's `Transport` is
publish-only, so you bind a `RedriveIO` (reserve / ack / publish) to your broker.

```php
use BabelQueue\Redrive\Redrive;
use BabelQueue\Redrive\RedriveOptions;
use BabelQueue\Redrive\ReplayBypass;

$result = Redrive::run($io, 'orders.dlq', new RedriveOptions(
    toQueue: 'orders.sandbox',
    bypass: true, // stamp bq-replay-bypass (needs a HeaderRedriveIO)
));

// the handler skips effects that already fired on a deliberate replay:
$handler = ReplayBypass::wrap(function (\BabelQueue\Contracts\ConsumedMessage $m): void {
    saveOrder($m);                                          // idempotent core
    ReplayBypass::bypassExternalEffects($m, fn () => sendEmail($m)); // skipped on replay
});
```

See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`BabelQueue\Gdpr\Gdpr::protect()` / `::unprotect()` encrypt only the `data` leaves a schema
marks `x-gdpr-sensitive`, in place, leaving `data` pure JSON and the envelope frozen. The
`Cipher` is caller-bound (so the core pulls no crypto dependency); `OpenSslCipher` is the
reference over `ext-openssl` (a `suggest`, not a `require`). **Validate cleartext** — protect
after validation on produce, unprotect before validation on consume.

```php
use BabelQueue\Gdpr\Gdpr;
use BabelQueue\Gdpr\OpenSslCipher;

$cipher = new OpenSslCipher($key); // or implement BabelQueue\Gdpr\Cipher over a KMS

Gdpr::protect($data, $schema, $cipher);   // producer: encrypt marked leaves
// ... encode + publish ... decode ...
try {
    Gdpr::unprotect($data, $schema, $cipher); // consumer: inverse
} catch (\BabelQueue\Gdpr\DecryptException $e) {
    // wrong key / tampered → retry / dead-letter
}
```

See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

`BabelQueue\Otel\Tracing` emits `publish <urn>` / `process <urn>` spans and, on a
header-carrying transport, injects/extracts the W3C `traceparent` so a consumer span is a
true child of the producer span — degrading to v0.1 `trace_id` correlation otherwise. The
in-repo Redis/AMQP/SQS transports carry the header; Kafka/Pulsar/STOMP producer wiring is a
documented follow-up. `open-telemetry/api` is an optional `suggest`, so the core stays
`ext-json`. See [Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`BabelQueue\Schema\SchemaValidated::assert()` (producer guard) / `::wrap()` (consumer safety
net) validate a message's `data` against the JSON Schema registered for its URN, bridged from
a `babelqueue-registry` manifest via `DirProvider`. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/laravel/1.x/introduction/

Introduction — PHP / Laravel BabelQueue driver (1.x).

The **PHP / Laravel** SDK is the canonical producer for BabelQueue. It lets your
Laravel application publish jobs as a strict, language-agnostic JSON envelope —
so Go, Python, Java, .NET and Node services consume them natively, without ever
touching PHP `serialize()`.

## What you get

- **Canonical wire envelope (`schema_version: 1`).** Every published message is a
  strict JSON document with a stable shape: `job`, `trace_id`, `data`, `meta` and
  `attempts`.
- **A `trace_id` on every message.** A required correlation UUID is stamped on
  publish and flows across every service that touches the job — no manual
  plumbing.
- **URN routing.** Jobs are identified by a URN (`urn:babel:<context>:<event>`),
  not a PHP class name, so consumers in any language can route on a stable key.
- **Drop-in broker drivers.** Choose your transport with one Laravel queue connection
  line — `babelqueue-redis`, `babelqueue-rabbitmq`, `babelqueue-sqs` or
  `babelqueue-artemis` (Apache ActiveMQ/Artemis over STOMP). Each extends the matching
  native driver, so standard jobs on the same connection pass straight through.
- **Dead-letter queue (DLQ).** Unknown URNs and exhausted retries can be routed
  to a dead-letter destination instead of being silently dropped.

## The envelope

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "default", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Continue to [Installation](/docs/laravel/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/laravel/1.x/installation/

Installation — PHP / Laravel BabelQueue driver (1.x).

Install the PHP / Laravel driver via Composer:

```bash
composer require babelqueue/laravel
```

## Requirements

- **PHP** `^8.2`
- **Laravel** 11 or 12
- A running broker for your driver — **Redis**, **RabbitMQ**, **Amazon SQS**
  (`babelqueue-sqs`, needs `aws/aws-sdk-php`) or **Apache ActiveMQ/Artemis**
  (`babelqueue-artemis`, needs `stomp-php/stomp-php`)

## Publish the configuration

Publish the package config to `config/babelqueue.php`:

```bash
php artisan vendor:publish --tag=babelqueue-config
```

This gives you the URN → handler map, the `on_unknown_urn` policy and the
dead-letter settings you'll wire up in [Configuration](/docs/laravel/1.x/configuration/).

Next: [Configuration](/docs/laravel/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/laravel/1.x/configuration/

Configuration — PHP / Laravel BabelQueue driver (1.x).

BabelQueue plugs into Laravel's queue system as a custom connection driver, then
adds its own URN routing and dead-letter config.

## Queue connection

Add a BabelQueue connection in `config/queue.php`. Use `babelqueue-redis` for the
Redis transport:

```php
// config/queue.php
'connections' => [

    'babelqueue' => [
        'driver'     => 'babelqueue-redis',
        'connection' => 'default',
        'queue'      => env('BABELQUEUE_QUEUE', 'default'),
        'retry_after' => 90,
    ],

],
```

Or `babelqueue-rabbitmq` for the RabbitMQ transport:

```php
// config/queue.php
'connections' => [

    'babelqueue' => [
        'driver' => 'babelqueue-rabbitmq',
        'host'   => env('RABBITMQ_HOST', '127.0.0.1'),
        'port'   => env('RABBITMQ_PORT', 5672),
        'queue'  => env('BABELQUEUE_QUEUE', 'default'),
    ],

],
```

## Package configuration

`config/babelqueue.php` maps incoming URNs to handler classes and controls what
happens to messages with an unknown URN:

```php
// config/babelqueue.php
return [

    // Map each URN to the handler that consumes it.
    'handlers' => [
        'urn:babel:orders:created' => App\Babel\Handlers\OrderCreatedHandler::class,
    ],

    // What to do when an incoming URN has no registered handler:
    // 'fail' | 'delete' | 'release' | 'dead_letter'
    'on_unknown_urn' => 'dead_letter',

    // Where exhausted / unroutable messages go. Failures from "orders"
    // are republished to "orders.dlq" as the same envelope plus an
    // additive `dead_letter` block, so any SDK can triage them.
    'dead_letter' => [
        'enabled' => true,
        'suffix'  => '.dlq',
    ],

];
```

Next: [Producing messages](/docs/laravel/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/laravel/1.x/producing-messages/

Producing messages — PHP / Laravel BabelQueue driver (1.x).

There are two ways to publish a message. Both emit the identical canonical
envelope on the wire.

## The interface (primary)

Implement `ShouldQueuePolyglot` on a job class. `getBabelUrn()` returns the URN
and `toPayload()` returns the `data` body. Dispatch it with Laravel's `dispatch()`
helper:

```php
use BabelQueue\Contracts\ShouldQueuePolyglot;

final class OrderCreated implements ShouldQueuePolyglot
{
    public function __construct(
        private int $orderId,
        private float $amount,
    ) {}

    public function getBabelUrn(): string
    {
        return 'urn:babel:orders:created';
    }

    public function toPayload(): array
    {
        return [
            'order_id' => $this->orderId,
            'amount'   => $this->amount,
        ];
    }
}

dispatch(new OrderCreated(1042, 99.90));
```

## The facade (sugar)

For one-off publishes, the `BabelQueue` facade takes the URN and payload directly
and returns the message id:

```php
use BabelQueue\Facades\BabelQueue;

$id = BabelQueue::publish('urn:babel:orders:created', [
    'order_id' => 1042,
    'amount'   => 99.90,
]);
```

## The emitted envelope

Either approach produces the canonical `schema_version: 1` envelope:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "default", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/laravel/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/laravel/1.x/consuming-messages/

Consuming messages — PHP / Laravel BabelQueue driver (1.x).

Consumers route on the URN. Map each URN to a handler class in
`config/babelqueue.php`, then process the worker as usual.

## Register the handler

```php
// config/babelqueue.php
return [
    'handlers' => [
        'urn:babel:orders:created' => App\Babel\Handlers\OrderCreatedHandler::class,
    ],
    'on_unknown_urn' => 'dead_letter',
];
```

## Write the handler

A handler receives the decoded `data`, the `meta` block and the message's
`trace_id` for cross-service correlation. Implement the optional `failed()` hook
to react when processing throws:

```php
namespace App\Babel\Handlers;

final class OrderCreatedHandler
{
    public function handle(array $data, array $meta, string $traceId): void
    {
        logger()->withContext(['trace_id' => $traceId])
            ->info('Order received', $data);

        // ... your business logic
    }

    public function failed(array $data, ?\Throwable $e): void
    {
        report($e);
    }
}
```

## Run the worker

BabelQueue is a drop-in Laravel queue driver, so you consume with the standard
worker — just point it at your polyglot connection:

```bash
php artisan queue:work babelqueue
```

The `$traceId` ties this consumer's logs back to the original producer and to
every other service that handled the same job — no matter which language emitted
it. Exhausted retries and unknown URNs follow your `on_unknown_urn` /
`dead_letter` policy from [Configuration](/docs/laravel/1.x/configuration/).

That's the whole loop: any producer, any consumer, one schema.

---

# Amazon SQS driver

Source: https://babelqueue.com/docs/laravel/1.x/amazon-sqs/

The babelqueue-sqs drop-in driver — a polyglot Amazon SQS queue on Laravel's native sqs connection. Polyglot jobs are sent as the canonical envelope with the §3 MessageAttributes; standard Laravel jobs pass straight through.

`babelqueue-sqs` is a **drop-in queue connection** that makes an Amazon SQS queue
polyglot. It extends Laravel's native `sqs` driver: jobs that opt into BabelQueue are
sent as the [canonical envelope](/docs/spec/1.x/envelope/) with the
[§3 SQS `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs), so a Go, Python,
Node, Java or .NET worker can route and trace them — while **standard Laravel jobs on the
same connection pass straight through unchanged**.

## Requirements

The AWS SDK for PHP is required (Laravel lists it as a suggestion):

```bash
composer require aws/aws-sdk-php
```

## Configuration

Add a connection in `config/queue.php` with `driver` set to `babelqueue-sqs`. The
config keys are exactly Laravel's stock `sqs` keys:

```php
'connections' => [
    'babelqueue-sqs' => [
        'driver' => 'babelqueue-sqs',
        'key'    => env('AWS_ACCESS_KEY_ID'),
        'secret' => env('AWS_SECRET_ACCESS_KEY'),
        'prefix' => env('SQS_PREFIX', 'https://sqs.eu-central-1.amazonaws.com/your-account-id'),
        'queue'  => env('SQS_QUEUE', 'orders'),
        'suffix' => env('SQS_SUFFIX', ''),
        'region' => env('AWS_DEFAULT_REGION', 'eu-central-1'),
    ],
],
```

The connector mirrors the stock `SqsConnector` (credentials, prefix/suffix), so anything
that works for `sqs` works here. For LocalStack/ElasticMQ, set an `endpoint` key as you
would for the native driver.

## Produce

Producing is identical to the other BabelQueue drivers — see
[Producing messages](/docs/laravel/1.x/producing-messages/). Dispatch a typed
`ShouldQueuePolyglot` job (or use the `BabelQueue` facade) onto the connection:

```php
use BabelQueue\Facades\BabelQueue;

BabelQueue::connection('babelqueue-sqs')
    ->publish('urn:babel:orders:created', ['order_id' => 1042]);
```

The job is sent as the canonical envelope (`MessageBody`) with the projected
`MessageAttributes` (`bq-job`/`bq-trace-id`/`bq-message-id`/`bq-schema-version`/
`bq-source-lang`/`bq-created-at`). A plain Laravel job dispatched to the same connection
is encoded the normal Laravel way — no envelope, no attributes — so existing workers keep
working.

## Consume

Run a worker against the connection as usual:

```bash
php artisan queue:work babelqueue-sqs
```

Polyglot messages — including those produced by **another language's SDK** — are decoded,
routed by URN through the dispatcher and handled; standard jobs run as normal. See
[Consuming messages](/docs/laravel/1.x/consuming-messages/) for handler registration.

> **`attempts` note.** As a Laravel driver, the reserved job surfaces the broker's
> **native** receive count via `SqsJob::attempts()` (i.e. `ApproximateReceiveCount`).
> This is the documented, exempt divergence from the framework-less SDKs, which reconcile
> `attempts` to `ApproximateReceiveCount − 1` — see the
> [SQS binding](/docs/spec/1.x/broker-bindings/#amazon-sqs).

Because the wire format is the canonical envelope, a message your Laravel app consumes
may have been produced by any BabelQueue SDK, and vice-versa.

---

# Apache ActiveMQ Artemis driver

Source: https://babelqueue.com/docs/laravel/1.x/apache-activemq-artemis/

The babelqueue-artemis drop-in driver — a polyglot Apache ActiveMQ Artemis queue on Laravel over STOMP. Produces and consumes the canonical envelope; Artemis bridges STOMP ↔ AMQP 1.0 ↔ JMS, so Java/.NET/Node/Python/Go SDKs interop natively.

`babelqueue-artemis` is a **drop-in queue connection** that makes an Apache ActiveMQ
**Artemis** broker polyglot from Laravel. Artemis speaks AMQP 1.0 (not RabbitMQ's
0-9-1) and has no strong native PHP client, so this driver reaches it over **STOMP**
(the §7 PHP path — [ADR-0018](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis)).
Artemis bridges **STOMP ↔ AMQP 1.0 ↔ JMS** on the same address, so a message this driver
produces is consumed natively by the Java (JMS) and .NET / Node / Python / Go (AMQP 1.0)
Artemis SDKs — and vice versa.

Unlike the produce-only framework-less [`StompTransport`](/docs/php-sdk/1.x/transports/#stomptransport-artemis),
this Laravel driver **produces and consumes**: it's a full queue connection, so the
standard worker pops, routes by URN and acks/releases.

## Requirements

The pure-PHP STOMP client is required:

```bash
composer require stomp-php/stomp-php
```

## Configuration

Add a connection in `config/queue.php` with `driver` set to `babelqueue-artemis`:

```php
'connections' => [
    'babelqueue-artemis' => [
        'driver'   => 'babelqueue-artemis',
        'host'     => env('ARTEMIS_HOST', '127.0.0.1'),
        'port'     => env('ARTEMIS_PORT', 61613),   // Artemis STOMP acceptor
        'username' => env('ARTEMIS_USER', 'artemis'),
        'password' => env('ARTEMIS_PASSWORD', 'artemis'),
        'queue'    => env('BABELQUEUE_QUEUE', 'orders'),
        'options'  => ['ssl' => false],             // 'ssl' => true uses stomp+ssl
    ],
],
```

The connector hands the queue a **lazy factory** (no eager socket), so it builds and
connects the STOMP client on first use and reconnects transparently after a drop —
mirroring the RabbitMQ connector.

## Produce

Producing is identical to the other BabelQueue drivers — see
[Producing messages](/docs/laravel/1.x/producing-messages/). Send to the **anycast**
address (the default for an auto-created JMS queue) so the JMS/AMQP-1.0 consumers receive
it:

```php
use BabelQueue\Facades\BabelQueue;

BabelQueue::connection('babelqueue-artemis')
    ->publish('urn:babel:orders:created', ['order_id' => 1042]);
```

The envelope JSON is the STOMP frame **body** (authoritative); `trace_id` rides the
native `correlation-id` header, and the §7 `bq_` application properties
(`bq_schema_version` / `bq_source_lang` / `bq_attempts` / `bq_app_id`) use **underscores**
because a JMS property name must be a valid Java identifier
([ADR-0017](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis)).

## Consume

Run the standard worker against the connection:

```bash
php artisan queue:work babelqueue-artemis
```

Polyglot messages — including those produced by **another language's SDK** over JMS or
AMQP 1.0 — are decoded, routed by URN through the dispatcher and handled; standard Laravel
jobs on the same connection run as normal. Register handlers exactly as for any other
driver — see [Consuming messages](/docs/laravel/1.x/consuming-messages/).

> **Routing is body-authoritative** (§7.8). A STOMP custom header can't set the
> `x-opt-jms-type` annotation the §7 consumers prefer, so routing falls back to the
> envelope's `job` URN — which is always present, so a STOMP-produced message routes
> correctly against every Artemis SDK.

Because the wire format is the [canonical envelope](/docs/spec/1.x/envelope/), a message
your Laravel app consumes may have been produced by any BabelQueue SDK, and vice-versa.
See the [Artemis binding](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis) for the
full native projection.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/laravel/1.x/reliability-and-governance/

Using the PHP core's reliability and governance helpers from the Laravel driver — idempotency (seen-set + persistent PdoStore/RedisStore), the transactional outbox, DLQ redrive + replay-bypass, GDPR field encryption, and OpenTelemetry traceparent. The standard worker stays untouched; the helpers compose with your handlers and add nothing to the frozen envelope.

The Laravel driver is a thin adapter on the framework-agnostic PHP core, so the core's
**optional reliability and governance helpers** are available to a Laravel app directly —
they live in `babelqueue/php-sdk`, which the driver already pulls in. None touch the frozen
envelope (`schema_version: 1`); each is a tooling layer that composes with your handlers and
the standard `queue:work` worker. They are the Laravel face of the cross-SDK spec — follow
the linked spec page for the full contract, and the
[php-sdk reference](/docs/php-sdk/1.x/reliability-and-governance/) for the complete API.

## Idempotency

A handler that fires an external effect (a charge, an email) should be idempotent under
at-least-once delivery. Wrap the work in `Idempotent::wrap` so a redelivered `meta.id` is
**skipped, not re-run**:

```php
use BabelQueue\Idempotency\Idempotent;
use BabelQueue\Idempotency\PdoStore;

final class OrderCreatedHandler
{
    public function handle(array $data, array $meta, string $traceId): void
    {
        $store = new PdoStore(\DB::connection()->getPdo()); // shared across the fleet
        Idempotent::wrap($store, fn () => $this->chargeOnce($data))($data, $meta);
    }
}
```

For a fleet, use the shipped persistent `PdoStore` / `RedisStore` (both implement the
`ClaimingStore` claim/release contract) instead of `InMemoryStore`; `PdoStore::ddl()` emits
the portable `CREATE TABLE` for a migration. See [Idempotency](/docs/spec/1.x/idempotency/)
and the [store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

To remove the **dual write** between your Eloquent write and the publish, persist the encoded
envelope in the **same DB transaction**, then drain it with a relay (a scheduled command or
queued job):

```php
use BabelQueue\Outbox\Outbox;

\DB::transaction(function () use ($order, $outbox) {
    $order->save();                 // business row
    $outbox->write($envelope);      // the message, same transaction — no commit of its own
});
```

`Outbox`, `OutboxRelay` and the four-method `OutboxStore` (`save` / `fetchUnpublished` /
`markPublished` / `markFailed`) live in `BabelQueue\Outbox`; back the store with a PDO/ORM
adapter whose `save()` runs on the open transaction. See
[Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`BabelQueue\Redrive\Redrive::run($io, $dlq, $options)` moves dead-lettered messages back onto
a queue (reset, dry-run, sandbox `toQueue`, `select`, `bypass`) — typically from an Artisan
command. The `ReplayBypass` guard lets a deliberate replay re-run the idempotent core while
**skipping effects that already fired**:

```php
use BabelQueue\Redrive\ReplayBypass;

public function handle(array $data, array $meta, string $traceId): void
{
    $this->saveOrder($data);                                   // idempotent core
    ReplayBypass::bypassExternalEffects($message, fn () => $this->sendEmail($data));
}
```

See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`BabelQueue\Gdpr\Gdpr::protect()` / `::unprotect()` encrypt only the `data` leaves a schema
marks `x-gdpr-sensitive`, leaving `data` pure JSON and the envelope frozen. `OpenSslCipher`
(over `ext-openssl`) is the reference; bind your own `Cipher` to a KMS in production. Protect
before publishing, unprotect at the top of the handler — and **validate cleartext** on both
sides. See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

`BabelQueue\Otel\Tracing` emits `publish <urn>` / `process <urn>` spans and, on a
header-carrying transport (Redis/AMQP/SQS), injects/extracts the W3C `traceparent` so a
consumer span is a true child of the producer span — degrading to v0.1 `trace_id`
correlation otherwise. The `trace_id` already threaded through your handler logs is the v0.1
floor. See [Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`BabelQueue\Schema\SchemaValidated::assert()` (producer guard) / `::wrap()` (consumer safety
net) validate a message's `data` against the JSON Schema registered for its URN, bridged from
a `babelqueue-registry` manifest via `DirProvider` — run `bqschema` (including the
`gdpr --require` PII audit) in CI. See [Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/symfony/1.x/introduction/

Introduction — Symfony Messenger BabelQueue adapter (1.x).

The **Symfony** adapter plugs BabelQueue into **Symfony Messenger**. You keep
Messenger's transports, handlers, worker and retry — BabelQueue only changes the
**wire format** to the canonical, language-agnostic envelope, so your Symfony
services exchange messages with Laravel, Go, Python, Java, .NET and Node over one
strict JSON format, on the broker you already run.

It is a thin adapter built on the shared core ([`babelqueue/php-sdk`](https://packagist.org/packages/babelqueue/php-sdk)),
which owns the one codec and contract.

## What you get

- **Messenger serializer.** A drop-in `SerializerInterface` (`babelqueue.messenger.serializer`)
  that encodes/decodes the canonical envelope (`schema_version: 1`) on any Messenger
  transport — AMQP/RabbitMQ, Redis, and more.
- **`PolyglotMessage` contract.** Mark a message class with its URN and a pure-array
  payload; everything else stays idiomatic Messenger.
- **URN routing on consume.** Inbound envelopes are mapped from their URN back to your
  message class via a small registry.
- **Retry bridge.** Messenger's `RedeliveryStamp` ⇄ the envelope's top-level
  `attempts`, in both directions.
- **Automatic `trace_id` propagation.** A middleware forwards the inbound `trace_id`
  onto any message a handler dispatches, so a chain of work stays in one trace.

## The envelope

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

See the full [wire contract](/docs/spec/1.x/envelope/). Continue to
[Installation](/docs/symfony/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/symfony/1.x/installation/

Installation — Symfony Messenger BabelQueue adapter (1.x).

```bash
composer require babelqueue/symfony
```

## Requirements

- **PHP** `^8.2`
- **Symfony** `^6.4 | ^7.0` (Messenger)
- A broker Messenger supports — **AMQP/RabbitMQ**, **Redis**, etc.

The adapter pulls in the framework-agnostic core (`babelqueue/php-sdk`)
automatically.

## Enable the bundle

With **Symfony Flex** the bundle is registered for you. Otherwise add it to
`config/bundles.php`:

```php
return [
    // ...
    BabelQueue\Symfony\BabelQueueBundle::class => ['all' => true],
];
```

That registers the Messenger serializer (`babelqueue.messenger.serializer`), the
trace-propagation middleware (`babelqueue.messenger.trace_middleware`) and the URN
message registry into the container.

Next: [Configuration](/docs/symfony/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/symfony/1.x/configuration/

Configuration — point a Symfony Messenger transport at the BabelQueue serializer and map URNs to message classes.

Two pieces: point a Messenger transport at the BabelQueue serializer, and map
inbound URNs to your message classes.

## Messenger transport

Set the transport's `serializer` to `babelqueue.messenger.serializer`. Routing and
retry stay exactly as Messenger defines them.

```yaml
# config/packages/messenger.yaml
framework:
    messenger:
        transports:
            babel:
                dsn: '%env(MESSENGER_TRANSPORT_DSN)%'   # e.g. amqp:// or redis://
                serializer: 'babelqueue.messenger.serializer'
        routing:
            'App\Message\OrderCreated': babel
        buses:
            messenger.bus.default:
                middleware:
                    # Forward trace_id from a handled message to any it dispatches,
                    # so a chain of work stays in one trace.
                    - 'babelqueue.messenger.trace_middleware'
```

## URN ↔ message-class map

To **consume**, the adapter needs to turn an inbound URN back into one of your
message classes. Declare the mapping, plus the default `queue` written to the
envelope's `meta.queue`:

```yaml
# config/packages/babelqueue.yaml
babelqueue:
    queue: 'orders'            # written to the envelope meta.queue
    messages:                  # urn => message class (needed to consume)
        'urn:babel:orders:created': 'App\Message\OrderCreated'
```

> Producing only needs the message to declare its own URN (see
> [Producing messages](/docs/symfony/1.x/producing-messages/)); the registry is what
> lets a consumer route an inbound envelope back to a class.

## The trace middleware

`babelqueue.messenger.trace_middleware` (added to the bus above) attaches the
inbound `trace_id` to the message as a `BabelTraceStamp` and ensures any message a
handler dispatches **inherits** that `trace_id` — so end-to-end tracing needs no
manual plumbing. It is optional but recommended.

Next: [Producing messages](/docs/symfony/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/symfony/1.x/producing-messages/

Producing messages — implement PolyglotMessage and dispatch through Symfony Messenger; BabelQueue encodes the canonical envelope.

Producing is a normal Messenger dispatch. The only requirement is that your message
implements `BabelQueue\Symfony\Contracts\PolyglotMessage`, which tells the serializer
its URN and how to turn it into the pure-JSON `data` payload.

## Define a message

```php
use BabelQueue\Symfony\Contracts\PolyglotMessage;

final class OrderCreated implements PolyglotMessage
{
    public function __construct(public int $orderId) {}

    public function getBabelUrn(): string
    {
        return 'urn:babel:orders:created';
    }

    public function toPayload(): array
    {
        return ['order_id' => $this->orderId];
    }

    public static function fromBabelPayload(array $data): static
    {
        return new self((int) $data['order_id']);
    }
}
```

- `getBabelUrn()` — the message's [URN](/docs/spec/1.x/urn/) identity on the wire.
- `toPayload()` — the pure, JSON-encodable `data` (no PHP objects; see the
  [data rules](/docs/spec/1.x/envelope/#cross-language-data-rules)).
- `fromBabelPayload()` — rebuilds the message from an inbound envelope on consume.

## Dispatch

```php
// a normal Messenger dispatch
$bus->dispatch(new OrderCreated(1042));
```

On the wire it becomes the canonical envelope — readable by every BabelQueue SDK:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "…",
  "data": { "order_id": 1042 },
  "meta": { "id": "…", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

`trace_id` is stamped automatically. If you dispatch while handling another
BabelQueue message (with the trace middleware enabled), the new message **inherits**
the inbound `trace_id`.

Next: [Consuming messages](/docs/symfony/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/symfony/1.x/consuming-messages/

Consuming messages — a normal Symfony Messenger handler, routed by message class; BabelQueue decodes the envelope and bridges retry + trace.

Consuming is a normal Messenger handler. The serializer decodes the inbound envelope,
uses the [URN → class map](/docs/symfony/1.x/configuration/#urn--message-class-map) to
rebuild your message via `fromBabelPayload()`, and Messenger routes it to the handler
for that class.

## A handler

```php
use Symfony\Component\Messenger\Attribute\AsMessageHandler;

#[AsMessageHandler]
final class OnOrderCreated
{
    public function __invoke(OrderCreated $message): void
    {
        // $message->orderId === 1042
    }
}
```

Run the worker exactly as you normally would:

```bash
php bin/console messenger:consume babel
```

## How it maps to Messenger

- **Routing** is Messenger's job: the decoded message class is routed to its handler.
  An inbound URN with no entry in the `messages` map cannot be rebuilt — keep the
  registry in sync with the URNs you consume.
- **Retry** bridges both ways: Messenger's `RedeliveryStamp` ⇄ the envelope's
  top-level `attempts`. Your existing Messenger retry strategy applies unchanged.
- **Tracing**: the inbound `trace_id` is attached as a `BabelTraceStamp`. With
  `babelqueue.messenger.trace_middleware` on the bus, any message a handler dispatches
  inherits that `trace_id`, so a whole chain of work stays in one trace.

## Cross-language

Because the wire format is the canonical envelope, a message your Symfony app consumes
may have been produced by **any** BabelQueue SDK — a Laravel app, a Go service, a
Python worker — and vice-versa. The URN is the only shared contract; no class is
shared across languages. See the [wire contract](/docs/spec/1.x/envelope/) and the
[cross-language example](/docs/examples/1.x/cross-language-redis/).

---

# Broker transports

Source: https://babelqueue.com/docs/symfony/1.x/broker-transports/

Run BabelQueue over any Symfony Messenger transport — built-in (AMQP, Redis) or third-party (Kafka, AMQP 1.0) — with the babelqueue.messenger.serializer. Interop is body-authoritative, so no broker-specific BabelQueue transport is needed.

The BabelQueue adapter is a Messenger **serializer**, not a transport. It encodes and
decodes the [canonical envelope](/docs/spec/1.x/envelope/) on whatever Messenger
transport you configure — so **Symfony interoperates over any broker Messenger can
reach**, and there is no broker-specific "BabelQueue transport" to install. Point a
transport at the serializer and you are done:

```yaml
# config/packages/messenger.yaml
framework:
  messenger:
    transports:
      babel:
        dsn: '%env(MESSENGER_TRANSPORT_DSN)%'
        serializer: 'babelqueue.messenger.serializer'
```

## Built-in transports (RabbitMQ, Redis)

Messenger ships first-party transports for **AMQP** (RabbitMQ —
[§2](/docs/spec/1.x/broker-bindings/)) and **Redis**
([§1](/docs/spec/1.x/broker-bindings/)). Set the DSN and the serializer; a Go / Python /
Laravel / … consumer on the same broker reads the byte-identical envelope:

```dotenv
# RabbitMQ
MESSENGER_TRANSPORT_DSN=amqp://guest:guest@localhost:5672/%2f/orders
# …or Redis
MESSENGER_TRANSPORT_DSN=redis://localhost:6379/messages
```

These are the GA path for Symfony; see the [broker setup recipes](/docs/spec/1.x/broker-recipes/)
for the cross-language pairings.

## Beyond the built-ins (Kafka, Pulsar, Artemis/AMQP 1.0)

Messenger has **no first-party transport** for Kafka ([§6](/docs/spec/1.x/broker-bindings/)),
Pulsar ([§5](/docs/spec/1.x/broker-bindings/)) or Artemis over AMQP 1.0
([§7](/docs/spec/1.x/broker-bindings/)). You do **not** need a BabelQueue-specific transport
for them — use a **third-party Messenger transport** for that broker and wire the same
`babelqueue.messenger.serializer` onto it. The serializer puts the canonical envelope in the
message **body**, so the wire format is identical regardless of which transport carries it.

```yaml
framework:
  messenger:
    transports:
      babel-kafka:
        dsn: '%env(KAFKA_TRANSPORT_DSN)%'   # from a community Kafka Messenger transport
        serializer: 'babelqueue.messenger.serializer'
```

> **Interop is body-authoritative.** A cross-language consumer routes on the body's `job`
> URN, which is always present — so a Symfony-produced message is consumed correctly on any
> of these brokers even when the transport doesn't reproduce the binding's *native* property
> projection (AMQP `type`, Pulsar/Kafka `bq-job` header, …). The serializer also emits the
> cross-broker `bq-*` headers (`bq-job`, `bq-trace-id`, `bq-message-id`, `bq-schema-version`,
> `bq-source-lang`, `bq-attempts`); a transport that maps serializer headers onto native
> broker headers (e.g. a Kafka Messenger transport → Kafka record headers) carries them
> through, so consumers can also route without decoding the body.

## When you want the native projection

If you need the **exact native projection** of a binding (so non-PHP consumers route on the
broker-native field — AMQP `type`, SQS `MessageAttributes`, the Pulsar/Kafka `bq-` headers,
the Artemis STOMP framing), reach for the framework-less
[`babelqueue/php-sdk`](/docs/php-sdk/1.x/transports/) transports directly instead of going
through Messenger:

| Broker | Native produce / consume on `php-sdk` |
| :--- | :--- |
| Apache Kafka (§6) | `KafkaTransport` (produce) · `KafkaConsumer` + retry topics (consume) |
| Apache Pulsar (§5) | `PulsarTransport` (produce) · `PulsarConsumer` (consume) |
| Apache Artemis (§7) | `StompTransport` (produce); Laravel ships the `babelqueue-artemis` consume driver |
| Amazon SQS (§3) | `SqsTransport` |

These emit the binding's native `bq-`/property projection verbatim. The envelope is the same
`schema_version: 1` either way — Messenger-serializer or framework-less transport is purely a
wiring choice.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/symfony/1.x/reliability-and-governance/

Using BabelQueue's reliability and governance helpers from the Symfony Messenger adapter — the native IdempotencyMiddleware (seen-set or ClaimingStore), TracePropagationMiddleware, plus the php-sdk core helpers the adapter re-exposes: the transactional outbox, DLQ redrive + replay-bypass, and GDPR field encryption. Messenger's worker, retry and failure transport stay untouched; nothing is added to the frozen envelope.

The Symfony adapter is a thin **Messenger serializer** on the framework-agnostic PHP core
([`babelqueue/php-sdk`](https://packagist.org/packages/babelqueue/php-sdk)). It adds two
Symfony-native pieces of wiring — an idempotency middleware and a trace-propagation
middleware — and otherwise re-exposes the core's **optional reliability and governance
helpers**, which you apply at produce/consume time. None touch the frozen envelope
(`schema_version: 1`); Messenger's worker, retry and failure transport stay as they are.
They are the Symfony face of the cross-SDK spec — follow each linked spec page for the full
contract, and the [php-sdk reference](/docs/php-sdk/1.x/reliability-and-governance/) for the
complete core API.

## Idempotency (native middleware)

The adapter ships `BabelQueue\Symfony\Messenger\IdempotencyMiddleware` — the Messenger-idiomatic
wrapper around the core's `Idempotent::wrap` / `ClaimingDispatch`. It deduplicates a
redelivered message on its canonical `meta.id` (carried on a `BabelMessageIdStamp` the
serializer attaches on decode), so a handler runs **once** per logical message under
at-least-once delivery. It acts only on the **receive** path; a fresh outbound dispatch passes
straight through.

Enable it in the bundle config and register it on the consuming bus, **before** the handler
middleware:

```yaml
# config/packages/babelqueue.yaml
babelqueue:
    idempotency:
        enabled: true
        store: App\Babel\RedisIdempotencyStore   # a BabelQueue\Idempotency\IdempotencyStore service
        ttl: 3600                                  # in-flight claim TTL (ClaimingStore only)

framework:
    messenger:
        buses:
            messenger.bus.default:
                middleware:
                    - 'babelqueue.messenger.idempotency_middleware'
```

With a plain `IdempotencyStore` the middleware is **post-success seen-set** dedupe; with a
`ClaimingStore` (a shared, persistent backend — the core's `PdoStore` / `RedisStore`) it uses
the stronger **claim/commit/release** lifecycle, so exactly one of N concurrent deliveries
wins. A delivery that loses to an in-flight peer throws `ClaimParkedException` so Messenger
does **not** ack it — the broker redelivers it later, by when the winner has committed. Left
unset, `store` falls back to a bundled in-memory store (single-process / tests only). See
[Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

The adapter has no outbox of its own — use the core `BabelQueue\Outbox` helpers to remove the
**dual write** between your Doctrine write and the publish. Persist the encoded envelope in the
**same DB transaction** as your business row, then drain it with a relay (a `messenger:consume`
worker on a dedicated bus, or a scheduled command):

```php
use BabelQueue\Outbox\Outbox;

$em->wrapInTransaction(function () use ($order, $outbox, $envelope) {
    $em->persist($order);        // business row
    $outbox->write($envelope);   // the message, same transaction — no commit of its own
});
```

`Outbox`, `OutboxRelay` and the four-method `OutboxStore` (`save` / `fetchUnpublished` /
`markPublished` / `markFailed`) live in `BabelQueue\Outbox`; back the store with a Doctrine
DBAL adapter whose `save()` runs on the open transaction. See
[Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`BabelQueue\Redrive\Redrive::run($io, $dlq, $options)` moves dead-lettered messages back onto a
queue (reset, dry-run, sandbox `toQueue`, `select`, `bypass`) — typically from a console
command bound to your broker. The core `ReplayBypass` guard lets a deliberate replay re-run the
idempotent core while **skipping effects that already fired**:

```php
use BabelQueue\Redrive\ReplayBypass;

ReplayBypass::bypassExternalEffects($message, fn () => $this->sendEmail($data));
```

See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`BabelQueue\Gdpr\Gdpr::protect()` / `::unprotect()` encrypt only the `data` leaves a schema
marks `x-gdpr-sensitive`, leaving `data` pure JSON and the envelope frozen. `OpenSslCipher`
(over `ext-openssl`) is the reference; bind your own `Cipher` to a KMS in production. Protect
before the message is dispatched/serialized, unprotect at the top of the handler — and
**validate cleartext** on both sides. See
[GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

The adapter's `TracePropagationMiddleware` already forwards an inbound `trace_id` onto any
message a handler dispatches, keeping a chain of work in one trace (the v0.1 floor). The core
`BabelQueue\Otel\Tracing` adds `publish <urn>` / `process <urn>` spans and, on a
header-carrying transport (Redis/AMQP/SQS), injects/extracts the W3C `traceparent` so a
consumer span is a true child of the producer span — degrading to v0.1 `trace_id` correlation
otherwise. See [Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`BabelQueue\Schema\SchemaValidated::assert()` (producer guard) / `::wrap()` (consumer safety
net) validate a message's `data` against the JSON Schema registered for its URN, bridged from a
`babelqueue-registry` manifest via `DirProvider` — run `bqschema` (including the
`gdpr --require` PII audit) in CI. See [Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/babelqueue-python/1.x/introduction/

Introduction — Python BabelQueue SDK (1.x).

The **Python** SDK implements the BabelQueue standard so your Python services
produce and consume the exact same strict JSON envelope as every other language
in the ecosystem — no PHP `serialize()`, no language lock-in.

Python is the natural **AI/ML & data** consumer in a BabelQueue topology: let a
Laravel app publish `urn:babel:orders:created`, and a Python worker picks it up to
run scoring, enrichment or an ML pipeline — reading the same canonical envelope.

The package ships two layers: the **zero-dependency core** (the `EnvelopeCodec`,
contracts and dead-letter helpers) and a small **`BabelQueue` runtime** that
publishes and consumes over a broker for end-to-end apps.

## What you get

- **Canonical wire envelope (`schema_version: 1`).** `job`, `trace_id`, `data`,
  `meta` and `attempts` — byte-for-byte compatible with the other SDKs.
- **A `trace_id` on every message** for cross-service correlation across your
  data and ML pipelines.
- **URN routing** (`urn:babel:<context>:<event>`) so consumers route on a stable
  key, not a class name.
- **Pluggable transports** chosen by URL scheme: `memory://` (built in),
  `redis://` (extra `[redis]`) and `amqp://` (RabbitMQ, extra `[amqp]`).

## The envelope

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "default", "lang": "python", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Continue to [Installation](/docs/babelqueue-python/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/babelqueue-python/1.x/installation/

Installation — Python BabelQueue SDK (1.x).

Install the Python SDK from PyPI:

```bash
pip install babelqueue
```

The base install is the zero-dependency core (codec, contracts and dead-letter
helpers) **plus** the `BabelQueue` runtime with an in-process `memory://`
transport — enough for tests and local development.

## Requirements

- **Python** `>=3.9`
- A running **Redis** or **RabbitMQ** broker (only for the matching transport)

## Broker extras

Real brokers are opt-in via extras, which pull in the standard client for that
transport:

```bash
pip install "babelqueue[redis]"   # redis://  (via redis-py)
pip install "babelqueue[amqp]"    # amqp://   (RabbitMQ, via pika)
```

Next: [Configuration](/docs/babelqueue-python/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/babelqueue-python/1.x/configuration/

Configuration — Python BabelQueue SDK (1.x).

Create a `BabelQueue` runtime and point it at your broker with a URL. The
transport is chosen from the URL scheme — no separate "broker" flag.

## Broker URL

```python
from babelqueue import BabelQueue

# Redis (extra: babelqueue[redis])
app = BabelQueue("redis://localhost:6379/0", queue="orders")

# RabbitMQ (extra: babelqueue[amqp])
# app = BabelQueue("amqp://guest:guest@localhost:5672/", queue="orders")

# Amazon SQS (extra: babelqueue[sqs])
# app = BabelQueue("sqs://eu-central-1", queue="orders")

# Azure Service Bus (extra: babelqueue[azureservicebus])
# app = BabelQueue("sb://<namespace>.servicebus.windows.net", queue="orders")

# Apache Pulsar (extra: babelqueue[pulsar])
# app = BabelQueue("pulsar://localhost:6650", queue="orders")

# Apache Kafka (extra: babelqueue[kafka])
# app = BabelQueue("kafka://localhost:9092", queue="orders")

# Apache ActiveMQ Artemis (extra: babelqueue[artemis])
# app = BabelQueue("artemis://localhost:5672", queue="orders")

# In-process, great for tests/local — no broker required
# app = BabelQueue("memory://", queue="orders")
```

| Scheme      | Transport                      | Requires            |
| ----------- | ------------------------------ | ------------------- |
| `memory://` | In-process (tests / local)     | built in            |
| `redis://`  | Redis (reliable-queue pattern) | `babelqueue[redis]` |
| `amqp://`   | RabbitMQ (via `pika`)          | `babelqueue[amqp]`  |
| `sqs://`    | Amazon SQS (via `boto3`)       | `babelqueue[sqs]`   |
| `sb://`     | Azure Service Bus (via `azure-servicebus`) | `babelqueue[azureservicebus]` |
| `pulsar://` | Apache Pulsar (via `pulsar-client`) | `babelqueue[pulsar]` |
| `kafka://`  | Apache Kafka (via `confluent-kafka`) | `babelqueue[kafka]` |
| `artemis://` | Apache ActiveMQ Artemis (via `python-qpid-proton`) | `babelqueue[artemis]` |

The `sqs://` URL is `sqs://<region>` with optional query params: `endpoint`
(LocalStack/ElasticMQ), `prefix` (a queue-URL prefix that skips `GetQueueUrl`), `fifo=1`,
`group_id`, `wait_time`. The transport sends the canonical envelope with the
[§3 `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs) and reconciles
`attempts` to `ApproximateReceiveCount − 1`.

The `sb://` URL is `sb://<namespace>.servicebus.windows.net` and authenticates with
`DefaultAzureCredential` (Azure AD); pass `connection_string=...` to `AsbTransport` for
SAS auth, or `BabelQueue(transport=...)` with a built client. It maps the envelope onto
native Service Bus fields ([§4](/docs/spec/1.x/broker-bindings/#azure-service-bus)) and
reconciles `attempts` to `max(body, DeliveryCount − 1)`.

The `pulsar://` URL is `pulsar://<host>:<port>` (or `pulsar+ssl://`); pass a built client
via `BabelQueue(transport=...)` for TLS/token auth. It projects the envelope onto native
Pulsar message properties (`bq-job` = URN, plus the `bq-` fields,
[§5](/docs/spec/1.x/broker-bindings/#apache-pulsar)), defaults to a `Shared` subscription
named `babelqueue`, and reconciles `attempts` to `max(body, redelivery_count)` — the
redelivery count is 0-based, so **no −1**.

The `kafka://` URL is `kafka://<host>:<port>[,<host>:<port>]`; pass an injected `producer` +
`consumer_factory` for custom auth. The record value is the envelope, projected onto `bq-`
headers ([§6](/docs/spec/1.x/broker-bindings/#apache-kafka)); consume is **process-then-commit**
(`enable.auto.commit=false`) and `attempts` reconciles to the authoritative `bq-attempts`
header (else the body). Kafka has no native delay/DLQ, so the runtime owns retry by
republishing with attempts+1 and dead-letters to `<queue>.dlq`.

The `artemis://` URL is `artemis://<host>:<port>` (or `artemis+ssl://`) — Apache ActiveMQ
Artemis over **AMQP 1.0** (`python-qpid-proton`, not RabbitMQ's 0-9-1 `pika`); pass a built
connection via `BabelQueue(transport=...)` for auth/TLS. It projects the envelope onto the AMQP
a JMS peer reads — `x-opt-jms-type` = URN, `correlation-id` = `trace_id`, plus the string `bq_`
application properties (underscored — JMS property names must be valid Java identifiers; [§7](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis)) — and
reconciles `attempts` to `max(body, delivery_count)`; the AMQP delivery-count is 0-based, so
**no −1**.

## Options

| Option           | Default   | Description                                              |
| ---------------- | --------- | ------------------------------------------------------- |
| `queue`          | `default` | Queue name written to `meta.queue`                      |
| `max_attempts`   | —         | Retries before a message is exhausted (bumps `attempts`)|
| `dead_letter`    | `False`   | Quarantine exhausted messages on `<queue>.dlq`          |
| `on_unknown_urn` | `fail`    | `fail` \| `delete` \| `release` \| `dead_letter`        |

```python
app = BabelQueue(
    "redis://localhost:6379/0",
    queue="orders",
    max_attempts=3,
    dead_letter=True,
    on_unknown_urn="dead_letter",
)
```

> Need just the wire codec, with no broker wiring? Import `EnvelopeCodec`
> directly — see [Producing messages](/docs/babelqueue-python/1.x/producing-messages/).

Next: [Producing messages](/docs/babelqueue-python/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/babelqueue-python/1.x/producing-messages/

Producing messages — Python BabelQueue SDK (1.x).

## With the runtime

Publish a message with its URN and payload. The `BabelQueue` runtime builds the
canonical envelope and pushes it to your broker:

```python
from babelqueue import BabelQueue

app = BabelQueue("redis://localhost:6379/0", queue="orders")

app.publish("urn:babel:orders:created", {
    "order_id": 1042,
    "amount": 99.90,
})
```

## With the codec only

If you manage the broker connection yourself, build the envelope with
`EnvelopeCodec` and publish the JSON to your own client:

```python
from babelqueue import EnvelopeCodec

env  = EnvelopeCodec.make("urn:babel:orders:created", {"order_id": 1042, "amount": 99.90})
body = EnvelopeCodec.encode(env)   # -> UTF-8 JSON string
# redis.rpush("queues:orders", body)
#   /  channel.basic_publish(body=body, ...)
```

The payload above is wrapped in the canonical `schema_version: 1` envelope —
identical to what every other SDK emits, with `meta.lang` set to `"python"`:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "python", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/babelqueue-python/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/babelqueue-python/1.x/consuming-messages/

Consuming messages — Python BabelQueue SDK (1.x).

## With the runtime

Register a handler for a URN with the `@app.handler` decorator, then start the
worker with `app.run()`. Your function receives the decoded `data` dict and the
`meta` block — regardless of which language produced the message. Add a third
`message` parameter to get the full envelope (including `trace_id`).

```python
from babelqueue import BabelQueue

app = BabelQueue("redis://localhost:6379/0", queue="orders")

@app.handler("urn:babel:orders:created")
def on_order_created(data, meta, message):
    trace_id = message["trace_id"]  # cross-service correlation
    print(f"[{trace_id}] order {data['order_id']} for {data['amount']}")
    # ... run scoring / enrichment / ML pipeline

app.run()  # consume forever (Ctrl-C to stop)
```

## With the codec only

Managing the broker yourself? Pull the bytes from your client, then decode and
route on the URN:

```python
from babelqueue import EnvelopeCodec

incoming = EnvelopeCodec.decode(body)   # body pulled from your broker
urn      = incoming["job"]              # "urn:babel:orders:created"
data     = incoming["data"]            # {"order_id": 1042, ...}
trace_id = incoming["trace_id"]        # correlate across services
```

Python is the **AI/ML & data** consumer in a polyglot topology: a Laravel app can
publish `urn:babel:orders:created`, and this Python worker enriches it or feeds it
into a model — reading the same canonical envelope, carrying the same `trace_id`
end to end.

That's the whole loop: any producer, any consumer, one schema.

---

# Celery adapter

Source: https://babelqueue.com/docs/babelqueue-python/1.x/celery/

Run the BabelQueue runtime on your existing Celery broker — from_celery(app) and install_worker(app) drain canonical envelopes alongside your Celery tasks.

The Celery adapter builds a BabelQueue runtime on your **existing Celery broker**, so
your Celery app can produce and consume canonical envelopes that interoperate with
every other BabelQueue SDK — no separate broker, no second worker.

## Install

```bash
pip install "babelqueue[celery]"
```

The `[celery]` extra adds `celery>=5`. The core stays zero-dependency; `celery` is
imported lazily.

## Build a runtime on the Celery app

`from_celery(celery_app, **kwargs)` reads the broker URL off the Celery app and
returns a `BabelQueue` runtime. Extra kwargs (`queue`, `max_attempts`,
`dead_letter`, `on_unknown_urn`, …) are forwarded to the runtime:

```python
from celery import Celery
from babelqueue.celery import from_celery, install_worker

celery_app = Celery("svc", broker="redis://localhost:6379/0")

bq = from_celery(celery_app, queue="orders", dead_letter=True)

# Consume a canonical envelope (produced by any SDK)
@bq.handler("urn:babel:orders:created")
def on_created(data, meta):
    print("order", data["order_id"])

# Produce a canonical envelope
bq.publish("urn:babel:orders:created", {"order_id": 1042})
```

A handler is `(data, meta)` — or `(data, meta, message)` to receive the full envelope
dict (including `trace_id`).

## Drain inside the Celery worker

`install_worker(celery_app, babel_app=None, *, queue=None, **kwargs)` registers a
Celery **bootstep** that, when `celery worker` boots, runs the BabelQueue consume
loop in a background daemon thread — so envelopes are drained alongside your normal
Celery tasks:

```python
install_worker(celery_app, bq, queue="orders")
```

```bash
celery -A svc worker
```

If you omit `babel_app`, it is built via `from_celery(celery_app, **kwargs)` for you.

See the [wire contract](/docs/spec/1.x/envelope/) for the envelope your handlers
receive.

---

# Django adapter

Source: https://babelqueue.com/docs/babelqueue-python/1.x/django/

Settings-driven BabelQueue for Django — a BABELQUEUE config, get_app()/publish() helpers, and a manage.py babelqueue_worker command.

The Django adapter wires the BabelQueue runtime into a Django project via settings,
with helpers to publish from your code and a management command to run the worker.

## Install

```bash
pip install "babelqueue[django]"
```

The `[django]` extra adds `django>=4.2`. Add the app to `INSTALLED_APPS`:

```python
# settings.py
INSTALLED_APPS = [
    # ...
    "babelqueue.django",
]
```

## Configure

A single `BABELQUEUE` settings dict configures the runtime. `broker_url` selects the
transport; the other recognised keys (`queue`, `on_unknown_urn`, `max_attempts`,
`dead_letter`, `dead_letter_queue`, `dead_letter_suffix`, `transport`) are forwarded
to the runtime:

```python
# settings.py
BABELQUEUE = {
    "broker_url": "redis://localhost:6379/0",
    "queue": "orders",
    "dead_letter": True,
}
```

## Publish

```python
# views.py
from babelqueue.django import publish

def create_order(request):
    publish("urn:babel:orders:created", {"order_id": 1042})
    ...
```

`publish(urn, data, **kwargs) → str` returns the message `meta.id`, using the
process-wide app from `get_app()`.

## Register handlers

Register handlers into the same app the worker uses — the app config's `ready()` is
the natural place:

```python
# apps.py
from django.apps import AppConfig
from babelqueue.django import get_app

class MyAppConfig(AppConfig):
    name = "myapp"

    def ready(self):
        @get_app().handler("urn:babel:orders:created")
        def on_created(data, meta):
            ...
```

## Run the worker

```bash
python manage.py babelqueue_worker --queue orders
```

Options: `--queue` (defaults to the configured queue), `--max-messages` (default:
run until interrupted), `--timeout` (per-poll seconds, default `1.0`).

The messages your Django app produces and consumes are the
[canonical envelope](/docs/spec/1.x/envelope/), interoperable with every BabelQueue SDK.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/babelqueue-python/1.x/reliability-and-governance/

The optional, stdlib-only reliability and governance helpers in the Python core — idempotency (wrap + seen-set store), the transactional outbox, DLQ redrive + replay-bypass, GDPR field encryption (with the [gdpr] extra for the reference cipher), and OpenTelemetry traceparent (the [otel] extra). Each composes with the runtime, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, the Python core ships **optional, opt-in** helpers that mechanise the
reliability and governance contracts the wire spec defines. The core stays stdlib-only; the
crypto and OTel references live behind extras. None touch the frozen envelope
(`schema_version: 1`); each is a tooling layer over the codec. They are the Python face of the
cross-SDK spec — follow the linked spec page for the full contract.

## Idempotency

`wrap(store, handler)` makes a handler **run at most once per `meta.id`**, even on
at-least-once redelivery. It keeps the handler's signature (via `functools.wraps`):

```python
from babelqueue import BabelQueue, wrap, InMemoryStore

app = BabelQueue("redis://localhost:6379/0", queue="orders")
store = InMemoryStore()  # tests / single process
app.register("urn:babel:orders:created", wrap(store, on_order_created))
```

The in-memory `InMemoryStore` is the reference; for a fleet, implement the `IdempotencyStore`
protocol (`seen` / `remember` / `forget`) over a shared backend (no persistent store ships
in core yet — bring your own). See [Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

`Outbox` / `OutboxRelay` remove the producer **dual write**: persist the encoded envelope in
the **same DB transaction** as your business row, then a relay publishes the durable rows
verbatim.

```python
from babelqueue import Outbox, OutboxRelay, InMemoryOutboxStore

store = InMemoryOutboxStore()   # production: a DB-backed OutboxStore
outbox = Outbox(store)

# inside YOUR DB transaction, beside the business write — no commit of its own:
row_id = outbox.write(envelope) # encodes via the codec, calls store.save()

# later, a relay drains the durable rows to the broker:
result = OutboxRelay(app.transport, store).drain()  # result.published / result.failed
```

`OutboxStore` is the four-method protocol (`save` / `fetch_unpublished` / `mark_published` /
`mark_failed`); the relay forwards through the runtime's publish-only `Transport`
(`publish(queue, body)`). The transaction boundary is **yours**, and the relay backoff is in
**seconds**. See [Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`redrive(transport, dlq, ...)` moves dead-lettered messages back onto a queue — `dead_letter`
removed, `attempts` reset to `0`, everything else preserved — with `dry_run`, `select`,
`to_queue` (sandbox) and `bypass`. The replay guard skips effects that already fired:

```python
from babelqueue import is_replay, bypass_external_effects
from babelqueue.redrive import redrive

# inspect / sandbox-replay, stamping bq-replay-bypass
res = redrive(app.transport, "orders.dlq", to_queue="sandbox", bypass=True)

@app.handler("urn:babel:orders:created")
def on_order_created(data, meta):
    save_order(data)                                  # idempotent core — always runs
    bypass_external_effects(lambda: send_email(data)) # external effect — skipped on replay
```

`is_replay()` reads the per-message replay flag the runtime sets (a `contextvars` flag), so it
takes no argument. See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`protect` / `unprotect` encrypt only the `data` leaves a schema marks `x-gdpr-sensitive`, in
place. `Cipher` is a caller-bound protocol; the core ships **no concrete cipher** (Python's
stdlib has no AES-GCM) — the AES-256-GCM reference rides the optional `babelqueue[gdpr]` extra
(`cryptography`). **Validate cleartext** — protect after validation on produce, unprotect
before validation on consume.

```python
from babelqueue import protect, unprotect, Cipher, DecryptError
from babelqueue.schema import sensitive_paths

protect(data, schema, cipher)      # producer: encrypt marked leaves (cipher implements Cipher)
try:
    unprotect(data, schema, cipher)  # consumer: inverse
except DecryptError:
    raise                            # wrong key / tampered → retry / dead-letter
```

See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

The `babelqueue.otel` module (the `babelqueue[otel]` extra, so importing the core stays
dependency-free) emits `publish <urn>` / `process <urn>` spans and, on a header-carrying
transport, injects/extracts the W3C `traceparent` so a consumer span is a true child of the
producer span — degrading to v0.1 `trace_id` correlation otherwise. In-memory, Redis, AMQP and
SQS carry it.

```python
from babelqueue import otel
from opentelemetry import trace

tracer = trace.get_tracer("orders")
app.register("urn:babel:orders:created", otel.wrap_handler(tracer, on_order_created))
otel.publish(tracer, app, "urn:babel:orders:created", {"order_id": 1042})
```

See [Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`validate(provider, urn, data)` (producer guard) / `wrap(provider, urn, handler)` (consumer
safety net) validate a message's `data` against the JSON Schema registered for its URN,
bridged from a `babelqueue-registry` manifest via `DirProvider`. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/babelqueue-go/1.x/introduction/

Introduction — Go BabelQueue SDK (1.x).

The **Go** SDK is the framework-agnostic **core**: a wire-envelope codec,
contracts and dead-letter helpers that read and write the exact same strict JSON
envelope as every other language in the ecosystem.

At its heart it is the **codec**: you build (or decode) the canonical envelope and
move the bytes with your own broker client, so it works over any broker. On top of
that — and **still zero-dependency** — an optional `App` runtime
(publish/consume/worker loop) and **transport modules for all seven brokers** are
available when you want them.

## What you get

- Canonical, schema-validated JSON envelopes (`schema_version: 1`) — no language
  lock-in.
- A built-in `trace_id` on every message for cross-service correlation.
- URN routing (`urn:babel:<context>:<event>`) on a stable key.
- **Zero dependencies** in the core (standard library only) — bring your own broker
  client.
- An **optional `App` runtime + transport modules** for Redis, RabbitMQ, Amazon SQS,
  Azure Service Bus, Apache Pulsar, Apache Kafka and Apache ActiveMQ/Artemis — opt in
  when you want publish/consume instead of just the codec. See
  [Runtime & transports](/docs/babelqueue-go/1.x/runtime-and-transports/).
- **Framework adapters** — `…/asynq` and `…/machinery` bind the codec to those Go
  task queues.

Continue to [Installation](/docs/babelqueue-go/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/babelqueue-go/1.x/installation/

Installation — Go BabelQueue SDK (1.x).

Install the Go SDK:

```bash
go get github.com/babelqueue/babelqueue-go
```

## Requirements

- **Go** `>=1.21`

The core is **broker-agnostic with zero dependencies** (standard library only).
You bring your own Redis or RabbitMQ client to move the encoded bytes — there is
no separate BabelQueue runtime to install.

Next: [Configuration](/docs/babelqueue-go/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/babelqueue-go/1.x/configuration/

Configuration — Go BabelQueue SDK (1.x).

The core is a codec, so there's almost nothing to configure — the "wiring" is
your own broker client plus a couple of optional behaviours.

## Envelope options

`Make` accepts functional options. Set the queue name (written to `meta.queue`)
and continue an inbound trace across a hop:

```go
env, _ := babelqueue.Make(
    "urn:babel:orders:created",
    map[string]any{"order_id": 1042},
    babelqueue.WithQueue("orders"),
    babelqueue.WithTraceID(inbound.TraceID), // omit to mint a fresh trace_id
)
```

## Dead-letter

On failure, wrap the envelope in an additive `dead_letter` block and publish the
copy to your dead-letter queue (e.g. `orders.dlq`):

```go
dl := babelqueue.Annotate(env, "failed", "orders", 3, err)
body, _ := dl.Encode()
// publish body to the "orders.dlq" queue with your broker client
```

`Annotate` returns a copy — the original envelope is preserved unchanged inside
the dead-lettered message, so any-language consumer can still read it.

## Unknown-URN strategy

For adapters routing inbound messages, the strategy constants `StrategyFail`,
`StrategyDelete`, `StrategyRelease` and `StrategyDeadLetter` describe what to do
with a URN you don't handle.

Next: [Producing messages](/docs/babelqueue-go/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/babelqueue-go/1.x/producing-messages/

Producing messages — Go BabelQueue SDK (1.x).

Build the canonical envelope with `Make`, encode it to JSON with `Encode`, then
publish the bytes with your own broker client.

```go
import babelqueue "github.com/babelqueue/babelqueue-go"

env, err := babelqueue.Make(
    "urn:babel:orders:created",
    map[string]any{"order_id": 1042, "amount": 99.90},
    babelqueue.WithQueue("orders"),
)
if err != nil {
    panic(err)
}

body, _ := env.Encode() // []byte of compact UTF-8 JSON
// redisClient.RPush(ctx, "queues:orders", body)
//   /  ch.PublishWithContext(ctx, "", "orders", false, false, amqp.Publishing{Body: body})
```

`Encode` emits the canonical `schema_version: 1` envelope, with `meta.lang` set
to `"go"`:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "go", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/babelqueue-go/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/babelqueue-go/1.x/consuming-messages/

Consuming messages — Go BabelQueue SDK (1.x).

Pull the bytes from your broker, `Decode` them, gate with `Accepts`, then switch
on the URN — regardless of which language produced the message. `TraceID` carries
the cross-service correlation id.

```go
import babelqueue "github.com/babelqueue/babelqueue-go"

// body pulled from your broker (Redis BLPOP, AMQP delivery, …)
in, err := babelqueue.Decode(body)
if err != nil || !in.Accepts() {
    return // malformed or unsupported — dead-letter / drop
}

switch in.URN() {
case "urn:babel:orders:created":
    log.Printf("[%s] order received: %+v", in.TraceID, in.Data)
}
```

That's the whole loop: any producer, any consumer, one schema.

---

# Runtime & transports

Source: https://babelqueue.com/docs/babelqueue-go/1.x/runtime-and-transports/

The optional zero-dependency App runtime (publish/consume/worker loop) and the Redis, RabbitMQ, Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka & Apache ActiveMQ Artemis transport modules for the Go SDK.

The Go core is a codec, but it also ships an **optional, still zero-dependency
`App` runtime** plus a `Transport` interface and an in-memory transport. Broker
drivers live in **separate modules** so the core stays dependency-free:
`…/redis` (built on `go-redis`), `…/amqp` (built on `amqp091-go`), `…/sqs` (built on
`aws-sdk-go-v2`), `…/azureservicebus` (built on `azservicebus`), `…/pulsar` (built on the
pure-Go `pulsar-client-go`), `…/kafka` (built on the pure-Go `segmentio/kafka-go`) and
`…/artemis` (Apache ActiveMQ Artemis over the pure-Go `Azure/go-amqp`, AMQP 1.0).

## The `App` runtime

```go
func NewApp(transport Transport, opts ...AppOption) *App
```

Options: `WithDefaultQueue(string)` (default `"default"`), `WithMaxAttempts(int)`
(default `3`), `WithUnknownURNStrategy(string)` (default `StrategyFail`),
`WithDeadLetter(bool)`, `WithDeadLetterQueue(string)`, `WithDeadLetterSuffix(string)`
(default `".dlq"`), `WithPollTimeout(time.Duration)`.

Methods:

- `Handle(urn string, handler Handler)` — register a handler. `Handler` is
  `func(ctx, env Envelope) error`; returning an error triggers retry/dead-letter,
  `nil` acks.
- `Publish(ctx, urn string, data map[string]any, opts ...Option) (string, error)` —
  builds, encodes and publishes the canonical envelope; returns `meta.id`.
- `Consume(ctx, queue ...string) error` — blocks, routing by URN until `ctx` is
  cancelled (one bad message never stops the loop). `Run(ctx)` is the alias on the
  default queue.
- `Drain(ctx, queue string, max int) (int, error)` — process up to `max` (or until
  empty), returns the count. Handy for tests / one-shot workers.

Unknown-URN strategies are the string constants `StrategyFail`, `StrategyDelete`,
`StrategyRelease`, `StrategyDeadLetter`.

### In-memory example

```go
package main

import (
	"context"
	"fmt"

	babelqueue "github.com/babelqueue/babelqueue-go"
)

func main() {
	ctx := context.Background()
	app := babelqueue.NewApp(babelqueue.NewInMemoryTransport(),
		babelqueue.WithDefaultQueue("orders"),
		babelqueue.WithDeadLetter(true),
	)

	app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
		fmt.Println(env.Data["order_id"], env.TraceID)
		return nil
	})

	app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
	app.Drain(ctx, "orders", 0) // Consume(ctx) blocks for a long-running worker
}
```

## Redis transport

```bash
go get github.com/babelqueue/babelqueue-go/redis
```

Reliable-queue pattern (`RPUSH` to produce, `BLMOVE` to a `:processing` list to
reserve, `LREM` to ack). Create it with a URL or your own client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/redis"
)

tr, err := redis.New("redis://localhost:6379/0") // or redis.NewWithClient(client)
if err != nil {
	return err
}
defer tr.Close()

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil // handle env.Data
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Option: `redis.WithProcessingSuffix(string)` (default `":processing"`).

## RabbitMQ (AMQP) transport

```bash
go get github.com/babelqueue/babelqueue-go/amqp
```

Durable queue, persistent delivery, `basic.get` + manual ack (at-least-once). The
connection is lazy — it (re)connects on first use:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/amqp"
)

tr := amqp.New("amqp://guest:guest@localhost:5672/")
defer tr.Close()

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx)
```

Each published message carries the contract AMQP properties so consumers can route
without parsing the body: `type` = URN, `correlationId` = `trace_id`, `messageId` =
`meta.id`, plus `x-attempts` / `x-schema-version` / `x-source-lang` headers,
`application/json` + persistent delivery.

## Amazon SQS transport

```bash
go get github.com/babelqueue/babelqueue-go/sqs
```

A `babelqueue.Transport` over the official AWS SDK (`aws-sdk-go-v2`): the canonical
envelope is the `MessageBody`, projected onto native `MessageAttributes`; a
visibility-timeout reserve → `DeleteMessage` ack gives at-least-once. A failed handler's
message is released with `ChangeMessageVisibility` (default delay 0 s, so it redelivers at
once); pair each queue with a native `RedrivePolicy` (`maxReceiveCount` → a DLQ) so a
message that always fails cannot loop forever. Build it with your
AWS config (region from the standard chain) or your own client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/sqs"
)

tr, err := sqs.New(ctx, sqs.WithRegion("eu-central-1")) // or sqs.NewWithClient(client)
if err != nil {
	return err
}

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Options: `WithEndpoint` (LocalStack/ElasticMQ), `WithQueueURLPrefix` (skip `GetQueueUrl`),
`WithWaitTimeSeconds`, `WithVisibilityTimeout`, `WithFIFO`, `WithMessageGroupID`,
`WithContentDedup`, `WithClient`. Each message carries the §3 `MessageAttributes`
(`bq-job` = URN, `bq-trace-id` = `trace_id`, `bq-message-id` = `meta.id`, plus
`bq-schema-version` / `bq-source-lang` / `bq-created-at`) so a consumer can route without
decoding the body; `attempts` is reconciled to `ApproximateReceiveCount − 1`. See the
[SQS binding](/docs/spec/1.x/broker-bindings/#amazon-sqs).

## Azure Service Bus transport

```bash
go get github.com/babelqueue/babelqueue-go/azureservicebus
```

A `babelqueue.Transport` over the official Azure SDK (`azservicebus`): the canonical
envelope is the message `Body`, projected onto native fields (`Subject` = URN,
`CorrelationID` = `trace_id`, `MessageID` = `meta.id`) plus the `bq-` application
properties; a PeekLock reserve → `CompleteMessage` ack gives at-least-once. Build it from a
connection string, your own `*azservicebus.Client`, or an injected client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/azureservicebus"
)

tr, err := azureservicebus.New(azureservicebus.WithConnectionString(connStr))
if err != nil {
	return err
}

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Options: `WithAzureClient(*azservicebus.Client)` (e.g. a namespace + token credential),
`WithClient`, `WithMaxWaitTime`. `attempts` is reconciled to
`max(body.attempts, DeliveryCount − 1)`. Requires **Go 1.23+** (the Azure SDK floor). See
the [Azure Service Bus binding](/docs/spec/1.x/broker-bindings/#azure-service-bus).

## Apache Pulsar transport

```bash
go get github.com/babelqueue/babelqueue-go/pulsar
```

A `babelqueue.Transport` over the **pure-Go** `apache/pulsar-client-go` (no CGo, no
libpulsar): the canonical envelope is the message payload, projected onto native Pulsar
message properties (`bq-job` = URN, `bq-trace-id` = `trace_id`, `bq-message-id` = `meta.id`,
plus `bq-schema-version` / `bq-source-lang` / `bq-attempts`, all string→string); a
receive → `Ack` gives at-least-once. Build it from a service URL, your own `pulsar.Client`,
or an injected client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/pulsar"
)

tr, err := pulsar.New(pulsar.WithURL("pulsar://localhost:6650"))
if err != nil {
	return err
}

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Options: `WithPulsarClient(pulsar.Client)`, `WithClient`, `WithSubscription` (default
`babelqueue`), `WithSubscriptionType` (default `Shared`), `WithTopicPrefix`,
`WithMaxWaitTime`. `attempts` is reconciled to `max(body.attempts, RedeliveryCount)` — the
redelivery count is 0-based, so **no −1**. Requires **Go 1.23+**. See the
[Apache Pulsar binding](/docs/spec/1.x/broker-bindings/#apache-pulsar).

## Apache Kafka transport

```bash
go get github.com/babelqueue/babelqueue-go/kafka
```

A `babelqueue.Transport` over the **pure-Go** `segmentio/kafka-go` (no CGo, no librdkafka):
the canonical envelope is the record value, projected onto native Kafka record headers
(`bq-job` = URN, plus `bq-trace-id` / `bq-message-id` / `bq-schema-version` / `bq-source-lang`
/ `bq-attempts`, all UTF-8 byte strings) with the record timestamp mirroring `meta.created_at`.
Consuming is **process-then-commit**: `FetchMessage` reserves a record (no auto-commit) and
`Ack` commits the offset. Build it from brokers + a group id, or an injected client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/kafka"
)

tr, err := kafka.New(kafka.WithBrokers("localhost:9092"), kafka.WithGroupID("orders-workers"))
if err != nil {
	return err
}

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Options: `WithBrokers`, `WithGroupID`, `WithWriter` / `WithReaderFactory` (for tests),
`WithMaxWaitTime`. Kafka has no native delivery count, so `attempts` is reconciled from the
**authoritative `bq-attempts` header** (else the body); the App owns retry by republishing
with attempts+1 and dead-letters to `<queue>.dlq`. Requires **Go 1.23+**. See the
[Apache Kafka binding](/docs/spec/1.x/broker-bindings/#apache-kafka).

## Apache ActiveMQ Artemis transport

```bash
go get github.com/babelqueue/babelqueue-go/artemis
```

A `babelqueue.Transport` over the **pure-Go** `Azure/go-amqp` (**AMQP 1.0**, no CGo) — Artemis
speaks AMQP 1.0, not RabbitMQ's 0-9-1, so this is a distinct module from `…/amqp`. The canonical
envelope is the message body, projected onto the AMQP a JMS peer reads: the `x-opt-jms-type`
annotation = URN, `correlation-id` = `trace_id`, `creation-time` = `meta.created_at`, plus the
string `bq_` application properties (`bq_schema_version` / `bq_source_lang` / `bq_attempts` /
`bq_app_id`; underscored, since JMS property names must be valid Java identifiers). Consuming is `Receive` → process → `Ack` (accept) for at-least-once. Build it from
an AMQP URL or an injected client:

```go
import (
	babelqueue "github.com/babelqueue/babelqueue-go"
	"github.com/babelqueue/babelqueue-go/artemis"
)

tr, err := artemis.New(ctx, artemis.WithURL("amqp://localhost:5672"))
if err != nil {
	return err
}

app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))
app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	return nil
})

app.Publish(ctx, "urn:babel:orders:created", map[string]any{"order_id": 1042})
return app.Consume(ctx) // blocks
```

Options: `WithURL` (`amqp://` / `amqps://`), `WithClient` (for tests), `WithMaxWaitTime`.
`attempts` is reconciled to `max(body, delivery-count)` — the AMQP delivery-count is 0-based, so
**no −1** — and the App owns retry by republishing with attempts+1, dead-lettering to
`<queue>.dlq`. Requires **Go 1.23+**. See the
[Apache ActiveMQ Artemis binding](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis).

## Retry, attempts & dead-letter

On a handler error the top-level `attempts` counter is incremented and the message
is re-published to the same queue until `WithMaxAttempts` is reached; after that, if
`WithDeadLetter(true)` is set, it is routed to the dead-letter queue (default: the
source queue + `.dlq`) with a `dead_letter` block — otherwise it is dropped. See the
[wire contract](/docs/spec/1.x/envelope/#optional-dead_letter-block).

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/babelqueue-go/1.x/reliability-and-governance/

The optional, stdlib-only reliability and governance subpackages of the Go core — idempotency (seen-set Wrap + persistent idempotency-postgres / idempotency-redis submodules with Claim), the transactional outbox, DLQ redrive + replay-bypass, GDPR field encryption, and OpenTelemetry traceparent. Each composes with the core, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, the Go core ships **optional, stdlib-only** subpackages that
mechanise the reliability and governance contracts the wire spec defines. None touch the
frozen envelope (`schema_version: 1`); each is a tooling layer over the codec. They are the
Go face of the cross-SDK spec — follow the linked spec page for the full contract.

## Idempotency

`idempotency.Wrap(store, handler)` makes a handler **run at most once per `meta.id`**, even
on at-least-once redelivery.

```go
import "github.com/babelqueue/babelqueue-go/idempotency"

store := idempotency.NewInMemoryStore() // tests / single process
app.Handle("urn:babel:orders:created", idempotency.Wrap(store, handler))
```

For a **fleet**, two persistent stores ship as **separate submodules** (so the core stays
driver-free) — both also expose an atomic `Claim(ctx, id) (bool, error)` to close the
in-flight race:

```go
import (
	idempgres "github.com/babelqueue/babelqueue-go/idempotency-postgres"
	idempredis "github.com/babelqueue/babelqueue-go/idempotency-redis"
)

pg, _ := idempgres.New(ctx, "postgres://…", idempgres.WithTTL(24*time.Hour))
_ = pg.Migrate(ctx) // CREATE TABLE IF NOT EXISTS
// or: rd, _ := idempredis.New("redis://…", idempredis.WithPrefix("bq:idemp:"))
```

See [Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

The `…/outbox` subpackage removes the producer **dual write**: persist the encoded envelope
in the **same DB transaction** as your business row, then a relay publishes the durable rows
verbatim.

```go
import "github.com/babelqueue/babelqueue-go/outbox"

store := outbox.NewInMemoryStore()  // production: a DB-backed outbox.Store
box := outbox.New(store)

// inside YOUR DB transaction, beside the business write — no commit of its own:
env, _ := babelqueue.Make("urn:babel:orders:created", data, babelqueue.WithQueue("orders"))
id, _ := box.Write(env)             // encodes via the frozen codec, calls Store.Save

// later, a relay drains the durable rows to the broker:
relay := outbox.NewRelay(transport, store, outbox.Options{})
res, _ := relay.Drain(ctx, 0)       // res.Published / res.Failed
```

`outbox.Store` is the four-method contract (`Save` / `FetchUnpublished` / `MarkPublished` /
`MarkFailed`); the transaction boundary is **yours**. See
[Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`babelqueue.Redrive(ctx, transport, dlq, RedriveOptions{…})` moves dead-lettered messages
back onto a queue — `dead_letter` removed, `attempts` reset to `0`, everything else preserved
— with `DryRun`, `Select`, `ToQueue` (sandbox) and `Bypass`. The replay guard skips effects
that already fired:

```go
res, _ := babelqueue.Redrive(ctx, transport, "orders.dlq", babelqueue.RedriveOptions{
	ToQueue: "orders.sandbox",
	Bypass:  true, // stamp bq-replay-bypass (header-carrying transport)
})

app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	saveOrder(env) // idempotent core — always runs
	return babelqueue.BypassExternalEffects(ctx, func() error {
		return sendEmail(env) // external effect — skipped on replay
	})
})
```

See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

The `…/gdpr` subpackage encrypts only the `data` leaves a schema marks `x-gdpr-sensitive`, in
place. `gdpr.Protect` / `gdpr.Unprotect` are free functions over the schema's
`SensitivePaths()`; `Cipher` is caller-bound and `AESGCMCipher` is the stdlib reference.
**Validate cleartext** — protect after validation on produce, unprotect before validation on
consume.

```go
import "github.com/babelqueue/babelqueue-go/gdpr"

cipher, _ := gdpr.NewAESGCMCipher(key) // or bind a KMS to gdpr.Cipher
sch, _ := schema.Load("urn:babel:orders:created")

_ = gdpr.Protect(data, sch, cipher)    // producer: encrypt marked leaves
if err := gdpr.Unprotect(data, sch, cipher); errors.Is(err, gdpr.ErrDecrypt) {
	// wrong key / tampered → retry / dead-letter
}
```

See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

The `…/otel` submodule emits `publish <urn>` / `process <urn>` spans and, on a header-carrying
transport, injects/extracts the W3C `traceparent` so a consumer span is a true child of the
producer span — degrading to v0.1 `trace_id` correlation otherwise. All in-tree transports
(in-memory, Redis, AMQP, SQS) carry it. `otel.WrapHandler` / `otel.Publish` are the entry
points. See [Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`schema.Check(provider, urn, data)` (producer guard) / `schema.Wrap(provider, handler)`
(consumer safety net) validate a message's `data` against the JSON Schema registered for its
URN, bridged from a `babelqueue-registry` manifest via `schema.NewDirProvider`. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/babelqueue-node/1.x/introduction/

Introduction — Node.js BabelQueue SDK (1.x).

The **Node.js / TypeScript** SDK is the framework-agnostic **core**: a
wire-envelope codec, contracts and dead-letter helpers that read and write the
exact same strict JSON envelope as every other language in the ecosystem.

It is intentionally **not** a runtime — there is no `publish` call, no worker
loop. You build (or decode) the canonical envelope here and move the bytes with
your own broker client, so it works over Redis, RabbitMQ or anything else.

## What you get

- Canonical, schema-validated JSON envelopes (`schema_version: 1`) — no language
  lock-in.
- A built-in `trace_id` on every message for cross-service correlation.
- URN routing (`urn:babel:<context>:<event>`) on a stable key.
- **Zero runtime dependencies**, shipped as a dual **ESM + CommonJS** package
  with bundled types — bring your own broker client.

Continue to [Installation](/docs/babelqueue-node/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/babelqueue-node/1.x/installation/

Installation — Node.js BabelQueue SDK (1.x).

Install the Node.js SDK (npm `@babelqueue/core`):

```bash
npm install @babelqueue/core
```

## Requirements

- **Node** `>=18`

The package ships dual **ESM + CommonJS** with bundled TypeScript types. The core
is **broker-agnostic with zero dependencies** — you bring your own Redis or
RabbitMQ client to move the encoded bytes; there is no separate BabelQueue
runtime to install.

Next: [Configuration](/docs/babelqueue-node/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/babelqueue-node/1.x/configuration/

Configuration — Node.js BabelQueue SDK (1.x).

The core is a codec, so there's almost nothing to configure — the "wiring" is
your own broker client plus a couple of optional behaviours.

## Envelope options

`EnvelopeCodec.make` takes options for the queue name (written to `meta.queue`)
and trace continuation across a hop:

```ts
import { EnvelopeCodec } from "@babelqueue/core";

const env = EnvelopeCodec.make(
  "urn:babel:orders:created",
  { order_id: 1042 },
  { queue: "orders", traceId: inbound.trace_id }, // omit traceId to mint a fresh one
);
```

## Dead-letter

On failure, wrap the envelope in an additive `dead_letter` block and publish the
copy to your dead-letter queue (e.g. `orders.dlq`):

```ts
import { annotate, EnvelopeCodec } from "@babelqueue/core";

const dlq = annotate(env, "failed", "orders", { attempts: 3, error: "boom" });
// publish EnvelopeCodec.encode(dlq) to the "orders.dlq" queue
```

`annotate` returns a copy — the original envelope is preserved unchanged inside
the dead-lettered message, so any-language consumer can still read it.

## Unknown-URN strategy

For adapters routing inbound messages, `UnknownUrnStrategy` (`FAIL`, `DELETE`,
`RELEASE`, `DEAD_LETTER`) describes what to do with a URN you don't handle.

Next: [Producing messages](/docs/babelqueue-node/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/babelqueue-node/1.x/producing-messages/

Producing messages — Node.js BabelQueue SDK (1.x).

Build the canonical envelope with `EnvelopeCodec.make`, encode it to JSON with
`EnvelopeCodec.encode`, then publish the bytes with your own broker client.

```ts
import { EnvelopeCodec } from "@babelqueue/core";

const env = EnvelopeCodec.make(
  "urn:babel:orders:created",
  { order_id: 1042, amount: 99.90 },
  { queue: "orders" },
);

const body = EnvelopeCodec.encode(env); // compact UTF-8 JSON string
// await redis.rpush("queues:orders", body);
//   /  channel.sendToQueue("orders", Buffer.from(body));
```

CommonJS works too: `const { EnvelopeCodec } = require("@babelqueue/core")`.

`encode` produces the canonical `schema_version: 1` envelope, with `meta.lang`
set to `"node"`:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "node", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/babelqueue-node/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/babelqueue-node/1.x/consuming-messages/

Consuming messages — Node.js BabelQueue SDK (1.x).

Pull the bytes from your broker, `decode` them, gate with `accepts` (a TypeScript
type guard), then switch on the URN — regardless of which language produced the
message. `trace_id` carries the cross-service correlation id.

```ts
import { EnvelopeCodec } from "@babelqueue/core";

// body pulled from your broker (Redis BLPOP, an AMQP message, …)
const incoming = EnvelopeCodec.decode(body);
if (EnvelopeCodec.accepts(incoming)) {
  // `incoming` is now narrowed to a fully-typed Envelope
  switch (EnvelopeCodec.urn(incoming)) {
    case "urn:babel:orders:created":
      console.log(`[${incoming.trace_id}] order received`, incoming.data);
      break;
  }
}
```

That's the whole loop: any producer, any consumer, one schema.

---

# Adapters & transports

Source: https://babelqueue.com/docs/babelqueue-node/1.x/adapters/

The nine npm packages beyond core: the BullMQ and NestJS framework adapters and the Redis, RabbitMQ, Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka and ActiveMQ Artemis transports — canonical-envelope jobs/messages, URN routing, and an injectable publisher.

The `@babelqueue/core` codec is framework-agnostic. Thin packages wire it into common
Node stacks: **`@babelqueue/bullmq`** (BullMQ jobs) and **`@babelqueue/nestjs`** (a NestJS
module built on the BullMQ adapter), plus broker transports — **`@babelqueue/redis`** (Redis),
**`@babelqueue/rabbitmq`** (RabbitMQ), **`@babelqueue/sqs`** (Amazon SQS),
**`@babelqueue/azure-service-bus`** (Azure Service Bus), **`@babelqueue/pulsar`**
(Apache Pulsar), **`@babelqueue/kafka`** (Apache Kafka) and **`@babelqueue/artemis`** (Apache
ActiveMQ Artemis).

## BullMQ — `@babelqueue/bullmq`

```bash
npm install @babelqueue/bullmq bullmq
```

`bullmq ^5` is a peer dependency; `@babelqueue/core` is pulled in for you.

It exports two functions:

- `publish(queue, urn, data, options?) → Promise<string>` — adds a BullMQ job whose
  **name is the URN** and whose **data is the canonical envelope**; returns `meta.id`.
  `options` is `{ traceId?, jobsOptions? }` (`jobsOptions` is BullMQ's own — delay,
  attempts, backoff…).
- `processor(handlers, options?) → (job) => Promise` — a BullMQ processor function
  that validates each envelope, resolves its URN and routes to `handlers[urn]`.
  `options.onUnknownUrn(envelope, job)` handles URNs with no mapped handler.

### Produce

```ts
import { Queue } from "bullmq";
import { publish } from "@babelqueue/bullmq";

const queue = new Queue("orders", { connection: { host: "localhost", port: 6379 } });

const id = await publish(queue, "urn:babel:orders:created", { order_id: 1042 });
```

### Consume

```ts
import { Worker } from "bullmq";
import { processor } from "@babelqueue/bullmq";

new Worker(
  "orders",
  processor(
    {
      "urn:babel:orders:created": async (env, job) => {
        console.log(env.data.order_id, env.trace_id);
      },
    },
    { onUnknownUrn: (env, job) => console.warn("no handler for", job.name) },
  ),
  { connection: { host: "localhost", port: 6379 } },
);
```

A handler is `(envelope, job) => unknown | Promise<unknown>`. A non-conformant
envelope is rejected (BullMQ then retries/fails per its options); an unmapped URN
throws unless `onUnknownUrn` is supplied.

## NestJS — `@babelqueue/nestjs`

```bash
npm install @babelqueue/nestjs @nestjs/common bullmq
```

`@nestjs/common ^10 || ^11` and `bullmq ^5` are peers; it builds on
`@babelqueue/bullmq`.

Register the module and inject the publisher:

```ts
import { Module } from "@nestjs/common";
import { BabelQueueModule } from "@babelqueue/nestjs";

@Module({
  imports: [
    BabelQueueModule.forRoot({
      queue: "orders",
      connection: { host: "localhost", port: 6379 },
    }),
  ],
})
export class AppModule {}
```

```ts
import { Injectable } from "@nestjs/common";
import { BabelQueuePublisher } from "@babelqueue/nestjs";

@Injectable()
export class Orders {
  constructor(private readonly babelQueue: BabelQueuePublisher) {}

  create() {
    return this.babelQueue.publish("urn:babel:orders:created", { order_id: 1042 });
  }
}
```

`forRoot({ queue, connection, queueOptions? })` provides an injectable
`BabelQueuePublisher` (`publish(urn, data, { traceId? }) → Promise<string>`) over the
BullMQ queue. For consuming, build a plain BullMQ `Worker` with the `processor`
re-exported from `@babelqueue/nestjs`:

```ts
import { Worker } from "bullmq";
import { processor } from "@babelqueue/nestjs";

new Worker(
  "orders",
  processor({ "urn:babel:orders:created": async (env) => { /* ... */ } }),
  { connection: { host: "localhost", port: 6379 } },
);
```

## Redis — `@babelqueue/redis`

```bash
npm install @babelqueue/redis ioredis
```

`ioredis` is an optional peer — **you provide the client** (an `ioredis` instance satisfies the
adapter structurally). It implements the [§1 reliable-queue pattern](/docs/spec/1.x/broker-bindings/#redis):
the list element **is** the canonical envelope JSON, byte-for-byte, with **no wrapping** (unlike
`@babelqueue/bullmq`, which uses BullMQ's own job layout) and **no property projection** — Redis
lists carry no native metadata, so routing and tracing read the body directly. Produce is `RPUSH`;
consume reserves the head into a `<queue>:processing` list (`BRPOPLPUSH`, so an in-flight message
survives a crash), routes by URN, and `LREM`s it on success.

### Produce

```ts
import Redis from "ioredis";
import { RedisPublisher } from "@babelqueue/redis";

const client = new Redis("redis://localhost:6379/0");

const id = await RedisPublisher.create(client, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });
```

`publish(urn, data, { traceId? })` returns the message `meta.id`.

### Consume

```ts
import { RedisConsumer, type BabelHandlers } from "@babelqueue/redis";

const handlers: BabelHandlers = {
  "urn:babel:orders:created": (envelope, raw) => {
    console.log(envelope.data.order_id, envelope.trace_id);
  },
};

const consumer = new RedisConsumer(client, "orders", handlers, {
  maxTries: 3, // requeue with attempts+1, then <queue>.dlq
  onError: (err) => console.error(err),
});

await consumer.run(() => true); // reserve → process → LREM, until you stop it
```

A successful handler `LREM`s the element from `<queue>:processing`. A throwing handler requeues the
envelope with `attempts + 1` (at-least-once) up to `maxTries`, then dead-letters to `<queue>.dlq`
with a `dead_letter` block — the body owns the attempt count. Unknown-URN strategy is one of
`fail` / `delete` / `release` / `dead_letter`.

> This is a Node-owned reliable queue. Full parity with Laravel's reserved-sorted-set reservation on
> a *shared* Redis queue is a separate task — for a mixed PHP+Node fleet, prefer a queue this
> consumer owns end-to-end. See the [Redis binding](/docs/spec/1.x/broker-bindings/#redis).

## RabbitMQ — `@babelqueue/rabbitmq`

```bash
npm install @babelqueue/rabbitmq amqplib
```

`amqplib` is an optional peer — **you provide the channel** (an amqplib `Channel` satisfies the
adapter structurally). It implements [§2 of the broker-bindings contract](/docs/spec/1.x/broker-bindings/#rabbitmq-amqp-0-9-1):
the envelope JSON is the message **body**, and the contract fields are projected onto native AMQP
0-9-1 properties so a consumer routes without decoding the body — `type` = URN, `correlation_id` =
`trace_id`, `message_id` = `meta.id`, `app_id` = `babelqueue`, plus the native-typed
`x-schema-version` / `x-source-lang` / `x-attempts` headers (AMQP field-tables carry typed values,
so integers stay integers). Consume is `basic.get` + manual ack (at-least-once).

### Produce

```ts
import amqp from "amqplib";
import { RabbitMQPublisher } from "@babelqueue/rabbitmq";

const conn = await amqp.connect("amqp://guest:guest@localhost:5672/");
const channel = await conn.createChannel();
await channel.assertQueue("orders", { durable: true });

const id = await RabbitMQPublisher.create(channel, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });
```

`publish(urn, data, { traceId? })` returns the message `meta.id`. Messages are persistent
(`delivery_mode = 2`).

### Consume

```ts
import { RabbitMQConsumer, type BabelHandlers } from "@babelqueue/rabbitmq";

const handlers: BabelHandlers = {
  "urn:babel:orders:created": (envelope, message) => {
    console.log(envelope.data.order_id, envelope.trace_id);
  },
};

const consumer = new RabbitMQConsumer(channel, "orders", handlers, {
  maxTries: 3,
  onError: (err) => console.error(err),
});

await consumer.run(() => true); // basic.get → process → ack, until you stop it
```

A successful handler `ack`s the message. A throwing handler republishes the envelope with
`attempts + 1` (at-least-once) up to `maxTries`, then dead-letters to `<queue>.dlq` with a
`dead_letter` block. The consumer routes on `properties.type` (falling back to the body URN).
Unknown-URN strategy is one of `fail` / `delete` / `release` / `dead_letter`. See the
[RabbitMQ binding](/docs/spec/1.x/broker-bindings/#rabbitmq-amqp-0-9-1).

## Amazon SQS — `@babelqueue/sqs`

```bash
npm install @babelqueue/sqs @aws-sdk/client-sqs
```

`@aws-sdk/client-sqs` is an optional peer — **you provide the SQS client** (the aggregated
`SQS` class satisfies the transport structurally). It sends the canonical envelope as the
`MessageBody` with the [§3 `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs),
and consumes by routing each message to a handler by URN.

### Produce

```ts
import { SQS } from "@aws-sdk/client-sqs";
import { SqsPublisher } from "@babelqueue/sqs";

const sqs = new SQS({ region: "eu-central-1" });
const url = "https://sqs.eu-central-1.amazonaws.com/123456789012/orders";

const id = await new SqsPublisher(sqs, url).publish("urn:babel:orders:created", { order_id: 1042 });
```

`publish(urn, data, { traceId? })` returns the message `meta.id`. FIFO queues:
`new SqsPublisher(sqs, url, { fifo: true })` (the queue URL must end in `.fifo`).

### Consume

```ts
import { SqsConsumer } from "@babelqueue/sqs";

const consumer = new SqsConsumer(
  sqs,
  url,
  {
    "urn:babel:orders:created": async (env, message) => {
      console.log(env.data.order_id, env.trace_id);
    },
  },
  { onUnknownUrn: (env, msg) => {}, onError: (err, env, msg) => {} },
);

await consumer.poll(); // receive one batch, route, delete handled; loop this
```

A throwing handler's message is released, never deleted: the consumer calls
`ChangeMessageVisibility` with `releaseDelay` (default 0 s, so SQS redelivers it at once,
at-least-once). Give the queue a native `RedrivePolicy` (`maxReceiveCount` → a DLQ) so a
message that always fails cannot loop forever; `attempts` is reconciled to `ApproximateReceiveCount − 1`. Point the
client's endpoint at LocalStack/ElasticMQ for local testing.

## Azure Service Bus — `@babelqueue/azure-service-bus`

```bash
npm install @babelqueue/azure-service-bus @azure/service-bus
```

`@azure/service-bus` is an optional peer — **you provide the sender/receiver** (a
`ServiceBusSender` / `ServiceBusReceiver` satisfies the adapter structurally). It sends the
canonical envelope as the message `body` with the native §4 projection (`subject` = URN,
`correlationId` = `trace_id`, `messageId` = `meta.id`, plus the `bq-` application
properties), and consumes by routing each message to a handler by URN.

```ts
import { ServiceBusClient } from "@azure/service-bus";
import { AsbPublisher, AsbConsumer } from "@babelqueue/azure-service-bus";

const client = new ServiceBusClient(connectionString); // or (namespace, credential)

// produce
const id = await new AsbPublisher(client.createSender("orders"))
  .publish("urn:babel:orders:created", { order_id: 1042 });

// consume (PeekLock)
const consumer = new AsbConsumer(
  client.createReceiver("orders"),
  {
    "urn:babel:orders:created": async (env, message) => {
      console.log(env.data.order_id, env.trace_id);
    },
  },
  { onError: (err) => console.error(err) },
);
await consumer.run();
```

Delayed delivery: `publish(urn, data, { delayMs: 300000 })` → native
`scheduledEnqueueTimeUtc`. A throwing handler `abandon`s the message (the broker redelivers,
incrementing `deliveryCount`); `attempts` is reconciled to
`max(body.attempts, deliveryCount − 1)`. See the
[Azure Service Bus binding](/docs/spec/1.x/broker-bindings/#azure-service-bus).

## Apache Pulsar — `@babelqueue/pulsar`

```bash
npm install @babelqueue/pulsar pulsar-client
```

`pulsar-client` is an optional peer — **you provide the producer/consumer** (a `Producer` /
`Consumer` satisfies the adapter structurally). It sends the canonical envelope as the
message payload with the §5 property projection (`bq-job` = URN, `bq-trace-id` = `trace_id`,
`bq-message-id` = `meta.id`, plus `bq-schema-version` / `bq-source-lang` / `bq-attempts`,
all string→string), and consumes by routing each message to a handler by URN.

```ts
import Pulsar from "pulsar-client";
import { PulsarPublisher, PulsarConsumer } from "@babelqueue/pulsar";

const client = new Pulsar.Client({ serviceUrl: "pulsar://localhost:6650" });

// produce
const producer = await client.createProducer({ topic: "orders" });
const id = await new PulsarPublisher(producer)
  .publish("urn:babel:orders:created", { order_id: 1042 });

// consume (Shared subscription)
const sub = await client.subscribe({
  topic: "orders",
  subscription: "babelqueue",
  subscriptionType: "Shared",
});
const consumer = new PulsarConsumer(
  sub,
  {
    "urn:babel:orders:created": async (env, message) => {
      console.log(env.data.order_id, env.trace_id);
    },
  },
  { onError: (err) => console.error(err) },
);
await consumer.run();
```

Delayed delivery: `publish(urn, data, { delayMs: 300000 })` → native `deliverAfter`. A
throwing handler `negativeAcknowledge`s the message (the broker redelivers, incrementing
`getRedeliveryCount()`); `attempts` is reconciled to `max(body.attempts, redeliveryCount)` —
the redelivery count is 0-based, so **no −1**. See the
[Apache Pulsar binding](/docs/spec/1.x/broker-bindings/#apache-pulsar).

## Apache Kafka — `@babelqueue/kafka`

```bash
npm install @babelqueue/kafka kafkajs
```

`kafkajs` is an optional peer — **you provide the producer/consumer**. Kafka has no native
ack/delay/DLQ/delivery-counter, so the adapter absorbs all four: the record **value** is the
envelope, the contract fields go to `bq-` headers (`bq-job` routes), the record timestamp
mirrors `meta.created_at`, and **`bq-attempts` is the authoritative attempt counter**. Consume
is **process-then-commit** (manual commit).

```ts
import { Kafka } from "kafkajs";
import { KafkaPublisher, KafkaConsumer, RetryTopics } from "@babelqueue/kafka";

const kafka = new Kafka({ brokers: ["localhost:9092"] });

// produce
const producer = kafka.producer();
await producer.connect();
const id = await KafkaPublisher.create(producer, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });

// consume (manual commit)
const consumer = kafka.consumer({ groupId: "orders-workers" });
await consumer.connect();
await consumer.subscribe({ topic: "orders" });

const retry = new RetryTopics("orders", [5_000, 60_000]); // orders.retry.1/.2 + orders.dlq
const babel = new KafkaConsumer(
  consumer,
  { "urn:babel:orders:created": async (env, message) => { console.log(env.data.order_id, env.trace_id); } },
  { producer, retryTopics: retry, maxTries: 3, onError: (err) => console.error(err) },
);
await babel.run();
```

A throwing handler republishes the envelope to the next `<topic>.retry.<n>` tier with
`bq-attempts + 1`, then commits; at `maxTries` it goes to `<topic>.dlq` with a `dead_letter`
block. A delay with no retry topics throws (Kafka has no native delay). See the
[Apache Kafka binding](/docs/spec/1.x/broker-bindings/#apache-kafka).

## Apache ActiveMQ Artemis — `@babelqueue/artemis`

```bash
npm install @babelqueue/artemis rhea
```

`rhea` is an optional peer — **you provide the sender/receiver**. Artemis speaks **AMQP 1.0**
(not RabbitMQ's 0-9-1), with native settlement, scheduled delivery, a delivery counter and a
dead-letter address. The envelope is the message **body**; the contract fields ride the slots a
JMS peer reads — `x-opt-jms-type` = URN (routes), `correlation-id` = `trace_id`, `creation-time`
= `meta.created_at` — plus the string `bq_` application properties (underscored, since JMS property names must be valid Java identifiers).

```ts
import { Container } from "rhea";
import { ArtemisPublisher, ArtemisConsumer } from "@babelqueue/artemis";

const connection = new Container().connect({ host: "localhost", port: 5672 });

// produce
const sender = connection.open_sender("orders");
const id = await ArtemisPublisher.create(sender, "orders")
  .publish("urn:babel:orders:created", { order_id: 1042 });

// consume (the consumer owns the disposition — autoaccept off)
const receiver = connection.open_receiver({ source: "orders", autoaccept: false, credit_window: 10 });
const dlqSender = connection.open_sender("orders.dlq");

const babel = new ArtemisConsumer(
  { "urn:babel:orders:created": (env, message) => { console.log(env.data.order_id, env.trace_id); } },
  { deadLetterSender: dlqSender, maxTries: 3, onError: (err) => console.error(err) },
);
babel.listen(receiver); // wires receiver.on("message") → accept / release / dead-letter
```

A successful handler `accept`s the message; a throwing handler `release`s it (the broker
redelivers and bumps `delivery-count`); at `maxTries` the envelope goes to `<queue>.dlq` with a
`dead_letter` block. `attempts` reconciles to `max(body, delivery-count)` — the AMQP counter is
0-based, so **no −1**. See the
[Apache ActiveMQ Artemis binding](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis).

Whatever a Node service produces is the canonical envelope, so it is consumed
natively by any other BabelQueue SDK — see the [wire contract](/docs/spec/1.x/envelope/).

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/babelqueue-node/1.x/reliability-and-governance/

The optional, zero-dependency reliability and governance helpers in @babelqueue/core — idempotency (Wrap + seen-set Store), the transactional outbox, DLQ redrive + replay-bypass, GDPR field encryption, and OpenTelemetry traceparent via the /otel subpath. Each composes with the core codec, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, `@babelqueue/core` ships **optional, zero-dependency** helpers that
mechanise the reliability and governance contracts the wire spec defines. None touch the
frozen envelope (`schema_version: 1`); each is a tooling layer over the codec. They are the
Node face of the cross-SDK spec — follow the linked spec page for the full contract.

## Idempotency

`Wrap(store, handler)` makes a handler **run at most once per `meta.id`**, even on
at-least-once redelivery. The `Store` methods may be sync or async, so a Redis/DB backend
drops straight in.

```ts
import { Wrap, InMemoryStore } from "@babelqueue/core";

const store = new InMemoryStore(); // tests / single process
const handler = Wrap(store, async (data, meta) => { await chargeOnce(data); });
```

The in-memory `InMemoryStore` is the reference; for a fleet, implement the `Store` interface
(`seen` / `remember` / `forget`) over a shared backend. See
[Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

`Outbox` / `OutboxRelay` remove the producer **dual write**: persist the encoded envelope in
the **same DB transaction** as your business row, then a relay publishes the durable rows
verbatim.

```ts
import { Outbox, OutboxRelay, InMemoryOutboxStore } from "@babelqueue/core";

const store = new InMemoryOutboxStore(); // production: a DB-backed OutboxStore
const outbox = new Outbox(store);

// inside YOUR DB transaction, beside the business write — no commit of its own:
const id = await outbox.write(envelope);  // encodes via EnvelopeCodec, calls store.save()

// later, a relay drains the durable rows to the broker:
const res = await new OutboxRelay(transport, store).drain(); // res.published / res.failed
```

`OutboxStore` is the four-method async contract (`save` / `fetchUnpublished` /
`markPublished` / `markFailed`); the relay forwards through an `OutboxTransport`. The
transaction boundary is **yours**. See [Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`resetForRedrive(envelope)` resets a dead-lettered envelope for re-publish (`dead_letter`
removed, `attempts` to `0`, everything else preserved); the replay guard skips effects that
already fired on a deliberate replay. The delivered headers are passed explicitly (the same
seam the OTel `wrapHandler` uses):

```ts
import { isReplay, bypassExternalEffects } from "@babelqueue/core";

async function handler(data, meta, headers) {
  saveOrder(data);                                            // idempotent core
  await bypassExternalEffects(headers, () => sendEmail(data)); // skipped on replay
}
```

`RedriveIO.publishWithHeaders` stamps the `bq-replay-bypass` marker on a header-carrying
transport. See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`protect` / `unprotect` encrypt only the `data` leaves a schema marks `x-gdpr-sensitive`, in
place. `Cipher` is caller-bound and `AesGcmCipher` (on the built-in `node:crypto`) is the
reference. **Validate cleartext** — protect after validation on produce, unprotect before
validation on consume.

```ts
import { protect, unprotect, AesGcmCipher } from "@babelqueue/core";

const cipher = new AesGcmCipher(key); // or implement Cipher over a KMS

protect(data, schema, cipher);        // producer: encrypt marked leaves
try {
  unprotect(data, schema, cipher);    // consumer: inverse
} catch (e) {
  // DecryptError → wrong key / tampered → retry / dead-letter
}
```

See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

The `@babelqueue/core/otel` subpath emits `publish <urn>` / `process <urn>` spans and, on a
header-carrying transport, injects/extracts the W3C `traceparent` so a consumer span is a
true child of the producer span — degrading to v0.1 `trace_id` correlation otherwise. Every
`babelqueue-node-adapters` transport now carries the header on its native metadata channel.

```ts
import { wrapHandler, publish } from "@babelqueue/core/otel";
```

`@opentelemetry/api` is the only optional dependency. See
[Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

`validateSchema` (producer guard) / `wrap` (consumer safety net) validate a message's `data`
against the JSON Schema registered for its URN; `MapProvider` embeds schemas in code. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Introduction

Source: https://babelqueue.com/docs/babelqueue-java/1.x/introduction/

Introduction — Java BabelQueue SDK (1.x).

The **Java** SDK is the framework-agnostic **core**: a wire-envelope codec,
contracts and dead-letter helpers that read and write the exact same strict JSON
envelope as every other language in the ecosystem.

It is intentionally **not** a runtime — there is no `publish` call, no worker
loop. You build (or decode) the canonical envelope here and move the bytes with
your own broker client, so it works over Redis, RabbitMQ or anything else.

## What you get

- Canonical, schema-validated JSON envelopes (`schema_version: 1`) — no language
  lock-in.
- A built-in `trace_id` on every message for cross-service correlation.
- URN routing (`urn:babel:<context>:<event>`) on a stable key.
- **Zero dependencies** (pure JDK, with its own minimal JSON codec — no
  Jackson/Gson forced on you) — bring your own broker client.

Continue to [Installation](/docs/babelqueue-java/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/babelqueue-java/1.x/installation/

Installation — Java BabelQueue SDK (1.x).

Add the Java SDK (`com.babelqueue:babelqueue-core`) to your build.

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-core</artifactId>
    <version>1.8.0</version>
</dependency>
```

Gradle:

```kotlin
implementation("com.babelqueue:babelqueue-core:1.8.0")
```

## Requirements

- **Java** `17+`

The core is **broker-agnostic with zero dependencies** (pure JDK). You bring your
own Redis or RabbitMQ client to move the encoded bytes — there is no separate
BabelQueue runtime to install.

Next: [Configuration](/docs/babelqueue-java/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/babelqueue-java/1.x/configuration/

Configuration — Java BabelQueue SDK (1.x).

The core is a codec, so there's almost nothing to configure — the "wiring" is
your own broker client plus a couple of optional behaviours.

## Envelope options

`EnvelopeCodec.make` takes the queue name (written to `meta.queue`) and an
optional inbound trace id to continue across a hop (pass `null` to mint a fresh
one):

```java
import com.babelqueue.*;
import java.util.Map;

Envelope env = EnvelopeCodec.make(
    "urn:babel:orders:created",
    Map.of("order_id", 1042L),
    "orders",
    null); // or inbound.traceId() to continue a trace
```

## Dead-letter

On failure, wrap the envelope in an additive `dead_letter` block and publish the
copy to your dead-letter queue (e.g. `orders.dlq`):

```java
Envelope dlq = DeadLetters.annotate(env, "failed", "orders", 3, "boom", "java.lang.RuntimeException");
// publish EnvelopeCodec.encode(dlq) to the "orders.dlq" queue
```

`DeadLetters.annotate` returns a copy — the original envelope is preserved
unchanged inside the dead-lettered message, so any-language consumer can still
read it.

## Unknown-URN strategy

For adapters routing inbound messages, `UnknownUrnStrategy` (`FAIL`, `DELETE`,
`RELEASE`, `DEAD_LETTER`) describes what to do with a URN you don't handle.

Next: [Producing messages](/docs/babelqueue-java/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/babelqueue-java/1.x/producing-messages/

Producing messages — Java BabelQueue SDK (1.x).

Build the canonical envelope with `EnvelopeCodec.make`, encode it to JSON with
`EnvelopeCodec.encode`, then publish the bytes with your own broker client.

```java
import com.babelqueue.*;
import java.util.Map;

Envelope env = EnvelopeCodec.make(
    "urn:babel:orders:created",
    Map.of("order_id", 1042L, "amount", 99.90),
    "orders",
    null);

String body = EnvelopeCodec.encode(env); // compact UTF-8 JSON
// jedis.rpush("queues:orders", body);
//   /  channel.basicPublish("", "orders", props, body.getBytes(StandardCharsets.UTF_8));
```

`encode` produces the canonical `schema_version: 1` envelope, with `meta.lang`
set to `"java"`:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "java", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/babelqueue-java/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/babelqueue-java/1.x/consuming-messages/

Consuming messages — Java BabelQueue SDK (1.x).

Pull the bytes from your broker, `decode` them, gate with `accepts`, then switch
on the URN — regardless of which language produced the message. `traceId()`
carries the cross-service correlation id.

```java
import com.babelqueue.*;

// body pulled from your broker (a Jedis BLPOP, an AMQP delivery, …)
Envelope in = EnvelopeCodec.decode(body);
if (EnvelopeCodec.accepts(in)) {
    switch (EnvelopeCodec.urn(in)) {
        case "urn:babel:orders:created" ->
            log.info("[{}] order received: {}", in.traceId(), in.data());
        default -> { /* unknown URN */ }
    }
}
```

That's the whole loop: any producer, any consumer, one schema.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/babelqueue-java/1.x/reliability-and-governance/

The optional, zero-dependency reliability and governance helpers in the Java core — idempotency (Idempotent.wrap + seen-set Store), the transactional outbox (com.babelqueue.outbox), DLQ redrive + replay-bypass, GDPR field encryption (com.babelqueue.gdpr, AesGcmCipher), and OpenTelemetry traceparent. Each composes with the core, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, the Java core ships **optional, zero-dependency** helpers that
mechanise the reliability and governance contracts the wire spec defines. None touch the
frozen envelope (`schema_version: 1`); each is a tooling layer over the codec. They are the
Java face of the cross-SDK spec — follow the linked spec page for the full contract.

## Idempotency

`com.babelqueue.idempotency.Idempotent.wrap(store, handler)` makes a handler **run at most
once per `meta.id`**, even on at-least-once redelivery.

```java
import com.babelqueue.idempotency.Idempotent;
import com.babelqueue.idempotency.InMemoryStore;

Store store = new InMemoryStore(); // tests / single process
Handler guarded = Idempotent.wrap(store, handler);
```

The in-memory `InMemoryStore` is the reference; for a fleet, implement the `Store` interface
(`seen` / `remember` / `forget`) over a shared backend (no persistent store ships in core yet
— bring your own). See [Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

The `com.babelqueue.outbox` package removes the producer **dual write**: persist the encoded
envelope in the **same DB transaction** as your business row, then a relay publishes the
durable rows verbatim.

```java
import com.babelqueue.outbox.Outbox;
import com.babelqueue.outbox.OutboxRelay;
import com.babelqueue.outbox.InMemoryOutboxStore;

OutboxStore store = new InMemoryOutboxStore(); // production: a JDBC-backed OutboxStore
Outbox outbox = new Outbox(store);

// inside YOUR DB transaction, beside the business write — no commit of its own:
String id = outbox.write(envelope);            // encodes via EnvelopeCodec, calls store.save()

// later, a relay drains the durable rows to the broker:
OutboxRelayResult res = new OutboxRelay(transport, store).drain(0); // res.published() / res.failed()
```

`OutboxStore` is the four-method interface (`save` / `fetchUnpublished` / `markPublished` /
`markFailed`); the relay forwards through `OutboxTransport.publish(queue, body)`. The
transaction boundary is **yours**. See [Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`com.babelqueue.Redrive.redrive(transport, dlq, options)` moves dead-lettered messages back
onto a queue — `dead_letter` removed, `attempts` reset to `0`, everything else preserved —
with `toQueue` (sandbox), `max`, `dryRun`, `select` and `bypass`. The `Replay` guard skips
effects that already fired on a deliberate replay; the replay flag rides a `ThreadLocal` scope
(`Replay.process(headers, …)`), so `Replay.isReplay()` takes no argument:

```java
import com.babelqueue.Replay;

Replay.process(headers, () -> {
    saveOrder(data);                                    // idempotent core — always runs
    Replay.bypassExternalEffects(() -> sendEmail(data)); // external effect — skipped on replay
});
```

`Redrive` with `bypass(true)` stamps the `bq-replay-bypass` marker through a `HeaderPublisher`
transport. See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`com.babelqueue.gdpr.Gdpr.protect()` / `.unprotect()` encrypt only the `data` leaves a schema
marks `x-gdpr-sensitive`, in place. `Cipher` is caller-bound; `AesGcmCipher` (on the JDK's
`javax.crypto`) is the reference. **Validate cleartext** — protect after validation on produce,
unprotect before validation on consume.

```java
import com.babelqueue.gdpr.Gdpr;
import com.babelqueue.gdpr.AesGcmCipher;

Cipher cipher = new AesGcmCipher(key); // or implement com.babelqueue.gdpr.Cipher over a KMS

Gdpr.protect(data, schema, cipher);    // producer: encrypt marked leaves
try {
    Gdpr.unprotect(data, schema, cipher); // consumer: inverse
} catch (DecryptException e) {
    // wrong key / tampered → retry / dead-letter
}
```

`com.babelqueue.schema.SensitivePaths.of(schema)` exposes the marked leaves directly. See
[GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

`com.babelqueue.otel.Tracing` emits `publish <urn>` / `process <urn>` spans and, on a
header-carrying transport, injects/extracts the W3C `traceparent` so a consumer span is a true
child of the producer span — degrading to v0.1 `trace_id` correlation otherwise.
`Tracing.publish(…, HeaderSender)` injects on produce and the
`Tracing.wrapHandler(tracer, handler, Supplier<Map<String,String>>)` overload extracts on
consume. The Spring AMQP, SQS and Redis transports carry the header. `opentelemetry-api` is the
core's already-optional dependency, so no new dependency is added. See
[Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

The `com.babelqueue.schema` validator (producer guard + consumer wrap, via `MapProvider`)
validates a message's `data` against the JSON Schema registered for its URN. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# Spring Boot adapter

Source: https://babelqueue.com/docs/babelqueue-java/1.x/spring/

The Spring Boot adapter (babelqueue-spring) — a Spring AMQP MessageConverter, an injectable publisher, and auto-configuration that wires the canonical envelope into RabbitTemplate and @RabbitListener.

`com.babelqueue:babelqueue-spring` plugs the Java core into **Spring AMQP**. Its
auto-configuration registers a `MessageConverter` that encodes/decodes the canonical
envelope, so Boot wires it into both the `RabbitTemplate` (producing) and your
`@RabbitListener` containers (consuming) — no manual setup.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-spring</artifactId>
    <version>1.1.0</version>
</dependency>
```

Gradle:

```kotlin
implementation("com.babelqueue:babelqueue-spring:1.1.0")
implementation("org.springframework.boot:spring-boot-starter-amqp")
```

Requirements: **Spring Boot 3**, **Java 17+**, RabbitMQ. The core
(`babelqueue-core`) comes transitively.

## Configuration

```yaml
# application.yaml
babelqueue:
  default-queue: orders   # written to meta.queue; used when no queue is given
```

Auto-configuration exposes one `MessageConverter` bean (`BabelQueueMessageConverter`)
and a `BabelQueuePublisher`. Both **back off** if your app defines its own. Boot's
RabbitMQ auto-config injects the single converter into the `RabbitTemplate` and the
default listener container factory.

## Produce

Inject `BabelQueuePublisher`. Its `publish(...)` methods return the message `meta.id`:

```java
import com.babelqueue.spring.BabelQueuePublisher;
import java.util.Map;
import org.springframework.stereotype.Service;

@Service
class Orders {
    private final BabelQueuePublisher babelQueue;

    Orders(BabelQueuePublisher babelQueue) {
        this.babelQueue = babelQueue;
    }

    void create() {
        babelQueue.publish("urn:babel:orders:created", Map.of("order_id", 1042L), "orders");
    }
}
```

Available overloads: `publish(urn, data)`, `publish(urn, data, queue)`,
`publish(urn, data, queue, traceId)`, and — for a typed message implementing
`com.babelqueue.PolyglotMessage` — `publish(message)` / `publish(message, queue)`.

Each message carries the contract AMQP properties: `type` = URN, `correlationId` =
`trace_id`, `messageId` = `meta.id`, plus `x-attempts` / `x-schema-version` /
`x-source-lang` headers, `application/json` and persistent delivery.

## Consume

A normal `@RabbitListener` receives a decoded core `Envelope`; route by URN with
`EnvelopeCodec.urn(...)`:

```java
import com.babelqueue.Envelope;
import com.babelqueue.EnvelopeCodec;
import org.springframework.amqp.rabbit.annotation.RabbitListener;
import org.springframework.stereotype.Component;

@Component
class OrderListener {
    @RabbitListener(queues = "orders")
    void onMessage(Envelope envelope) {
        if ("urn:babel:orders:created".equals(EnvelopeCodec.urn(envelope))) {
            var orderId = envelope.data().get("order_id");
            // ... use envelope.traceId(), envelope.data()
        }
    }
}
```

Non-conformant messages (missing URN, unsupported `meta.schema_version`, blank
`trace_id`, or missing `data`) are rejected by the converter — route them to a
dead-letter exchange the usual Spring AMQP way.

Because the wire format is the [canonical envelope](/docs/spec/1.x/envelope/), a
message your Spring app consumes may have been produced by any BabelQueue SDK, and
vice-versa.

---

# Redis transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/redis/

The babelqueue-redis transport — a canonical-envelope publisher and a URN-routed consumer over the §1 reliable-queue list pattern, built on Lettuce, on the framework-agnostic core.

`com.babelqueue:babelqueue-redis` is a Redis transport on the Java core, built on the
[Lettuce](https://lettuce.io) client. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as a Redis list element with the
[§1 reliable-queue pattern](/docs/spec/1.x/broker-bindings/#redis), and consumes by routing each
message to a handler by URN — so a message it produces is consumed by any other BabelQueue SDK,
and vice-versa.

Redis lists carry **no native metadata**, so — unlike the SQS, RabbitMQ or Kafka bindings — there
is **no property projection**. The list element **is** the canonical envelope JSON, byte-for-byte,
with no wrapping and no added fields; routing and tracing read the body directly.

## Install

Maven:

```xml
<dependency>
  <groupId>com.babelqueue</groupId>
  <artifactId>babelqueue-redis</artifactId>
  <version>1.1.0</version>
</dependency>
```

Requirements: **Java 17+**. It pulls `babelqueue-core` and `io.lettuce:lettuce-core` transitively.

The consumer reserves with `BLMOVE`, so it needs **Redis 6.2+**. On Redis Cluster, use
hash-tagged queue names; see [Redis compatibility](/docs/spec/1.x/broker-bindings/#redis-compatibility).

## Produce

```java
import com.babelqueue.redis.RedisPublisher;
import io.lettuce.core.RedisClient;
import io.lettuce.core.api.sync.RedisCommands;
import java.util.Map;

RedisClient client = RedisClient.create("redis://localhost:6379");
RedisCommands<String, String> redis = client.connect().sync(); // the command seam

String id = RedisPublisher.create(redis, "orders")
    .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
```

`publish(urn, data)` returns the message `meta.id`; an overload adds a `traceId` to continue a
trace. Produce is `RPUSH <queue> <envelope>`.

## Consume

```java
import com.babelqueue.redis.RedisConsumer;

RedisConsumer consumer = RedisConsumer.builder(redis, "orders")
    .handler("urn:babel:orders:created", (env, body) -> {
        // env.data(), env.traceId(), env.attempts() ...
    })
    .onError((err, env, body) -> log.warn("bad message", err))
    .build();

consumer.run(); // blocking-reserves until the thread is interrupted
```

A reserve atomically moves the head of the queue onto a per-queue `<queue>:processing` list
(`BLMOVE <queue> <queue>:processing LEFT RIGHT`), so an in-flight message survives a crash; a
successful handler `LREM`s it. Retry is **at-least-once**: a throwing handler leaves the element on
the processing list (a recovery sweep can requeue it). The poll loop never stops on a bad message —
observe via `onError` / `onUnknownUrn`. Unknown-URN strategy is one of
`fail` / `delete` / `release` / `dead_letter`.

The command seam is Lettuce's `RedisCommands<String, String>` interface, so the publisher and
consumer unit-test against a mock with no Redis and no network.

## Contract mapping (§1)

| Envelope | Redis |
| :--- | :--- |
| body | the list element — the canonical envelope JSON, **byte-for-byte, no wrapping** |
| `job` (URN) | read from the body and routed consumer-side (Redis lists carry no native metadata) |
| produce | `RPUSH <queue> <envelope>` |
| reserve | `BLMOVE <queue> <queue>:processing LEFT RIGHT` (head → tail; crash-safe in-flight) |
| ack | `LREM <queue>:processing 1 <envelope>` |
| `attempts` | taken from the body unchanged (Redis has no native delivery counter) |

This is a **Java-owned reliable queue**, mirroring the Go runtime's reliable-queue mechanism —
pointed at a queue this SDK owns end-to-end it is a complete, crash-safe transport. Full parity with
Laravel's reserved-sorted-set reservation on a *shared* PHP+Java Redis queue is a separate task (see
broker-bindings §1.4). The envelope is unchanged (`schema_version` stays `1`); Redis is purely
additive.

---

# Amazon SQS transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/amazon-sqs/

The babelqueue-sqs transport — a canonical-envelope publisher and a URN-routed consumer over the AWS SDK for Java v2, on the framework-agnostic core.

`com.babelqueue:babelqueue-sqs` is an Amazon SQS transport on the Java core. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the `MessageBody` with the
[§3 SQS `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs), and consumes by
routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-sqs</artifactId>
    <version>1.2.0</version>
</dependency>
```

Gradle:

```kotlin
implementation("com.babelqueue:babelqueue-sqs:1.2.0")
```

Requirements: **Java 17+**. It pulls `babelqueue-core` and the AWS SDK for Java v2
(`software.amazon.awssdk:sqs`) transitively. You supply the AWS `SqsClient` (an interface),
so its credentials/region come from your standard AWS configuration chain.

## Produce

```java
import com.babelqueue.sqs.SqsPublisher;
import java.util.Map;
import software.amazon.awssdk.services.sqs.SqsClient;

SqsClient sqs = SqsClient.create(); // your AWS config / credentials chain
String url = "https://sqs.eu-central-1.amazonaws.com/123456789012/orders";

String id = SqsPublisher.create(sqs, url)
    .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
```

`publish(urn, data)` returns the message `meta.id`; `publish(urn, data, traceId)`
continues an existing trace. FIFO queues use the builder:
`SqsPublisher.builder(sqs, url).fifo(true).build()` (the queue URL must end in `.fifo`).

## Consume

Register handlers by URN; `poll()` receives one batch, routes each message, and deletes
the ones handled. A throwing handler's message is released, never deleted: the consumer
calls `ChangeMessageVisibility` with the configured release delay (default 0 s, so SQS
redelivers it at once, at-least-once). Give the queue a native `RedrivePolicy`
(`maxReceiveCount` → a DLQ) so a message that always fails cannot loop forever:

```java
import com.babelqueue.sqs.SqsConsumer;

SqsConsumer consumer = SqsConsumer.builder(sqs, url)
    .handler("urn:babel:orders:created", (envelope, message) -> {
        var orderId = envelope.data().get("order_id");
        // ... use envelope.traceId(), envelope.data()
    })
    .onUnknownUrn((envelope, message) -> { /* no handler for this URN */ })
    .onError((error, envelope, message) -> { /* loop never stops */ })
    .build();

while (running) {
    consumer.poll();
}
```

`attempts` is reconciled to `ApproximateReceiveCount − 1` for the handler (never lowering
a runtime-incremented count). Point the `SqsClient`'s endpoint at LocalStack/ElasticMQ for
local testing.

Because the wire format is the [canonical envelope](/docs/spec/1.x/envelope/), the message
a handler receives may have been produced by any BabelQueue SDK.

---

# Azure Service Bus transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/azure-service-bus/

The babelqueue-azure-servicebus transport — a canonical-envelope publisher and a URN-routed consumer over azure-messaging-servicebus, on the framework-agnostic core.

`com.babelqueue:babelqueue-azure-servicebus` is an Azure Service Bus transport on the Java
core. It sends the [canonical envelope](/docs/spec/1.x/envelope/) as the message body with
the [§4 native projection](/docs/spec/1.x/broker-bindings/#azure-service-bus), and consumes
by routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-azure-servicebus</artifactId>
    <version>1.0.0</version>
</dependency>
```

Requirements: **Java 17+**. It pulls `babelqueue-core` and `azure-messaging-servicebus`
transitively. You supply the Azure `ServiceBusClient`.

## Produce

```java
import com.babelqueue.azureservicebus.AsbPublisher;
import com.azure.messaging.servicebus.ServiceBusClientBuilder;
import com.azure.messaging.servicebus.ServiceBusSenderClient;
import java.util.Map;

ServiceBusClientBuilder builder = new ServiceBusClientBuilder().connectionString(cs);
ServiceBusSenderClient sender = builder.sender().queueOrTopicName("orders").buildClient();

String id = AsbPublisher.create(sender)
    .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
```

`publish(urn, data)` returns the message `meta.id`; overloads add a `traceId` and a
relative `Duration delay` (native `ScheduledEnqueueTime`).

## Consume

```java
import com.babelqueue.azureservicebus.AsbConsumer;
import com.azure.messaging.servicebus.ServiceBusReceiverClient;

ServiceBusReceiverClient receiver = builder.receiver().queueName("orders").buildClient();
AsbConsumer consumer = AsbConsumer.builder(receiver)
    .handler("urn:babel:orders:created", (envelope, message) -> {
        // envelope.data(), envelope.traceId(), envelope.attempts() ...
    })
    .onError((error, envelope, message) -> error.printStackTrace())
    .build();

while (running) {
    consumer.poll();
}
```

A throwing handler `abandon`s the message — the broker redelivers it and increments
`DeliveryCount` (at-least-once); at `MaxDeliveryCount` it auto-moves to the native
dead-letter sub-queue. Auth is a connection string or the fully-qualified namespace + a
`TokenCredential` (`DefaultAzureCredential`).

## Contract mapping (§4)

| Envelope | Azure Service Bus |
| :--- | :--- |
| body | `Body` (byte-identical across SDKs) |
| `job` (URN) | `Subject` |
| `trace_id` | `CorrelationId` |
| `meta.id` | `MessageId` |
| `meta.schema_version` | `ApplicationProperties["bq-schema-version"]` |
| `meta.lang` | `ApplicationProperties["bq-source-lang"]` |
| `meta.created_at` | `ApplicationProperties["bq-created-at"]` (ms) |
| `attempts` | `max(body, DeliveryCount − 1)` |
| reserve / ack / retry | PeekLock → `complete` / `abandon` |

The final Service Bus clients are mocked with Mockito 5 — no Azure, no network. The
envelope is unchanged (`schema_version` stays `1`); Azure Service Bus is purely additive.

---

# Apache Pulsar transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/apache-pulsar/

The babelqueue-pulsar transport — a canonical-envelope publisher and a URN-routed consumer over pulsar-client, on the framework-agnostic core.

`com.babelqueue:babelqueue-pulsar` is an Apache Pulsar transport on the Java core. It sends
the [canonical envelope](/docs/spec/1.x/envelope/) as the message payload with the
[§5 property projection](/docs/spec/1.x/broker-bindings/#apache-pulsar), and consumes by
routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-pulsar</artifactId>
    <version>1.0.0</version>
</dependency>
```

Requirements: **Java 17+**. It pulls `babelqueue-core` and `org.apache.pulsar:pulsar-client`
transitively. You supply the Pulsar `Producer` / `Consumer`.

## Produce

```java
import com.babelqueue.pulsar.PulsarPublisher;
import org.apache.pulsar.client.api.*;
import java.util.Map;

PulsarClient client = PulsarClient.builder().serviceUrl("pulsar://localhost:6650").build();
Producer<byte[]> producer = client.newProducer().topic("orders").create();

String id = PulsarPublisher.create(producer)
    .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
```

`publish(urn, data)` returns the message `meta.id`; overloads add a `traceId` and a relative
`Duration delay` (native `deliverAfter`).

## Consume

```java
import com.babelqueue.pulsar.PulsarConsumer;
import org.apache.pulsar.client.api.*;

Consumer<byte[]> sub = client.newConsumer()
    .topic("orders").subscriptionName("babelqueue")
    .subscriptionType(SubscriptionType.Shared).subscribe();

PulsarConsumer consumer = PulsarConsumer.builder(sub)
    .handler("urn:babel:orders:created", (envelope, message) -> {
        // envelope.data(), envelope.traceId(), envelope.attempts() ...
    })
    .onError((error, envelope, message) -> error.printStackTrace())
    .build();

while (running) {
    consumer.poll();
}
```

A throwing handler `negativeAcknowledge`s the message — the broker redelivers it and
increments `getRedeliveryCount()` (at-least-once); with a native `DeadLetterPolicy` it
eventually moves to the cross-language `<queue>.dlq` topic. The consumer routes purely on the
`bq-job` property, so it never decodes a message it cannot handle.

## Contract mapping (§5)

| Envelope | Apache Pulsar |
| :--- | :--- |
| body | message payload (byte-identical across SDKs) |
| `job` (URN) | property `bq-job` (consumer routes on this) |
| `trace_id` | property `bq-trace-id` |
| `meta.id` | property `bq-message-id` |
| `meta.schema_version` | property `bq-schema-version` |
| `meta.lang` | property `bq-source-lang` |
| `meta.created_at` | publish time (mirror; body authoritative) |
| `attempts` | property `bq-attempts`, reconciled to `max(body, getRedeliveryCount())` |
| reserve / ack / retry | `acknowledge` / `negativeAcknowledge` |

Pulsar properties are string→string, so `bq-attempts` carries the contract `attempts` and is
**authoritative**; `getRedeliveryCount()` is 0-based, so the reconciliation maps it directly
with **no −1**. The Pulsar `Producer` / `Consumer` / `Message` interfaces are mocked with
Mockito 5 — no Pulsar, no network. The envelope is unchanged (`schema_version` stays `1`);
Apache Pulsar is purely additive.

---

# Apache Kafka transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/apache-kafka/

The babelqueue-kafka transport — a canonical-envelope publisher and a URN-routed, process-then-commit consumer over kafka-clients, on the framework-agnostic core.

`com.babelqueue:babelqueue-kafka` is an Apache Kafka transport on the Java core. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the record value with the
[§6 header projection](/docs/spec/1.x/broker-bindings/#apache-kafka), and consumes by routing
each record to a handler by URN — so a record it produces is consumed by any other BabelQueue
SDK, and vice-versa. Kafka has no native ack/delay/DLQ/delivery-counter, so the binding
absorbs all four in the transport.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-kafka</artifactId>
    <version>1.0.0</version>
</dependency>
```

Requirements: **Java 17+**. It pulls `babelqueue-core` and `org.apache.kafka:kafka-clients`
transitively. You supply the Kafka `Producer` / `Consumer`.

## Produce

```java
import com.babelqueue.kafka.KafkaPublisher;
import org.apache.kafka.clients.producer.*;
import org.apache.kafka.common.serialization.ByteArraySerializer;
import java.util.Map;

Map<String, Object> cfg = Map.of(
    ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
    ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class,
    ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, ByteArraySerializer.class);

try (Producer<byte[], byte[]> producer = new KafkaProducer<>(cfg)) {
    String id = KafkaPublisher.create(producer, "orders")
        .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
}
```

`publish(urn, data)` returns the message `meta.id`; overloads add a `traceId` and a relative
`Duration delay` (delays require a `RetryTopics` topology, else they raise).

## Consume

```java
import com.babelqueue.kafka.*;
import org.apache.kafka.clients.consumer.*;
import org.apache.kafka.common.serialization.ByteArrayDeserializer;
import java.time.Duration;
import java.util.List;

Consumer<byte[], byte[]> consumer = new KafkaConsumer<>(Map.of(
    ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092",
    ConsumerConfig.GROUP_ID_CONFIG, "orders-workers",
    ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, false,      // manual commit (process-then-commit)
    ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class,
    ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, ByteArrayDeserializer.class));
consumer.subscribe(List.of("orders"));

RetryTopics retry = RetryTopics.forTopic("orders")
    .tier(Duration.ofSeconds(5)).tier(Duration.ofMinutes(1)).build();

com.babelqueue.kafka.KafkaConsumer worker = com.babelqueue.kafka.KafkaConsumer.builder(consumer)
    .producer(producer)
    .retryTopics(retry)
    .maxTries(3)
    .handler("urn:babel:orders:created", (envelope, record) -> {
        // envelope.data(), envelope.traceId(), envelope.attempts() ...
    })
    .build();

worker.run(() -> true); // poll → process → commit
```

A throwing handler republishes the envelope to the next `<topic>.retry.<n>` tier with
`bq-attempts + 1`, then commits; once `maxTries` is reached it goes to `<topic>.dlq` with a
`dead_letter` block. The consumer routes on the `bq-job` header. Unknown-URN strategy is one
of `fail` / `delete` / `release` / `dead_letter`.

## Contract mapping (§6)

| Envelope | Apache Kafka |
| :--- | :--- |
| body | record `value` (byte-identical across SDKs) |
| `job` (URN) | header `bq-job` (consumer routes on this) |
| `trace_id` | header `bq-trace-id` |
| `meta.id` | header `bq-message-id` |
| `meta.schema_version` | header `bq-schema-version` |
| `meta.lang` | header `bq-source-lang` |
| `meta.created_at` | record `timestamp` (Unix ms) |
| `attempts` | header `bq-attempts` (**authoritative**; body is the fallback) |
| reserve / ack | poll → process → **commit offset** (manual) |
| retry / delay | republish to `<topic>.retry.<n>` (`bq-attempts + 1`) |
| dead-letter | `<topic>.dlq` + `dead_letter` block |

The Kafka `Consumer` is mocked with Mockito and the producer with the official `MockProducer`
— no Kafka, no network. The envelope is unchanged (`schema_version` stays `1`); Apache Kafka
is purely additive.

---

# Apache ActiveMQ Artemis transport

Source: https://babelqueue.com/docs/babelqueue-java/1.x/apache-activemq-artemis/

The babelqueue-artemis transport — a canonical-envelope publisher and a URN-routed AMQP 1.0 consumer over JMS (Jakarta Messaging), on the framework-agnostic core.

`com.babelqueue:babelqueue-artemis` is an Apache ActiveMQ Artemis transport on the Java core,
built on **JMS** (Jakarta Messaging 3.x). It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the message body with the
[§7 projection](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis), and consumes by routing
each message to a handler by URN — so a message it produces is consumed by any other BabelQueue
SDK, and vice-versa. Artemis speaks AMQP 1.0, so the other SDKs interoperate over the wire even
though Java uses JMS.

## Install

Maven:

```xml
<dependency>
    <groupId>com.babelqueue</groupId>
    <artifactId>babelqueue-artemis</artifactId>
    <version>1.0.0</version>
</dependency>
```

Requirements: **Java 17+**. It pulls `babelqueue-core` transitively; bring your Artemis JMS
client (`org.apache.activemq:artemis-jakarta-client`), which supplies both the `jakarta.jms` API
and the broker connection.

## Produce

```java
import com.babelqueue.artemis.ArtemisPublisher;
import jakarta.jms.*;
import org.apache.activemq.artemis.jms.client.ActiveMQJMSConnectionFactory;
import java.util.Map;

ConnectionFactory factory = new ActiveMQJMSConnectionFactory("tcp://localhost:61616");
try (JMSContext ctx = factory.createContext("user", "pass")) {
    Session session = ctx.createSession(Session.CLIENT_ACKNOWLEDGE);
    MessageProducer producer = session.createProducer(session.createQueue("orders"));

    String id = ArtemisPublisher.create(session, producer)
        .publish("urn:babel:orders:created", Map.of("order_id", 1042L));
}
```

`publish(urn, data)` returns the message `meta.id`; overloads add a `traceId` and a relative
`Duration delay` (native JMS 2.0 `setDeliveryDelay`).

## Consume

```java
import com.babelqueue.artemis.*;
import jakarta.jms.*;

Session session = connection.createSession(false, Session.CLIENT_ACKNOWLEDGE);
MessageConsumer consumer = session.createConsumer(session.createQueue("orders"));
MessageProducer dlqProducer = session.createProducer(null); // anonymous, for <queue>.dlq

ArtemisConsumer worker = ArtemisConsumer.builder(consumer, session)
    .handler("urn:babel:orders:created", (envelope, message) -> {
        // envelope.data(), envelope.traceId(), envelope.attempts() ...
    })
    .deadLetterQueue(dlqProducer, "orders.dlq")
    .maxTries(3)
    .build();

connection.start();
worker.run(() -> true); // receive → process → acknowledge
```

A successful handler `acknowledge`s the message. A throwing handler leaves it unacknowledged and
`recover`s the session, so the broker redelivers it and bumps `JMSXDeliveryCount`; once `maxTries`
is reached the envelope goes to `<queue>.dlq` with a `dead_letter` block. The consumer routes on
`JMSType`. Unknown-URN strategy is one of `fail` / `delete` / `release` / `dead_letter`.

One message per `poll()` keeps the session-wide `acknowledge()` / `recover()` correct — a JMS
session is single-threaded, so run one `ArtemisConsumer` per thread.

## Contract mapping (§7)

| Envelope | Apache ActiveMQ Artemis (JMS) |
| :--- | :--- |
| body | message body (`TextMessage`, byte-identical across SDKs) |
| `job` (URN) | `JMSType` (consumer routes on this) |
| `trace_id` | `JMSCorrelationID` |
| `meta.id` | `JMSMessageID` (broker-set for JMS; body is authoritative) |
| `meta.schema_version` | property `bq_schema_version` |
| `meta.lang` | property `bq_source_lang` |
| `meta.created_at` | `JMSTimestamp` (Unix ms) |
| `attempts` | `max(body, JMSXDeliveryCount − 1)` (broker counter is 1-based) |
| reserve / ack | `receive` → process → **`acknowledge()`** (CLIENT_ACKNOWLEDGE) |
| retry / delay | `recover()` redelivery · native `setDeliveryDelay` |
| dead-letter | `<queue>.dlq` + `dead_letter` block (alongside the native DLA) |

The JMS interfaces are mocked with Mockito — no Artemis, no network. The envelope is unchanged
(`schema_version` stays `1`); Apache ActiveMQ Artemis is purely additive.

---

# Introduction

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/introduction/

Introduction — .NET BabelQueue SDK (1.x).

The **.NET / C#** SDK is the framework-agnostic **core**: a wire-envelope codec,
contracts and dead-letter helpers that read and write the exact same strict JSON
envelope as every other language in the ecosystem.

It is intentionally **not** a runtime — there is no `publish` call, no worker
loop. You build (or decode) the canonical envelope here and move the bytes with
your own broker client, so it works over Redis, RabbitMQ or anything else.

## What you get

- Canonical, schema-validated JSON envelopes (`schema_version: 1`) — no language
  lock-in.
- A built-in `trace_id` on every message for cross-service correlation.
- URN routing (`urn:babel:<context>:<event>`) on a stable key.
- **Zero dependencies** (in-box `System.Text.Json` only) — bring your own broker
  client.

Continue to [Installation](/docs/babelqueue-dotnet/1.x/installation/).

---

# Installation

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/installation/

Installation — .NET BabelQueue SDK (1.x).

Install the .NET SDK (NuGet `BabelQueue.Core`):

```bash
dotnet add package BabelQueue.Core
```

The package id is `BabelQueue.Core`; the namespace you import is `BabelQueue`.

## Requirements

- **.NET 8**

The core is **broker-agnostic with zero dependencies** (in-box
`System.Text.Json` only). You bring your own Redis or RabbitMQ client to move the
encoded bytes — there is no separate BabelQueue runtime to install.

Next: [Configuration](/docs/babelqueue-dotnet/1.x/configuration/).

---

# Configuration

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/configuration/

Configuration — .NET BabelQueue SDK (1.x).

The core is a codec, so there's almost nothing to configure — the "wiring" is
your own broker client plus a couple of optional behaviours.

## Envelope options

`EnvelopeCodec.Make` takes the queue name (written to `meta.queue`) and an
optional inbound trace id to continue across a hop:

```csharp
using BabelQueue;

var env = EnvelopeCodec.Make(
    "urn:babel:orders:created",
    new Dictionary<string, object?> { ["order_id"] = 1042L },
    queue: "orders",
    traceId: inbound.TraceId); // omit to mint a fresh trace_id
```

## Dead-letter

On failure, wrap the envelope in an additive `dead_letter` block and publish the
copy to your dead-letter queue (e.g. `orders.dlq`):

```csharp
var dlq = DeadLetters.Annotate(env, "failed", "orders", attempts: 3, error: "boom");
// publish EnvelopeCodec.Encode(dlq) to the "orders.dlq" queue
```

`DeadLetters.Annotate` returns a copy — the original envelope is preserved
unchanged inside the dead-lettered message, so any-language consumer can still
read it.

## Unknown-URN strategy

For adapters routing inbound messages, `UnknownUrnStrategy` (`Fail`, `Delete`,
`Release`, `DeadLetter`) describes what to do with a URN you don't handle.

Next: [Producing messages](/docs/babelqueue-dotnet/1.x/producing-messages/).

---

# Producing messages

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/producing-messages/

Producing messages — .NET BabelQueue SDK (1.x).

Build the canonical envelope with `EnvelopeCodec.Make`, encode it to JSON with
`EnvelopeCodec.Encode`, then publish the bytes with your own broker client.

```csharp
using BabelQueue;

var env = EnvelopeCodec.Make(
    "urn:babel:orders:created",
    new Dictionary<string, object?> { ["order_id"] = 1042L, ["amount"] = 99.90 },
    queue: "orders");

string body = EnvelopeCodec.Encode(env); // compact UTF-8 JSON
// await db.ListRightPushAsync("queues:orders", body);
//   /  channel.BasicPublish("", "orders", props, Encoding.UTF8.GetBytes(body));
```

`Encode` produces the canonical `schema_version: 1` envelope, with `meta.lang`
set to `"dotnet"`:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "dotnet", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

Next: [Consuming messages](/docs/babelqueue-dotnet/1.x/consuming-messages/).

---

# Consuming messages

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/consuming-messages/

Consuming messages — .NET BabelQueue SDK (1.x).

Pull the bytes from your broker, `Decode` them, gate with `Accepts`, then switch
on the URN — regardless of which language produced the message. `TraceId` carries
the cross-service correlation id.

```csharp
using BabelQueue;

// body pulled from your broker (a StackExchange.Redis pop, an AMQP delivery, …)
var incoming = EnvelopeCodec.Decode(body);
if (EnvelopeCodec.Accepts(incoming))
{
    switch (EnvelopeCodec.Urn(incoming))
    {
        case "urn:babel:orders:created":
            logger.LogInformation("[{Trace}] order received: {Data}", incoming.TraceId, incoming.Data);
            break;
    }
}
```

That's the whole loop: any producer, any consumer, one schema.

---

# Reliability & governance helpers

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/reliability-and-governance/

The optional, zero-dependency reliability and governance helpers in BabelQueue.Core — idempotency (Idempotency.Wrap + seen-set IIdempotencyStore), the transactional outbox (BabelQueue.Outbox, async), DLQ redrive + replay-bypass, GDPR field encryption (BabelQueue.Gdpr, AesGcmCipher), and OpenTelemetry traceparent (BabelQueue.Tracing, on System.Diagnostics.Activity). Each composes with the core, adds nothing to the frozen envelope, and mirrors the cross-SDK spec.

Beyond produce/consume, `BabelQueue.Core` ships **optional, zero-dependency** helpers that
mechanise the reliability and governance contracts the wire spec defines. None touch the
frozen envelope (`schema_version: 1`); each is a tooling layer over the codec. They are the
.NET face of the cross-SDK spec — follow the linked spec page for the full contract.

## Idempotency

`Idempotency.Wrap(store, handler)` (namespace `BabelQueue`) makes a handler **run at most once
per `meta.id`**, even on at-least-once redelivery. The store contract is **synchronous**:

```csharp
using BabelQueue;

IIdempotencyStore store = new InMemoryStore(); // tests / single process
Handler guarded = Idempotency.Wrap(store, handler);
```

The in-memory `InMemoryStore` is the reference; for a fleet, implement `IIdempotencyStore`
(`Seen` / `Remember` / `Forget`) over a shared backend (no persistent store ships in core yet
— bring your own). See [Idempotency](/docs/spec/1.x/idempotency/) and the
[store deep-dive](/docs/spec/1.x/idempotency-stores/).

## Transactional outbox

The `BabelQueue.Outbox` namespace removes the producer **dual write**: persist the encoded
envelope in the **same DB transaction** as your business row, then a relay publishes the
durable rows verbatim. It is async throughout (`CancellationToken`-threaded):

```csharp
using BabelQueue.Outbox;

IOutboxStore store = new InMemoryOutboxStore(); // production: an ADO.NET-backed IOutboxStore
var outbox = new Outbox(store);

// inside YOUR DB transaction, beside the business write — no commit of its own:
string id = await outbox.WriteAsync(envelope, ct); // encodes via EnvelopeCodec, calls SaveAsync

// later, a relay drains the durable rows to the broker:
OutboxRelayResult res = await new OutboxRelay(publisher, store).DrainAsync(0, ct);
// res.Published / res.Failed
```

`IOutboxStore` is the four-method async contract (`SaveAsync` / `FetchUnpublishedAsync` /
`MarkPublishedAsync` / `MarkFailedAsync`); the relay forwards through the `OutboxPublisher`
delegate (`(body, queue, ct)`). The transaction boundary is **yours**. (Under
`using BabelQueue.Outbox;` the writer type `Outbox` is disambiguated from the same-named
namespace with a `using` alias — the standard C# pattern.) See
[Transactional Outbox](/docs/spec/1.x/outbox/).

## DLQ redrive & replay-bypass

`Redrive.RedriveAsync(transport, dlq, options)` (namespace `BabelQueue`) moves dead-lettered
messages back onto a queue — `dead_letter` removed, `attempts` reset to `0`, everything else
preserved — with `ToQueue` (sandbox), `Max`, `DryRun`, `Select` and `Bypass`. The `Replay`
guard takes the delivered headers explicitly and skips effects that already fired:

```csharp
using BabelQueue;

public async Task Handle(IDictionary<string, object?> data, IReadOnlyDictionary<string, string> headers)
{
    SaveOrder(data);                                                  // idempotent core
    await Replay.BypassExternalEffectsAsync(headers, () => SendEmailAsync(data)); // skipped on replay
}
```

`Redrive` with `Bypass = true` stamps the `bq-replay-bypass` marker through an
`IHeaderPublisher` transport. See [DLQ redrive & replay-bypass](/docs/spec/1.x/redrive-and-replay/).

## GDPR field encryption

`Gdpr.Protect()` / `Gdpr.Unprotect()` (namespace `BabelQueue.Gdpr`) encrypt only the `data`
leaves a schema marks `x-gdpr-sensitive`, in place. `ICipher` is caller-bound; `AesGcmCipher`
(on the in-box `System.Security.Cryptography.AesGcm`) is the reference. **Validate cleartext** —
protect after validation on produce, unprotect before validation on consume.

```csharp
using BabelQueue.Gdpr;

ICipher cipher = new AesGcmCipher(key); // or implement ICipher over a KMS

Gdpr.Protect(data, schema, cipher);     // producer: encrypt marked leaves
try
{
    Gdpr.Unprotect(data, schema, cipher); // consumer: inverse
}
catch (ProtectedFieldException)
{
    // wrong key / tampered → retry / dead-letter
}
```

`BabelQueue.Schema.SchemaSensitivity.SensitivePaths(schema)` exposes the marked leaves
directly. See [GDPR field encryption](/docs/spec/1.x/gdpr-field-encryption/).

## OpenTelemetry (`traceparent`)

`BabelQueue.Tracing.Telemetry` emits `publish <urn>` / `process <urn>` activities and, on a
header-carrying transport, injects/extracts the W3C `traceparent` so a consumer span is a true
child of the producer span — degrading to v0.1 `trace_id` correlation otherwise.
`Telemetry.Wrap(handler, headers)` and the header-aware `Telemetry.PublishAsync(…, headers, …)`
overloads are the entry points; `Traceparent.Inject` / `Traceparent.RemoteParentFromHeaders`
do the W3C work. It is built only on the in-box `System.Diagnostics.Activity`, so the core
stays zero-dependency. The SQS, Redis and MassTransit transports carry the header. See
[Observability](/docs/spec/1.x/observability/).

## Per-URN schema validation

The core validator (producer guard + consumer wrap, via `MapProvider`) validates a message's
`data` against the JSON Schema registered for its URN. See
[Per-URN schema validation](/docs/spec/1.x/schema-validation/).

---

# MassTransit adapter

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/masstransit/

The MassTransit adapter (BabelQueue.MassTransit) — a System.Text.Json converter, an injectable publisher, and bus configuration so MassTransit speaks the canonical envelope.

`BabelQueue.MassTransit` plugs the .NET core into **MassTransit**. It registers a
`System.Text.Json` converter that reads/writes the canonical envelope byte-for-byte,
switches the bus to raw JSON, and adds an ergonomic publisher.

## Install

```bash
dotnet add package BabelQueue.MassTransit
```

Requirements: **.NET 8**, **MassTransit 8+**. `BabelQueue.Core` comes transitively.

## Configuration

Call `UseBabelQueueEnvelopes()` inside your bus configurator and register the
publisher:

```csharp
using BabelQueue.MassTransit;

builder.Services.AddMassTransit(x =>
{
    x.UsingRabbitMq((context, cfg) =>
    {
        cfg.UseBabelQueueEnvelopes();   // canonical envelope on the wire (raw JSON)
        cfg.ConfigureEndpoints(context);
    });
});

builder.Services.AddBabelQueuePublisher(defaultQueue: "orders");
```

`UseBabelQueueEnvelopes()` adds `BabelEnvelopeJsonConverter` to MassTransit's
`SystemTextJsonMessageSerializer.Options` and calls `UseRawJsonSerializer()`, so the
wire body is the bare canonical envelope (no MassTransit wrapper).

## Produce

Inject `BabelQueuePublisher`; `PublishAsync(...)` returns `meta.id`:

```csharp
using BabelQueue.MassTransit;

public class Orders(BabelQueuePublisher babelQueue)
{
    public Task Create() =>
        babelQueue.PublishAsync(
            "urn:babel:orders:created",
            new Dictionary<string, object?> { ["order_id"] = 1042L },
            "orders");
}
```

Overloads: `PublishAsync(urn, data, ct?)` (default queue) and
`PublishAsync(urn, data, queue, traceId?, ct?)`.

## Consume

Implement `IConsumer<Envelope>` (the core type, namespace `BabelQueue`) and route by
URN with `EnvelopeCodec.Urn(...)`:

```csharp
using BabelQueue;
using MassTransit;

public class OrderConsumer : IConsumer<Envelope>
{
    public Task Consume(ConsumeContext<Envelope> context)
    {
        var envelope = context.Message;
        if (!EnvelopeCodec.Accepts(envelope)) return Task.CompletedTask;

        switch (EnvelopeCodec.Urn(envelope))
        {
            case "urn:babel:orders:created":
                // use envelope.Data, envelope.TraceId …
                break;
        }
        return Task.CompletedTask;
    }
}
```

## Standalone converter

The converter also works with plain `System.Text.Json`, no bus required:

```csharp
var options = new JsonSerializerOptions().AddBabelQueueEnvelopes();
string wire = JsonSerializer.Serialize(envelope, options); // == EnvelopeCodec.Encode(envelope)
```

The wire body is the [canonical envelope](/docs/spec/1.x/envelope/) (`meta.lang`
`"dotnet"`), so it interoperates with every other BabelQueue SDK.

---

# Redis transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/redis/

The BabelQueue.Redis transport — a canonical-envelope publisher and a URN-routed consumer over the §1 reliable-queue list pattern, built on StackExchange.Redis, on the framework-agnostic core.

`BabelQueue.Redis` is a Redis transport on the .NET core, built on
[StackExchange.Redis](https://www.nuget.org/packages/StackExchange.Redis). It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as a Redis list element with the
[§1 reliable-queue pattern](/docs/spec/1.x/broker-bindings/#redis), and consumes by routing each
message to a handler by URN — so a message it produces is consumed by any other BabelQueue SDK,
and vice-versa.

Unlike the SQS or RabbitMQ bindings, a Redis list element carries **no native metadata**, so there
is **no header/attribute projection**: the list element **is** the canonical envelope JSON,
byte-for-byte, with no wrapping and no added fields. A consumer in any language pops that same body
and decodes it.

## Install

```bash
dotnet add package BabelQueue.Redis
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and `StackExchange.Redis` transitively.

The consumer reserves with `LMOVE`, so it needs **Redis 6.2+**. On Redis Cluster, use
hash-tagged queue names; see [Redis compatibility](/docs/spec/1.x/broker-bindings/#redis-compatibility).

## Produce

```csharp
using BabelQueue.Redis;
using StackExchange.Redis;

var redis = await ConnectionMultiplexer.ConnectAsync("localhost:6379");
IDatabase db = redis.GetDatabase();

var id = await new RedisPublisher(db, "orders")
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync(urn, data)` returns the message `meta.id`; pass a `traceId` to continue a trace.
Produce is `RPUSH <queue> <envelope>`.

## Consume

```csharp
using BabelQueue;
using BabelQueue.Redis;

var handlers = new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = async (env, rawBody, ct) =>
    {
        // env.Data, env.TraceId, env.Attempts ...
    },
};

var consumer = new RedisConsumer(db, "orders", handlers, new RedisConsumerOptions
{
    OnError = (err, env, raw) => Console.Error.WriteLine(err),
});

await consumer.RunAsync(cancellationToken); // polls until cancelled
```

A reserve atomically moves the head of the queue onto a per-queue `<queue>:processing` list
(`LMOVE <queue> <queue>:processing LEFT RIGHT`), so an in-flight message survives a crash; a
successful handler `LREM`s it. Retry is **at-least-once**: a throwing handler leaves the element on
the processing list. The poll loop never stops on a bad message — observe via `OnError` /
`OnUnknownUrn`. Unknown-URN strategy is one of `fail` / `delete` / `release` / `dead_letter`.

## Contract mapping (§1)

| Envelope | Redis |
| :--- | :--- |
| body | the list element (byte-identical across SDKs, no wrapping) |
| `job` (URN) | read from the decoded body (no native metadata to route on) |
| `trace_id` / `meta.id` / … | read from the decoded body |
| produce | `RPUSH <queue> <envelope>` |
| reserve | `LMOVE <queue> <queue>:processing LEFT RIGHT` (an in-flight message survives a crash) |
| ack | `LREM <queue>:processing 1 <envelope>` |

This is a **.NET-owned reliable queue**, mirroring the Go reference: produce/reserve/ack are
self-consistent and crash-safe on a queue this SDK owns end-to-end. Full parity with Laravel's
reserved-sorted-set reservation on a *shared* PHP+.NET Redis queue is a separate task — see
broker-bindings §1.4. `IDatabase` is an interface, so the publisher and consumer unit-test against a
mock with no Redis and no network. The envelope is unchanged (`schema_version` stays `1`); Redis
support is purely additive.

---

# Amazon SQS transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/amazon-sqs/

The BabelQueue.Sqs transport — a canonical-envelope publisher and a URN-routed consumer over the AWS SDK for .NET, on the framework-agnostic core.

`BabelQueue.Sqs` is an Amazon SQS transport on the .NET core. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the `MessageBody` with the
[§3 SQS `MessageAttributes`](/docs/spec/1.x/broker-bindings/#amazon-sqs), and consumes by
routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

```bash
dotnet add package BabelQueue.Sqs
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and the AWS SDK for .NET
(`AWSSDK.SQS`) transitively. You supply an `IAmazonSQS` client, so its credentials/region
come from your standard AWS configuration.

## Produce

```csharp
using Amazon.SQS;
using BabelQueue.Sqs;

IAmazonSQS sqs = new AmazonSQSClient(); // your AWS config / credentials
var url = "https://sqs.eu-central-1.amazonaws.com/123456789012/orders";

var publisher = new SqsPublisher(sqs, url);
string id = await publisher.PublishAsync(
    "urn:babel:orders:created",
    new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync` returns the message `meta.id`; pass a `traceId` to continue an existing
trace. FIFO queues: `new SqsPublisher(sqs, url, fifo: true)` (the queue URL must end in
`.fifo`).

## Consume

Map URNs to handlers; `PollAsync` receives one batch, routes each message, and deletes the
ones handled. A throwing handler's message is released, never deleted: the consumer
calls `ChangeMessageVisibility` with the configured release delay (default 0 s, so SQS
redelivers it at once, at-least-once). Give the queue a native `RedrivePolicy`
(`maxReceiveCount` → a DLQ) so a message that always fails cannot loop forever:

```csharp
using Amazon.SQS;
using BabelQueue;
using BabelQueue.Sqs;

var handlers = new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = (envelope, message, ct) =>
    {
        var orderId = envelope.Data["order_id"];
        // ... use envelope.TraceId, envelope.Data
        return Task.CompletedTask;
    },
};

var consumer = new SqsConsumer(sqs, url, handlers, new SqsConsumerOptions
{
    OnUnknownUrn = (envelope, message, ct) => Task.CompletedTask, // no handler for this URN
    OnError = (error, envelope, message) => { /* loop never stops */ },
});

while (running)
{
    await consumer.PollAsync();
}
```

A handler is `BabelHandler` — `delegate Task BabelHandler(Envelope, Message, CancellationToken)`.
`attempts` is reconciled to `ApproximateReceiveCount − 1` for the handler (never lowering a
runtime-incremented count). Point the `AmazonSQSClient`'s endpoint at LocalStack/ElasticMQ
for local testing.

Because the wire format is the [canonical envelope](/docs/spec/1.x/envelope/), the message a
handler receives may have been produced by any BabelQueue SDK.

---

# Azure Service Bus transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/azure-service-bus/

The BabelQueue.AzureServiceBus transport — a canonical-envelope publisher and a URN-routed consumer over Azure.Messaging.ServiceBus, on the framework-agnostic core.

`BabelQueue.AzureServiceBus` is an Azure Service Bus transport on the .NET core. It sends
the [canonical envelope](/docs/spec/1.x/envelope/) as the `Body` with the
[§4 native projection](/docs/spec/1.x/broker-bindings/#azure-service-bus), and consumes by
routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

```bash
dotnet add package BabelQueue.AzureServiceBus
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and `Azure.Messaging.ServiceBus`
transitively.

## Produce

```csharp
using Azure.Messaging.ServiceBus;
using BabelQueue.AzureServiceBus;

await using var client = new ServiceBusClient("<connection-string>"); // or a namespace + TokenCredential

var sender = client.CreateSender("orders");
var id = await new AsbPublisher(sender)
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync` returns the message `meta.id`; pass a `traceId` to continue a trace, or a
`delay` (`TimeSpan`) to schedule native delayed delivery (`ScheduledEnqueueTime`).

## Consume

```csharp
var receiver = client.CreateReceiver("orders"); // PeekLock
var handlers = new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = (envelope, message, ct) =>
    {
        // envelope.Data, envelope.TraceId, envelope.Attempts ...
        return Task.CompletedTask;
    },
};
var consumer = new AsbConsumer(receiver, handlers, new AsbConsumerOptions
{
    OnError = (err, env, msg) => Console.Error.WriteLine(err),
});
await consumer.RunAsync(cancellationToken);
```

A throwing handler `Abandon`s the message — the broker redelivers it and increments
`DeliveryCount` (at-least-once); at `MaxDeliveryCount` it auto-moves to the native
dead-letter sub-queue. Auth is a connection string or the fully-qualified namespace + a
`TokenCredential` (`DefaultAzureCredential`).

## Contract mapping (§4)

| Envelope | Azure Service Bus |
| :--- | :--- |
| body | `Body` (byte-identical across SDKs) |
| `job` (URN) | `Subject` |
| `trace_id` | `CorrelationId` |
| `meta.id` | `MessageId` |
| `meta.schema_version` | `ApplicationProperties["bq-schema-version"]` |
| `meta.lang` | `ApplicationProperties["bq-source-lang"]` |
| `meta.created_at` | `ApplicationProperties["bq-created-at"]` (ms) |
| `attempts` | `max(body, DeliveryCount − 1)` |
| reserve / ack / retry | PeekLock → `Complete` / `Abandon` |

`ServiceBusSender` / `ServiceBusReceiver` are mockable, so the unit tests use Moq +
`ServiceBusModelFactory` — no Azure, no network. The envelope is unchanged
(`schema_version` stays `1`); Azure Service Bus is purely additive.

---

# Apache Pulsar transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/apache-pulsar/

The BabelQueue.Pulsar transport — a canonical-envelope publisher and a URN-routed consumer over DotPulsar, on the framework-agnostic core.

`BabelQueue.Pulsar` is an Apache Pulsar transport on the .NET core. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the message payload with the
[§5 property projection](/docs/spec/1.x/broker-bindings/#apache-pulsar), and consumes by
routing each message to a handler by URN — so a message it produces is consumed by any
other BabelQueue SDK, and vice-versa.

## Install

```bash
dotnet add package BabelQueue.Pulsar
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and `DotPulsar` (a pure-C# Pulsar
client) transitively.

## Produce

```csharp
using DotPulsar;
using DotPulsar.Extensions;
using BabelQueue.Pulsar;

await using var client = PulsarClient.Builder()
    .ServiceUrl(new Uri("pulsar://localhost:6650")).Build();

await using var producer = client.NewProducer(Schema.ByteArray).Topic("orders").Create();
var id = await new PulsarPublisher(producer)
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync` returns the message `meta.id`; pass a `traceId` to continue a trace, or a
`delay` (`TimeSpan`) to schedule native delayed delivery (`DeliverAtTime`).

## Consume

```csharp
await using var consumer = client.NewConsumer(Schema.ByteArray)
    .Topic("orders").SubscriptionName("babelqueue")
    .SubscriptionType(SubscriptionType.Shared).Create();

var handlers = new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = (envelope, message, ct) =>
    {
        // envelope.Data, envelope.TraceId, envelope.Attempts ...
        return Task.CompletedTask;
    },
};
var babel = new PulsarConsumer(consumer, handlers, new PulsarConsumerOptions
{
    OnError = (err, env, msg) => Console.Error.WriteLine(err),
});
await babel.RunAsync(cancellationToken);
```

A throwing handler `negativeAcknowledge`s the message — the broker redelivers it and
increments `RedeliveryCount` (at-least-once); with a native `DeadLetterPolicy` it eventually
moves to the cross-language `<queue>.dlq` topic. The consumer routes purely on the `bq-job`
property, so it never decodes a message it cannot handle.

## Contract mapping (§5)

| Envelope | Apache Pulsar |
| :--- | :--- |
| body | message payload (byte-identical across SDKs) |
| `job` (URN) | property `bq-job` (consumer routes on this) |
| `trace_id` | property `bq-trace-id` |
| `meta.id` | property `bq-message-id` |
| `meta.schema_version` | property `bq-schema-version` |
| `meta.lang` | property `bq-source-lang` |
| `meta.created_at` | `PublishTime` (mirror; body authoritative) |
| `attempts` | property `bq-attempts`, reconciled to `max(body, RedeliveryCount)` |
| reserve / ack / retry | `Acknowledge` / redeliver |

Pulsar properties are string→string, so `bq-attempts` carries the contract `attempts` and is
**authoritative**; `RedeliveryCount` is 0-based, so the reconciliation maps it directly with
**no −1**. The DotPulsar `IProducer` / `IConsumer` / `IMessage` interfaces are mockable, so
the unit tests use Moq — no Pulsar, no network. The envelope is unchanged (`schema_version`
stays `1`); Apache Pulsar is purely additive.

---

# Apache Kafka transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/apache-kafka/

The BabelQueue.Kafka transport — a canonical-envelope publisher and a URN-routed, process-then-commit consumer over Confluent.Kafka, on the framework-agnostic core.

`BabelQueue.Kafka` is an Apache Kafka transport on the .NET core. It sends the
[canonical envelope](/docs/spec/1.x/envelope/) as the record value with the
[§6 header projection](/docs/spec/1.x/broker-bindings/#apache-kafka), and consumes by routing
each record to a handler by URN — so a record it produces is consumed by any other BabelQueue
SDK, and vice-versa. Kafka has no native ack/delay/DLQ/delivery-counter, so the binding
absorbs all four in the transport.

## Install

```bash
dotnet add package BabelQueue.Kafka
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and `Confluent.Kafka` transitively.

## Produce

```csharp
using Confluent.Kafka;
using BabelQueue.Kafka;

using var producer = new ProducerBuilder<byte[], byte[]>(
    new ProducerConfig { BootstrapServers = "localhost:9092" }).Build();

var id = await KafkaPublisher.Create(producer, "orders")
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync` returns the message `meta.id`; pass a `traceId` to continue a trace. A `delay`
(`TimeSpan`) requires a retry topology (`KafkaPublisher.Create(producer, retryTopics)`) and
routes to the matching tier; on a plain publisher a delay raises `BabelQueueException`.

## Consume

```csharp
using var consumer = new ConsumerBuilder<byte[], byte[]>(new ConsumerConfig
{
    BootstrapServers = "localhost:9092",
    GroupId = "orders-workers",
    EnableAutoCommit = false,            // manual commit is required (process-then-commit)
    AutoOffsetReset = AutoOffsetReset.Earliest,
}).Build();
consumer.Subscribe("orders");

var retry = RetryTopics.ForTopic("orders")
    .Tier(TimeSpan.FromSeconds(5)).Tier(TimeSpan.FromMinutes(1)).Build(); // .retry.1/.2 + orders.dlq

var worker = new KafkaConsumer(consumer, new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = (env, result, ct) =>
    {
        // env.Data, env.TraceId, env.Attempts ...
        return Task.CompletedTask;
    },
}, new KafkaConsumerOptions { Producer = producer, RetryTopics = retry, MaxTries = 3 });

await worker.RunAsync(cancellationToken); // consume → process → commit
```

A throwing handler republishes the envelope to the next `<topic>.retry.<n>` tier with
`bq-attempts + 1`, then commits; once `MaxTries` is reached it goes to `<topic>.dlq` with a
`dead_letter` block. The consumer routes on the `bq-job` header. Unknown-URN strategy is one
of `fail` / `delete` / `release` / `dead_letter`.

## Contract mapping (§6)

| Envelope | Apache Kafka |
| :--- | :--- |
| body | record `value` (byte-identical across SDKs) |
| `job` (URN) | header `bq-job` (consumer routes on this) |
| `trace_id` | header `bq-trace-id` |
| `meta.id` | header `bq-message-id` |
| `meta.schema_version` | header `bq-schema-version` |
| `meta.lang` | header `bq-source-lang` |
| `meta.created_at` | record `Timestamp` (Unix ms) |
| `attempts` | header `bq-attempts` (**authoritative**; body is the fallback) |
| reserve / ack | consume → process → **commit offset** (manual) |
| retry / delay | republish to `<topic>.retry.<n>` (`bq-attempts + 1`) |
| dead-letter | `<topic>.dlq` + `dead_letter` block |

The `IProducer` / `IConsumer` interfaces are mockable, so the unit tests use Moq — no Kafka,
no network. The envelope is unchanged (`schema_version` stays `1`); Apache Kafka is purely
additive.

---

# Apache ActiveMQ Artemis transport

Source: https://babelqueue.com/docs/babelqueue-dotnet/1.x/apache-activemq-artemis/

The BabelQueue.Artemis transport — a canonical-envelope publisher and a URN-routed AMQP 1.0 consumer over AMQPNetLite, on the framework-agnostic core.

`BabelQueue.Artemis` is an Apache ActiveMQ Artemis transport on the .NET core, built on **AMQP
1.0** (AMQPNetLite). It sends the [canonical envelope](/docs/spec/1.x/envelope/) as the message
body with the [§7 projection](/docs/spec/1.x/broker-bindings/#apache-activemq-artemis), and
consumes by routing each message to a handler by URN — so a message it produces is consumed by
any other BabelQueue SDK, and vice-versa. Artemis gives the binding native settlement, scheduled
delivery, a delivery counter and a dead-letter address, so it maps onto them rather than
re-implementing.

## Install

```bash
dotnet add package BabelQueue.Artemis
```

Requirements: **.NET 8**. It pulls `BabelQueue.Core` and `AMQPNetLite.Core` transitively.

## Produce

```csharp
using Amqp;
using BabelQueue.Artemis;

var connection = new Connection(new Address("amqp://user:pass@localhost:5672"));
var session = new Session(connection);
var sender = new AmqpSenderLink(new SenderLink(session, "orders-sender", "orders"), "orders");

var id = await new ArtemisPublisher(sender)
    .PublishAsync("urn:babel:orders:created", new Dictionary<string, object?> { ["order_id"] = 1042 });
```

`PublishAsync` returns the message `meta.id`; pass a `traceId` to continue a trace, or a `delay`
(`TimeSpan`) for native AMQP scheduled delivery.

## Consume

```csharp
using Amqp;
using BabelQueue;
using BabelQueue.Artemis;

var receiver = new AmqpReceiverLink(new ReceiverLink(session, "orders-worker", "orders"));
var dlq = new AmqpSenderLink(new SenderLink(session, "dlq-sender", "orders.dlq"), "orders.dlq");

var handlers = new Dictionary<string, BabelHandler>
{
    ["urn:babel:orders:created"] = (envelope, message, ct) =>
    {
        // envelope.Data, envelope.TraceId, envelope.Attempts ...
        return Task.CompletedTask;
    },
};

var options = new ArtemisConsumerOptions { DeadLetterSender = dlq, MaxTries = 3 };
var worker = new ArtemisConsumer(receiver, handlers, options);
await worker.RunAsync(cancellationToken); // receive → process → Accept
```

A successful handler `Accept`s the message. A throwing handler `Release`s it (the broker
redelivers and bumps `delivery-count`); once `MaxTries` is reached the envelope goes to
`<queue>.dlq` with a `dead_letter` block. The consumer routes on the `x-opt-jms-type` annotation
(falling back to the body URN). Unknown-URN strategy is one of `fail` / `delete` / `release` /
`dead_letter`.

`AmqpSenderLink` / `AmqpReceiverLink` wrap the AMQPNetLite links; the publisher and consumer
depend on the `IAmqpSender` / `IAmqpReceiver` seams, so they unit-test against mocks with no
broker.

## Contract mapping (§7)

| Envelope | Apache ActiveMQ Artemis (AMQP 1.0) |
| :--- | :--- |
| body | message body (byte-identical across SDKs) |
| `job` (URN) | `x-opt-jms-type` annotation → JMSType (consumer routes on this) |
| `trace_id` | `correlation-id` → JMSCorrelationID |
| `meta.created_at` | `creation-time` → JMSTimestamp (Unix ms) |
| `meta.schema_version` | application property `bq_schema_version` |
| `meta.lang` | application property `bq_source_lang` |
| `attempts` | `max(body, delivery-count)` (AMQP counter is 0-based) |
| reserve / ack | `Receive` → process → **`Accept`** |
| retry / delay | `Release` redelivery · native `x-opt-delivery-time` |
| dead-letter | `<queue>.dlq` + `dead_letter` block (alongside the native DLA) |

The AMQP send/receive seams are mocked with Moq — no Artemis, no network. The envelope is
unchanged (`schema_version` stays `1`); Apache ActiveMQ Artemis is purely additive.

---

# Cross-language: Python → Go over Redis

Source: https://babelqueue.com/docs/examples/1.x/cross-language-redis/

A runnable demo — a Python service produces the canonical BabelQueue envelope and a Go service consumes it from the same Redis queue.

The whole point of BabelQueue: a message produced in **one** language is consumed
natively in **another**, over the broker you already run, using one strict JSON
envelope. This demo proves it end-to-end — a **Python** *or* **PHP** producer and
a **Go** consumer sharing a Redis queue. The consumer never knows which language
wrote the message; it only sees the canonical envelope and `meta.lang`.

Every SDK uses the identical reliable-queue pattern (`RPUSH` to produce, `BLMOVE`
to a `:processing` list to reserve, `LREM` to ack), so they interoperate on the
plain `orders` Redis list. Full runnable sources:
**[github.com/BabelQueue/babelqueue-examples](https://github.com/BabelQueue/babelqueue-examples)**.

## Producer — Python

```python
# pip install "babelqueue[redis]"
from babelqueue import BabelQueue

app = BabelQueue("redis://localhost:6379/0", queue="orders")
app.publish("urn:babel:orders:created", {"order_id": 1042, "amount": 99.90, "currency": "USD"})
app.publish("urn:babel:catalog:item.indexed", {"sku": "WIDGET-1", "title": "Café Widget ☕"})
```

## Producer — PHP (same queue, no framework)

The framework-less core builds the envelope and the bundled `RedisTransport` does
a plain `RPUSH` onto the identical `orders` list — so the Go consumer below reads
PHP-produced messages without changing a line.

```php
// composer require babelqueue/php-sdk predis/predis
use BabelQueue\Codec\EnvelopeCodec;
use BabelQueue\Transport\RedisTransport;

$transport = new RedisTransport(new Predis\Client("redis://localhost:6379/0"), "orders");

$envelope = EnvelopeCodec::fromJob($job, "orders"); // $job: a PolyglotJob (URN + toPayload())
$transport->publish(EnvelopeCodec::encode($envelope), "orders");
```

## Consumer — Go

```go
// go get github.com/babelqueue/babelqueue-go/redis
import (
    babelqueue "github.com/babelqueue/babelqueue-go"
    bqredis "github.com/babelqueue/babelqueue-go/redis"
)

tr, _ := bqredis.New("redis://localhost:6379/0")
app := babelqueue.NewApp(tr, babelqueue.WithDefaultQueue("orders"))

app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
    fmt.Printf("order %v (produced by %q)\n", env.Data["order_id"], env.Meta.Lang)
    return nil
})
app.Consume(context.Background()) // routes by URN, forever
```

## The proof

Run either producer, then the Go consumer. The Go program prints messages it
never produced — **`produced by "php"`** (or `"python"`) — with the data,
`trace_id` and even unicode intact:

```text
[go] order created  id=1042 amount=99.9 USD  trace=1d3b…  (produced by "php")
[go] order created  id=1043 amount=12.5 EUR  trace=79c3…  (produced by "php")
[go] item indexed   sku=WIDGET-1 title="Café Widget ☕"  (produced by "php")
[go] processed 3 message(s) — same envelope, different language.
```

The envelope on the wire is the canonical shape every SDK emits:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "…",
  "data": { "order_id": 1042, "amount": 99.9, "currency": "USD" },
  "meta": { "id": "…", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

## Swap the ends

The queue carries the canonical envelope, so any SDK can be on either side — a Go
producer (`app.Publish(...)`) and a Python consumer (`@app.handler(...)` + `app.run()`)
work the same way. PHP (framework-less `RedisTransport`, or Laravel/Symfony on a
`babelqueue-*` connection), Node, Java and .NET read and write the identical
envelope on their own transports.

---

# BabelQueue vs Kafka, gRPC, and the hand-rolled bridge

Source: https://babelqueue.com/blog/babelqueue-vs-kafka-grpc-bridge/

Four ways to make queue messages cross languages, compared honestly — and why changing only the serialization beats new infrastructure or losing your framework.

A PHP app puts a job on a queue, and a Go service needs to read it — but the bytes are a PHP object graph only PHP understands. There are four common ways out of that, and they differ enormously in cost: BabelQueue changes only the serialization, while the others ask you to give up your framework, stand up new infrastructure, or own a bridge service forever.

This post compares all four on the criteria that decide an adoption: the broker you keep, the infrastructure you add, whether any language can read the message, whether you keep your framework's worker, tracing, and dependency weight. We'll be specific about where Kafka and gRPC genuinely belong, because they do.

## The four paths

The problem is narrow: a message produced in one language has to be consumed in another, and your queue serializer doesn't speak across the boundary. Laravel's native queue, for example, serializes jobs with PHP's `serialize()` — a PHP-object graph that only PHP can read ([wire contract §1](/docs/spec/1.x/envelope/)). Four answers:

1. **BabelQueue** — keep the broker and the worker, change only the serialization to a strict JSON envelope.
2. **A language-native format** — stay on `serialize()` (or Python pickle, Java serialization). Fine until a second language shows up.
3. **Kafka or gRPC** — adopt a new transport and contract built for cross-language work.
4. **A hand-rolled bridge** — a service that reads one format and re-publishes another.

## How they compare

The criteria below are the ones the [homepage comparison](/#compare) uses, because they map to real operational cost — not feature checklists.

| Criterion | BabelQueue | Native format (`serialize()`) | Kafka / gRPC rewrite | Hand-rolled bridge |
| :--- | :---: | :---: | :---: | :---: |
| Keep your Redis / RabbitMQ broker | ✓ | ✓ | ✗ | ✓ |
| No new infrastructure or sidecar | ✓ | ✓ | ✗ | ~ |
| Consumed natively in any language | ✓ | ✗ | ✓ | ✓ |
| Drop-in — reuse your framework's worker | ✓ | ✓ | ✗ | ✗ |
| Built-in cross-service `trace_id` | ✓ | ✗ | ~ | ✗ |
| Zero heavy dependencies | ✓ | ✓ | ✗ | ~ |

A `~` means "possible, but it's on you." Below is what each column actually costs.

### 1. BabelQueue — change only the serialization

BabelQueue replaces the language-specific serializer with one strict JSON envelope and routes by a URN instead of a class name. The broker stays (Redis or RabbitMQ), the worker stays, and any of the six SDKs reads the same bytes:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

You publish by URN and your existing worker dispatches it:

```php
BabelQueue::publish('urn:babel:orders:created', ['order_id' => 1042]);
```

A Go worker consumes the identical envelope, routing on the `urn:babel:orders:created` string without sharing a type with the producer. `trace_id` is part of the contract — generated by the first producer and forwarded unchanged across every hop ([wire contract §3](/docs/spec/1.x/envelope/)) — so end-to-end tracing needs no extra plumbing. The codec adds well under 2% over the plain-JSON serialization a publisher already pays, and it talks straight to native drivers with nothing in between.

The trade-off is scope. BabelQueue is a queue serialization standard, not a streaming platform. It does not give you log replay, partitioned ordering, or consumer-group rebalancing. If you need those, you need Kafka — see below.

### 2. A language-native format — fine until it isn't

Staying on `serialize()` is the cheapest option as long as every producer and consumer is the same language. No new dependency, no new infrastructure, your worker untouched. This is the right call for a single-language system, and switching away from it buys you nothing there.

It fails on exactly one axis, and it's the one that started this: a second language cannot read the bytes. A `serialize()`-encoded job, a Python pickle, or a Java `ObjectOutputStream` blob is a language-specific object graph. There's no honest "consume natively in any language" story, and no shared `trace_id` either, because the format carries whatever the runtime decided to embed. The moment a second language appears, this path is over.

### 3. Kafka or gRPC — the right tool for a different job

Kafka and gRPC are genuinely cross-language, and for the problems they target they are the correct choice — this is not a strawman.

**Kafka** is a distributed log. Reach for it when you need a durable, replayable event stream, high-throughput partitioned ordering, multiple independent consumer groups reading the same topic at their own pace, or stream processing. None of that is a queue-serialization problem, and BabelQueue does not pretend to solve it.

**gRPC** is for typed request/response (and streaming) RPC between services, with code-generated stubs from a shared `.proto`. When two services need a synchronous, strongly-typed contract, gRPC is excellent.

The cost shows up when you adopt either *only* to make queue messages cross languages. You're adding new infrastructure — a Kafka cluster with its broker fleet, ZooKeeper or KRaft, and the operational load of running it, or for gRPC a schema registry and generated-stub build step. Your Redis or RabbitMQ broker doesn't carry the load anymore, so "keep your broker" is a no. You leave your framework's queue worker behind and rewrite producers and consumers against a new client and a new dispatch model, so "drop-in" is a no. gRPC carries no cross-service `trace_id` by default — you add OpenTelemetry interceptors to get it, hence the `~`. And the client libraries are heavyweight: the Kafka and gRPC stacks are real dependencies, not a stdlib JSON codec.

If your actual need is a replayable event log or typed RPC, that cost is worth it. If your need is "let Go read the job PHP enqueued," it's a rewrite to solve a serialization problem.

### 4. A hand-rolled bridge — a service you now own forever

The bridge is a service that subscribes to messages in one format and re-publishes them in another — a translator sitting between producer and consumer. It can keep your broker and it does deliver cross-language consumption, so two columns are green.

Then the bill arrives. It's an extra hop and an extra deployable: a new process to run, monitor, scale, and page on, which is why "no new infrastructure" is a `~` — you've added a sidecar in all but name. Your downstream consumer no longer uses your framework's worker; it speaks the bridge's bespoke format, so "drop-in" is a no. The bridge is the single point where every message is reserialized, so it's the single point of failure and the single place a schema mismatch corrupts traffic silently. There's no built-in `trace_id` unless you build and preserve one through the translation yourself. And the format is bespoke: undocumented, untested across languages, and maintained by exactly one team — yours.

A bridge is a reasonable stopgap for a one-off legacy integration. As the standing answer to polyglot queues, it's a custom standard with one maintainer and no conformance suite — which is the thing BabelQueue exists to replace.

## When to pick what

Be honest about the actual need, not the most impressive option:

- **One language, one queue.** Stay on your native format. Adopting anything cross-language buys you nothing.
- **Polyglot queue jobs over the broker you already run.** BabelQueue. You change the serialization, keep Redis or RabbitMQ, keep your worker, and get a frozen envelope with `trace_id` built in.
- **A durable, replayable event stream or partitioned ordering.** Kafka. That's its job, and BabelQueue does not do it.
- **Synchronous, strongly-typed service-to-service calls.** gRPC.
- **A short-lived legacy translation you'll delete.** A small bridge is fine. Don't make it your platform.

The distinction that matters: Kafka and gRPC change your *infrastructure and your code*; a bridge adds a *service you maintain*; a native format *doesn't cross languages at all*. BabelQueue changes only the *serialization* — and the bytes are a frozen, conformance-tested standard ([§9](/docs/spec/1.x/envelope/)) shared by six SDKs, not a format one team invented.

All six SDKs are at 1.0 (GA) — PHP, Python, Go, Node.js, Java, and .NET, over Redis or RabbitMQ. Pick your stack on the [ecosystem](/#ecosystem), or read the [wire contract](/docs/spec/1.x/envelope/) to see exactly what travels on the queue.

---

# Cross-language dead-letter queues

Source: https://babelqueue.com/blog/cross-language-dead-letter-queues/

Route failed and unroutable messages to a dead-letter queue any language can read, with a structured reason — opt-in, additive, and still schema_version 1.

A message fails after its retries run out, or arrives with a URN no handler maps.
BabelQueue routes it to a [dead-letter queue](/glossary/#dead-letter-queue) that is an ordinary queue of canonical
envelopes, so a Go operator can triage a failure a PHP producer caused — and the
original `trace_id` survives the trip.

This is opt-in and additive. When you don't enable it, nothing changes; when you
do, the envelope stays at `schema_version: 1`.

## When a message goes to the DLQ

Three things send a message to the dead-letter queue:

- **Permanent failure.** A handler throws, the worker retries up to its policy,
  and retries are exhausted. The `failed(...)` hook runs first; then the message
  is dead-lettered.
- **An unroutable message** when you set the unknown-URN strategy to
  `dead_letter` (more on that below).
- **A poison body** — undecodable JSON, or an oversized payload — handled
  best-effort so it can be inspected rather than retried forever or lost.

The DLQ is one mechanism for all three, across Redis and RabbitMQ. It does not
replace your framework's native tools: Redis/Laravel permanent failures still
land in `failed_jobs` (`php artisan queue:failed`), and RabbitMQ's broker-native
DLX still works if you configure it. The cross-language DLQ adds a
language-neutral, inspectable copy on top.

## The four unknown-URN strategies

When a consumed message's URN has no mapped handler, the configured strategy
decides what happens. Every SDK offers the same four, with the same names and
semantics (in Laravel, set via `on_unknown_urn` in `config/babelqueue.php`):

| Strategy | Behaviour |
| :--- | :--- |
| `fail` (default) | Throw `UnknownUrnException`; the worker retries, then the message moves to `failed_jobs`. |
| `delete` | Ack and drop the message — for shared queues where this service legitimately ignores some URNs. |
| `release` | Put it back on the queue after a configured delay — for rolling deploys where the handler may not be registered *yet*. |
| `dead_letter` | Quarantine it on the DLQ, then ack. Degrades to `delete` if the DLQ is disabled. |

One caution on `release`: if **no** service has the handler, releasing creates an
infinite redelivery loop. Use it only during a known migration window, with a
delay, and monitor it. When you want unroutable messages kept for triage rather
than retried or dropped, `dead_letter` is the strategy.

## How a dead-lettered message is built

The DLQ holds the **original envelope, preserved verbatim** — same `trace_id`,
same `meta.id`, same `data`. On top of that, BabelQueue attaches one optional
top-level field, `dead_letter`, describing why the message failed.

That field appears *only* on DLQ messages, never on a normal queue. Because it is
additive and optional, the envelope stays at `schema_version: 1`, and any
consumer reading a normal queue ignores it. Here is a payment job that
dead-lettered after three attempts:

```json
{
  "job": "urn:babel:orders:charge",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": {
    "order_id": 1042,
    "amount_cents": 9990
  },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 3,
  "dead_letter": {
    "reason": "failed",
    "error": "Payment gateway timeout",
    "exception": "App\\Exceptions\\GatewayTimeout",
    "failed_at": 1749132730000,
    "original_queue": "orders",
    "attempts": 3,
    "lang": "php"
  }
}
```

Everything above `dead_letter` is the message exactly as it was produced. The
block itself carries the failure context:

| Field | Meaning |
| :--- | :--- |
| `reason` | Why it was dead-lettered — `failed`, `unknown_urn`, or `poison`. |
| `error` | The exception message, or `null`. |
| `exception` | The fully-qualified exception type, or `null`. |
| `failed_at` | When it was dead-lettered, as Unix epoch milliseconds (UTC). |
| `original_queue` | The queue the message came from. |
| `attempts` | How many delivery attempts were made before it gave up. |
| `lang` | The language of the SDK that dead-lettered it. |

Note `lang` here is the SDK that *failed* the message, which can differ from the
`meta.lang` of the producer — a Python worker can dead-letter a job a PHP service
produced. The `dead_letter` shape and the queue naming are identical in every
SDK that supports a DLQ, verified by the shared conformance fixtures.

## Naming and turning it on

The DLQ is disabled by default. Enable it, and a failed message goes to an
explicit configured queue, or else to `<original_queue>` plus a suffix (default
`.dlq`):

```
orders  →  orders.dlq
```

In the Laravel adapter that is the `babelqueue.dead_letter` config; a
`DeadLetterPublisher` does the routing when the dispatcher fails a job or when
the `dead_letter` unknown-URN strategy fires. Other SDKs mirror the same config
surface and the same naming.

## Why a structured, language-neutral DLQ matters

`failed_jobs` is a PHP database table. A Go or Python operator cannot see or
triage failures from a PHP producer there — and for a polyglot product, "where do
dead messages go?" needs a language-neutral answer.

Making the DLQ an ordinary queue of canonical envelopes gives you that:

- **Any SDK can read a failed message.** The DLQ is just another queue; you
  consume it with the SDK you already run. No new infrastructure, no PHP runtime
  on the triage side.
- **Correlation survives.** The original `trace_id` and `meta.id` are preserved,
  so a dead-lettered message joins the same end-to-end trace as the rest of its
  causal chain.
- **The failure context travels with the message.** `reason`, `error`,
  `exception`, and `attempts` are in the envelope, not in a sidecar log — so
  whoever inspects the DLQ, in whatever language, sees why it landed there.

One caveat: re-driving messages from the DLQ back to the source queue is an
operator and tooling concern for v1, not a built-in command. The DLQ gives you a
durable, inspectable, language-neutral record of every failure; replaying it is
your call.

## Takeaway

Turn the DLQ on, and failed or unroutable messages land on an ordinary queue any
of the six SDKs can read, each carrying a structured `dead_letter` reason and its
original `trace_id`. It is opt-in, additive, and keeps the envelope at
`schema_version: 1` — so triage stops being a PHP-only problem.

Read the full structure in the
[optional `dead_letter` block](/docs/spec/1.x/envelope/#optional-dead_letter-block).

---

# BabelQueue vs CloudEvents: two envelopes, different problems

Source: https://babelqueue.com/blog/babelqueue-vs-cloudevents/

CloudEvents standardizes how an event describes itself across protocols; BabelQueue freezes what a queue job looks like so six languages run it. An honest comparison — the real overlap, the real differences, and where CloudEvents is the better choice.

CloudEvents and BabelQueue both put a structured, language-neutral envelope around a message,
and that is roughly where the resemblance ends. **CloudEvents standardizes how an event
describes itself** so any producer, protocol and platform can agree on the metadata around an
occurrence. **BabelQueue freezes what a queue job looks like on the wire** — the [wire envelope](/glossary/#wire-envelope) — so six language SDKs
execute the same work item off the broker you already run.

That difference in scope decides everything downstream: what each one pins down, what each one
deliberately leaves open, and which of them you should actually adopt. This post compares them
on the details, and says plainly where CloudEvents is the better answer — because for a large
class of systems it is.

## What each one standardizes

### CloudEvents

[CloudEvents](https://cloudevents.io/) is a CNCF specification (graduated in January 2024) for
describing event data in a common way. Version 1.0 defines four **required** context attributes
— `id`, `source`, `specversion` and `type` — and four **optional** ones: `datacontenttype`,
`dataschema`, `subject` and `time`. An event MAY carry domain data in `data`, whose media type
is declared by `datacontenttype`. Producers MUST ensure `source` + `id` is unique per distinct
event, and `type` SHOULD be prefixed with a reverse-DNS name, e.g.
`com.github.pull_request.opened`.

Around that core, the spec family defines **event formats** (JSON, Avro, Avro Compact, Protobuf,
XML) and **protocol bindings** (HTTP, AMQP, Kafka, MQTT, NATS, WebSockets), each with a
*structured* content mode (metadata and data together in the body, e.g.
`application/cloudevents+json`), a *binary* mode (data in the body, attributes mapped to the
protocol's own metadata — `ce-`-prefixed headers in the HTTP binding), and a *batched* mode. Any
number of **extension attributes** may be added; the documented ones include Distributed Tracing,
which carries W3C `traceparent` and `tracestate` as event attributes.

The scope line is drawn explicitly. "Inclusion of protocol-level routing information" is a stated
non-goal, and the specifications "will not focus on the processing model of either the event
producer or event consumer." There is no attempt counter, no dead-letter concept and no
redelivery rule anywhere in the format — that belongs to whatever platform is moving the event.

### BabelQueue

BabelQueue defines exactly one thing — the bytes on the queue — and freezes it:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 0
}
```

All five top-level fields are required. Identity is the **URN** in `job`, never a class name.
`data` is a **pure JSON object** — the cross-language rules for numbers, time and binary are part
of the contract, not implementation notes. `meta` has five required keys; a producer may add its
own, but consumers must ignore unknown ones and nothing extra may be required for routing.
`attempts` is the single mutable field, kept deliberately outside the immutable `meta` block.
There is a published list of **forbidden** fields, and `meta.schema_version` is frozen at `1`
([the envelope](/docs/spec/1.x/envelope/)).

On top of that sit the pieces a queue actually needs: a `<queue>.dlq` dead-letter convention with
an additive `dead_letter` block, consumer dedupe keyed on `meta.id`, a transactional outbox, DLQ
redrive — and framework adapters (a drop-in Laravel queue driver, a Symfony Messenger serializer,
Spring AMQP, NestJS/BullMQ, MassTransit, Celery) so your existing worker keeps its retry and
supervision and only the serialization changes.

## Where they genuinely overlap

Three overlaps are real, not superficial.

**A structured envelope with required metadata.** Both refuse to let a message be an opaque blob,
and both define where its identity, its message id and its timestamp live instead of leaving that
to each producer.

**A stable string type identifier.** CloudEvents `type` and BabelQueue `job` solve the same
problem: name the kind of message with a string the receiving code can match on, rather than a
class or type shared between services. CloudEvents recommends reverse-DNS
(`com.github.pull_request.opened`); BabelQueue recommends
[`urn:babel:<context>:<event>`](/docs/spec/1.x/urn/). Neither is enforced by the libraries; both
are conventions teams are expected to hold themselves to.

**Bindings that project onto native protocol metadata.** CloudEvents binary mode maps attributes
onto the protocol's own metadata — `ce-`-prefixed headers in HTTP — so a router never parses the
body. BabelQueue's
[broker bindings](/docs/spec/1.x/broker-bindings/) do the analogous thing: the URN goes to AMQP
`type` and `trace_id` to `correlation_id`, or to SQS `MessageAttributes`, or to an Azure Service
Bus `Subject`.

There is an important asymmetry inside that last one. In CloudEvents **binary** mode the headers
*are* the event metadata and the body holds only `data`. In BabelQueue the body is **always** the
complete canonical envelope, and the native metadata is a redundant projection of it — never a
replacement. If a broker, proxy or bridge drops headers, a binary-mode CloudEvent loses its
identity while a BabelQueue message does not.

Correlation is a fourth, weaker overlap. CloudEvents can carry `traceparent`/`tracestate` via an
**optional** extension; BabelQueue **requires** `trace_id` on every envelope and forwards it
unchanged across every hop, with W3C `traceparent` riding out of band on the transport for exact
span linkage ([observability](/docs/spec/1.x/observability/)).

## Side by side

| | CloudEvents 1.0 | BabelQueue (`schema_version: 1`) |
| :--- | :--- | :--- |
| Standardizes | how an event describes itself, across protocols | what a queue job looks like on the wire, across languages |
| Envelope | 4 required + 4 optional context attributes, plus extensions | 5 required top-level fields; `meta` has 5 required keys |
| Identity | `type` — SHOULD be reverse-DNS | `job` — a URN, never a class name |
| Payload | `data`, any media type via `datacontenttype` | `data`, a pure JSON object, with cross-language encoding rules |
| Serializations | JSON, Avro, Avro Compact, Protobuf, XML | one: compact UTF-8 JSON |
| Where metadata lives | body (structured) or protocol headers (binary) | always the body; native metadata is a redundant projection |
| Version field | `specversion` (`1.0`) | `meta.schema_version` (`1`, frozen) |
| Extensibility | any number of extension attributes | additive optional keys; unknown keys ignored; forbidden-field list |
| Transports | HTTP, AMQP, Kafka, MQTT, NATS, WebSockets | Redis, RabbitMQ, SQS, Azure Service Bus, Pulsar, Kafka, Artemis |
| Retry / delivery | not part of the event format | `attempts` in the envelope; `<queue>.dlq` + `dead_letter` block |
| Correlation | optional Distributed Tracing extension | required `trace_id` on every message |
| Runtime | SDKs in nine languages | 6 SDK cores + framework adapters that reuse your worker |
| Governance | CNCF graduated project, 40+ listed adopters | ADR-governed, cross-SDK conformance suite in every SDK's CI |

One caveat on that transports row: per-SDK broker coverage is not uniform. PHP has no Azure
Service Bus transport, and that is a documented deferral rather than a backlog item — Service Bus
speaks AMQP 1.0 and has no modern official PHP client.

## Where they genuinely differ

### Describing a message versus running one

CloudEvents is deliberately neutral about what a message *means*: "the purpose, or semantic
meaning, of an event is out of scope," so a CloudEvent can perfectly well carry a command. What
it does not carry is any of the bookkeeping a worker queue runs on. BabelQueue assumes from the
first line that something will pick the message up, execute it, fail, retry and eventually give
up — which is why `attempts` is a top-level mutable field, why `<queue>.dlq` is a convention
rather than an afterthought, and why the dedupe key is fixed at `meta.id`.

That is a scope difference, not a defect on either side. A format that also has to describe an
HTTP webhook cannot sensibly mandate a retry counter; a format for queue jobs cannot sensibly
leave one out.

### What is frozen versus what is left open

CloudEvents leaves the serialization format, the payload media type and the extension attribute
set open, and that openness is exactly what lets it span HTTP webhooks, Kafka topics and MQTT
devices. The cost is that "we both speak CloudEvents" does not by itself mean "I can read your
messages": a Protobuf-format event over Kafka and a JSON-format event over HTTP are both perfectly
conformant, so two systems still have to agree on the event format, the content mode and the shape
of `data` before anything interoperates.

BabelQueue takes the opposite trade. One serialization, one required shape for `data`, a fixed
required `meta` key set, a forbidden-field list, and byte-comparable golden fixtures every SDK
runs in CI — so two SDKs that pass conformance can read each other's messages with no further
agreement. That buys drop-in cross-language consumption and pays for it in flexibility. A wart in
`schema_version: 1` is a wart we live with, because
[the freeze is the product](/blog/why-the-envelope-is-frozen/).

### Queue operations are in scope for one of them

Dead-lettering to `<queue>.dlq`, redrive with dry-run and sandbox routing, dedupe on `meta.id`,
a transactional outbox: all of that ships with the SDKs and none of it has a CloudEvents
equivalent, by scope rather than by omission. The practical consequence is simply that if you
standardize on CloudEvents for queue work, those parts are yours to build, buy or inherit from
the platform underneath.

### What you actually install

CloudEvents SDKs give you the event type and the protocol bindings; the dispatch loop is yours.
BabelQueue ships framework adapters, so a Laravel, Symfony, Spring or NestJS worker keeps its
retry, backoff and supervision and changes only its serializer. One honest caveat: the PHP
reference transports are produce-side, and apart from the framework-less Kafka and Pulsar
consumers, consuming in PHP goes through a framework worker.

## Where CloudEvents is the better fit

Not hedging — these are cases where adopting BabelQueue would be the wrong call:

- **You publish beyond queues.** HTTP webhooks, MQTT devices, NATS, WebSockets. CloudEvents has a
  binding for each; every BabelQueue binding is a message broker, and there is no HTTP binding.
- **Your platform already speaks it.** Knative, Azure Event Grid, Google Eventarc, Dapr, Tekton,
  OpenFaaS and others emit and consume CloudEvents natively. Putting a BabelQueue envelope into
  that world means writing and maintaining the mapping yourself.
- **Your payload is not a JSON object.** Avro, Protobuf, XML, a large binary blob, or any
  arbitrary media type. BabelQueue's `data` is a pure JSON object, full stop — base64 and a
  documented field is the only escape hatch.
- **You want a description, not a job.** If nothing retries and nothing dead-letters, and the
  consumers are unknown to the producer, CloudEvents' smaller required attribute set is simply
  less to agree on.
- **Maturity and reach matter to you.** CloudEvents is a CNCF graduated project with SDKs in nine
  languages and a long adopter list. BabelQueue is far younger and far smaller. That is a real
  risk factor and worth weighing honestly against the drop-in fit.

## Can you use both?

Nothing in the contract stops you from carrying a JSON-format CloudEvent as the `data` object of
a BabelQueue envelope — `data` must be a pure JSON object, and a structured-mode CloudEvent is
one. The identity fields map mechanically: `type` ↔ `job`.

But no SDK does that mapping for you today, so you would own it, and you would be carrying two
identity fields — decide up front which one routes, or you will eventually have messages where
they disagree. It is worth doing when a CloudEvents-native system genuinely sits on one side of
the queue, and not otherwise.

## Choosing

- **Cross-language queue jobs over the broker you already run, with a framework worker you want
  to keep.** BabelQueue. That is the case it was designed for.
- **Event notifications fanning out across protocols, platforms and unknown consumers.**
  CloudEvents, and it is not close.
- **A CloudEvents platform on one side and polyglot workers on the other.** Both, with an explicit
  mapping you own at the boundary.

The two specs answer different questions, so "which is better" is the wrong frame. The useful
question is whether your messages need delivery bookkeeping — retries, a dead-letter destination,
dedupe — or a faithful description that can travel over anything. If it is the former, read
[the wire contract](/docs/spec/1.x/envelope/) for exactly what travels on the queue, or start with
[what a polyglot queue is](/blog/what-is-a-polyglot-queue/) and the [glossary](/glossary/).

---

# What is a polyglot queue?

Source: https://babelqueue.com/blog/what-is-a-polyglot-queue/

A polyglot queue is a message queue whose jobs are produced in one language and consumed natively in another. Why language-native serialization blocks that, the failure modes teams actually hit, and what a working solution has to provide.

A **[polyglot queue](/glossary/#polyglot-queue)** is a message queue whose jobs are produced in one programming language and
consumed natively in another — a PHP app enqueues the work, a Go service runs it, and neither
needs the other's runtime installed. It is not a kind of broker or a product you buy: any Redis,
RabbitMQ or SQS queue becomes polyglot the moment the bytes on it are encoded in a format every
consumer can read.

That distinction is worth stating plainly, because the thing that stops most queues from being
polyglot is not the broker. It is the serializer.

## Why an ordinary queue is not polyglot

Your broker is already language-neutral. Redis moves byte strings; it has no opinion about what
is inside them. What binds a queue to one runtime is the framework's default serializer, and
nearly every framework defaults to a **language-native** one.

Laravel's native queue serializes the job object with [PHP's `serialize()`](/glossary/#php-serialize-lock-in). The bytes describing
the work look roughly like this:

```text
O:21:"App\Jobs\ProcessOrder":1:{s:8:"order_id";i:1042;}
```

Three separate problems are packed into that one line:

- **The identity is a class name.** `App\Jobs\ProcessOrder` is a path in a PHP source tree.
- **The types are PHP's.** `O:` means "a PHP object"; the payload is an object graph, not data.
- **The format is PHP's.** Only a PHP parser reconstructs it.

Every ecosystem has its own version. Python `pickle` writes a byte stream that references a
module path and rebuilds the object by importing it. Java's `ObjectOutputStream` writes a class
name plus a `serialVersionUID`. .NET had `BinaryFormatter`, which Microsoft has since deprecated
and removed from the runtime, partly over the risks below.

They differ in encoding and agree on the fatal part: they serialize *an object* rather than
*data*, so identity is a class, the payload is a runtime's type system, and the format is a
runtime's private business.

## The failure modes teams actually hit

This stays invisible while there is one language in the system. It surfaces all at once when a
second one shows up.

**The second service cannot decode the message at all.** This is not a bug in the consumer and
no amount of work on the consumer side fixes it — the bytes genuinely do not describe anything a
Go program can build. The usual first response is to write a bridge service that reads one format
and republishes another, which means a new deployable, a new single point of failure, and a
bespoke format maintained by one team.

**Your source tree becomes the wire contract.** Because identity is a class name, renaming
`App\Jobs\ProcessOrder` or moving it to another namespace breaks every message already sitting
on the queue and every message another service enqueued before the deploy. Ordinary refactors
turn into coordinated releases, and deploy windows become breakage windows.

**Deserializing untrusted bytes is a code-execution surface.** Python's own documentation warns
that unpickling data can execute arbitrary code; Java deserialization gadget chains have been a
recurring vulnerability class for a decade; PHP `unserialize()` on attacker-influenced input has
the same shape. When queue payloads are object graphs, anything that can write to your queue is
close to being able to run code in your workers.

**Nothing correlates across the boundary.** A language-native format carries whatever the runtime
happened to embed, so there is no agreed field tying a message to the request that caused it.
When a job fails two services downstream, you have a stack trace and no way back to the origin.

**You cannot read your own queue.** `redis-cli LRANGE` returns a blob only one runtime can open.
The broker's management UI shows nothing useful, and a dead-letter queue full of those payloads
is a pile of bytes rather than a triage tool.

**The workarounds compound.** Teams end up either owning a translation service forever, or
adopting a whole new streaming platform to solve what was a serialization problem — new
infrastructure, a new dispatch model, and the framework's worker left behind.

## What a solution actually has to do

Make the queue polyglot and every one of those problems goes away at the source. That takes
surprisingly little, but the little it takes must be strict.

**1. A language-neutral payload format.** JSON is the pragmatic default: every language parses
it, and a human can read it in a broker UI. The format matters less than the strictness — the
encoding rules have to be pinned, or you trade an obvious interop failure for a subtle one. Fix
UTF-8, ban `NaN`/`Infinity`, state the integer range (and what happens when a JavaScript consumer
is in the chain, where `2^53 − 1` is the real ceiling), forbid floats for currency, decide how
binary and timestamps are represented. A payload that "is JSON" but silently rounds a float or
truncates a 64-bit id is not interoperable.

**2. Identity as a string, not a type.** The message has to name what it is with a stable string
that each consumer maps to its own handler, in its own language, through its own registry.
Decoupling identity from the source tree is what makes both cross-language routing and ordinary
refactoring possible — the string is the contract, the class is an implementation detail.

**3. A correlation id preserved across every hop.** Minted once by the first producer and
forwarded unchanged by everything that touches the chain. Without it, polyglot means "another
language can decode it" but not "another language can debug it."

**4. A version field on the format itself,** so a consumer that meets bytes it does not
understand rejects or quarantines them loudly instead of best-effort parsing them into corrupt
state.

**5. An agreed destination for failures** — a dead-letter queue holding the same neutral format,
so whichever team is on call can read a failure that another language produced.

None of this requires a new broker, a sidecar, or a rewrite of your workers. The queue you
already run moves bytes; what changes is the encoder and the routing key.

## One implementation

BabelQueue is one way to get those five properties, and a useful way to see them concretely.
Every SDK — PHP, Python, Go, Node.js, Java and .NET — produces and consumes exactly this
document:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 0
}
```

Map that back to the list. `data` is the neutral payload — a **pure JSON object**, with the
number, time and binary rules written into the contract rather than left to each SDK. `job` is
the identity: a **URN** such as `urn:babel:orders:created`, never a class name, which each
consumer maps to its own handler. `trace_id` is the correlation id, minted by the first producer
and forwarded unchanged across every hop. `meta.schema_version` is the version field, frozen at
`1`, which consumers must reject or quarantine when they do not recognize it. Failures go to
`<queue>.dlq` carrying the same canonical envelope plus an additive `dead_letter` block. The one
extra field, top-level `attempts`, is the transport's retry counter — the only value a broker or
worker is allowed to change.

On the producing side that is a one-liner, and the broker connection does not change:

```php
BabelQueue::publish('urn:babel:orders:created', ['order_id' => 1042]);
```

A Go, Python, Java, .NET or Node worker reads the identical bytes off the same queue and routes
on the string `urn:babel:orders:created`, sharing no type with the producer. Rename the PHP class
that afternoon and nothing downstream notices.

Two things make that hold in practice rather than on paper. The envelope is **frozen** at
`schema_version: 1` — additive optional fields keep the version, while removing or retyping
anything requires a deliberate bump and a migration — and every SDK runs a shared
[conformance suite](https://github.com/BabelQueue/conformance) of golden envelopes in CI, so two
SDKs that pass it can read each other's messages. A standard that is only asserted drifts; one
that is executed in every repository's CI does not.

## What a polyglot queue does not give you

Being honest about the boundary is part of the definition:

- **It is not a streaming platform.** No replayable log, no partitioned ordering, no consumer
  groups. If you need those, you need Kafka, and a neutral envelope is not a substitute.
- **It does not make your domain neutral.** Producer and consumer still have to agree on what
  `data` means behind a given identity. The envelope makes the *transport* language-agnostic;
  the payload's meaning is a contract your teams own, ideally with a per-identity schema.
- **It buys a single-language system nothing.** If PHP produces and PHP consumes, `serialize()`
  is fine and switching costs you effort for no gain. The value appears exactly when a second
  runtime does.
- **Delivery is still at-least-once.** A neutral format does not change your broker's semantics,
  so handlers must stay idempotent.

## Takeaway

A polyglot queue is an ordinary queue whose messages are encoded so that any language can read
them: strict neutral data instead of an object graph, a stable string identity instead of a class
name, a correlation id that survives every hop, a version field, and a failure destination in the
same format. The broker does not change and neither does your worker — only the serializer does.

Definitions for the vocabulary in this post are in the [glossary](/glossary/); the exact bytes are
in [the wire contract](/docs/spec/1.x/envelope/). For a worked example, see
[PHP produces, Go consumes — over Redis](/blog/php-produces-go-consumes-redis/).

---

# Why the envelope is frozen at schema_version 1

Source: https://babelqueue.com/blog/why-the-envelope-is-frozen/

Freezing the wire envelope is the whole product. Here's what 'frozen' means, what can still change, and how versioning keeps a v1 producer readable by every v1 consumer.

The envelope is the product. Six SDKs, one queue, no PHP `serialize()` — that only
works if every language agrees, byte for byte, on the shape of a message, and keeps
agreeing as the project ships releases for years. So we froze it. The [wire
envelope](/glossary/#wire-envelope) stays at [`schema_version: 1`](/glossary/#schema-version), and a v1 producer is readable by every v1 consumer, in
any language, indefinitely.

This post is about what that freeze actually buys you, what it costs us, and where the
seams are — because "frozen" does not mean "nothing can ever change."

## What "frozen" means

A BabelQueue message looks like this, in every language:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 0
}
```

Freezing means the set of required fields, their names, their types, and their meanings
do not change while `schema_version` stays `1`. A consumer that understood this shape on
the day BabelQueue 1.0 shipped will understand it next year and the year after. No SDK is
allowed to quietly rename `job` to `urn`, move `attempts` into `meta`, or start emitting
`created_at` in seconds instead of milliseconds. Those are not bug fixes; each one would
break a consumer in some other language that you do not control and may not even know is
running.

The freeze is enforced, not aspirational. The [wire contract](/docs/spec/1.x/envelope/)
is the source of truth, every SDK vendors a shared set of golden-envelope fixtures, and
two SDKs producing the same logical message emit byte-comparable envelopes — only
`meta.id`, `trace_id`, `meta.lang`, and `meta.created_at` legitimately differ, because
they are per-message or per-language by definition.

## What the contract pins down

"Frozen" is concrete because the contract is concrete. These are the fields, all
required, and what they are:

| Field | Type | What it is |
| :--- | :--- | :--- |
| `job` | string | The message **URN** — its language-agnostic identity, never a class name. |
| `trace_id` | string (UUID) | Cross-service correlation id, preserved and forwarded unchanged across every hop. |
| `data` | object | The business payload as **pure** JSON — no language-native types. |
| `meta` | object | Producer-set, immutable metadata (`id`, `queue`, `lang`, `schema_version`, `created_at`). |
| `attempts` | integer | Transport-level retry counter, starting at `0`. The one field a broker may mutate. |

Two of those entries carry most of the design weight. `job` holds a URN such as
`urn:babel:orders:created`, never `App\Jobs\OrderCreated`, because a PHP class name means
nothing to a Go worker — identity has to be a string every language can route on.
`attempts` lives at the top level, deliberately outside the immutable `meta` block: it is
*transport* state, not *producer* metadata, and keeping it top-level lets the BabelQueue
Redis driver reuse Laravel's native reservation script, which increments
`payload.attempts` on every pop. Inside `meta`, `created_at` is Unix epoch milliseconds in
UTC — not seconds, not a locale-formatted string — so six languages parse the same number
the same way.

The contract also fixes how `data` is encoded, because the same bytes are decoded by six
runtimes. UTF-8 JSON only; no NaN or Infinity; integers within signed 64-bit range (and
under `2^53-1` if a Node consumer is in the chain, or carried as a string); no currency in
floats — use integer minor units. Booleans are `true`/`false`, never `0`/`1`. These read
like style notes in one language and become interop bugs the moment a second language
reads the payload, so they are part of the contract, not implementation detail.

## What can still change

Freezing the envelope does not freeze the project. The rule is narrow:

- **Additive, optional, backward-compatible fields keep `schema_version: 1`.** A new
  optional `meta.*` key that consumers already ignore is not a breaking change, so it does
  not bump the version. Consumers MUST tolerate unknown extra keys — never hard-fail on
  them — which is exactly what makes additive growth safe.
- **Removing, renaming, or retyping a field — or changing what an existing field means —
  bumps the version** to `2`, and that is treated as a major ecosystem event, not a routine
  change. It requires an ADR, an update to the contract and the JSON Schema, new golden
  fixtures, a compatibility window where consumers read both versions, and coordinated SDK
  releases.

The asymmetry is the whole point. Adding is cheap because old consumers ignore what they
do not recognize. Removing or changing is expensive because some consumer, somewhere, in a
language you did not write, depends on the old shape.

### A worked example: the `dead_letter` block

The optional `dead_letter` block is additive-without-a-bump in practice. A message that
lands on a dead-letter queue carries one extra top-level field describing why it failed:

```json
"dead_letter": {
  "reason": "failed",
  "error": "Payment gateway timeout",
  "exception": "App\\Exceptions\\GatewayTimeout",
  "failed_at": 1749132730000,
  "original_queue": "orders",
  "attempts": 3,
  "lang": "php"
}
```

It appears *only* on DLQ messages, never on a normal queue, and the envelope stays at
`schema_version: 1`. A consumer that has never heard of `dead_letter` ignores it, exactly
as the forward-compatibility rule requires. A DLQ-aware tool reads it. Nobody had to
rebuild and redeploy the fleet to introduce a cross-language dead-letter format — that is
additive evolution working as designed.

## Two version axes, on purpose

The most common way to break a polyglot fleet is to confuse two numbers that move
independently:

| Axis | Versions | Lives in | Example |
| :--- | :--- | :--- | :--- |
| **Envelope schema** | the wire format itself | `meta.schema_version` (integer) | `1` |
| **SDK package** | a library release per language | the package registry (SemVer) | `babelqueue/laravel` `v1.4.2` |

The SDK packages follow [Semantic Versioning](https://semver.org) and move on their own
schedules. The Go SDK at `v1.3.0` and the PHP SDK at `v1.4.2` is normal — they are coupled
through the envelope schema, not through their package numbers. An SDK can ship a MAJOR for
a breaking change to *its own API* (a renamed method, a changed constructor) while the bytes
on the queue stay at `schema_version: 1`, untouched. The wire envelope is explicitly exempt
from package SemVer: it changes only via a `schema_version` bump plus an ADR, never as a
side effect of any package's major release.

There is a third axis worth naming: the `data` shape behind a single URN. If
`urn:babel:orders:created` needs an incompatible payload change, you do not touch the
envelope and you do not bump anyone's package for it — you mint a new URN like
`urn:babel:orders:created.v2`, run both in parallel, migrate, and retire the old one. Three
axes, three reasons to change, kept apart so a change on one does not masquerade as a change
on another.

## How a v1 producer stays readable everywhere

When an envelope does eventually need a breaking change, one rule keeps the fleet intact:
**consumers upgrade before producers.** You never emit a `schema_version` that no deployed
consumer understands. Deploy the consumers that read both v1 and v2 first, switch producers
to emit v2 second, and remove old-version support only after the migration window. A
consumer MUST reject or quarantine an envelope whose `schema_version` it does not support —
never best-effort parse it — so an out-of-order rollout fails loudly instead of corrupting
data silently.

Until that day, a v1 producer in any language stays readable by every v1 consumer in every
language. That is the guarantee the freeze exists to provide.

## The forbidden fields, and why

A few fields are explicitly banned from the envelope. They are not just unused — emitting or
relying on them is a contract violation:

| Forbidden | Use instead | Why |
| :--- | :--- | :--- |
| `timestamp` (top-level) | `meta.created_at` (Unix **ms**) | Seconds precision and a duplicate of `created_at`. |
| `meta.max_retries` | consumer-side config / per-URN policy | Retry policy is the consumer's, not the producer's. |
| `meta.source` | `meta.lang` | Two names for the producer language; one wins. |
| `meta.ts` | `meta.created_at` | A second, ambiguous time field. |

These shapes are real — they appeared in early drafts of the PRD and the marketing site
before the contract was reconciled in [ADR-0002](/docs/spec/1.x/envelope/), which picked one
canonical envelope out of three competing ones. They are forbidden because the freeze only
means something if there is exactly one way to express each fact. Two time fields, or two
ways to name the producer language, hand every consumer a choice — and a polyglot fleet
cannot survive each language guessing differently.

## The honest trade-off

A freeze is a constraint, and it costs us flexibility. We cannot fix the envelope's small
imperfections by editing it — a wart in v1 is a wart we live with, or a `schema_version: 2`
migration that touches every SDK and every deployment. Adding a genuinely new field means
finding an additive, optional shape that every one of six languages can carry, or paying the
full coordinated-bump price. Most "small" envelope changes are not small once you count the
consumers you do not control.

We took that constraint deliberately. The alternative — letting each SDK evolve the wire on
its own schedule — is precisely the `serialize()` problem we set out to delete, where the
bytes only make sense to the language that wrote them. The discipline is the feature. A
frozen envelope is the reason you can produce in Laravel today and trust a Go worker to read
it next year without a meeting.

## Takeaway

Frozen means the required fields, types, and meanings of a `schema_version: 1` envelope do
not change — additive optional fields stay v1, while removing or retyping anything bumps the
version behind an ADR and a consumers-first migration. The envelope schema and your SDK
package versions move on separate axes, so you pin and upgrade libraries freely while the
bytes on the queue stay stable. Read the field-by-field guarantees in the
[wire contract](/docs/spec/1.x/envelope/).

---

# Publishing polyglot jobs from Laravel

Source: https://babelqueue.com/blog/publishing-polyglot-jobs-from-laravel/

Add BabelQueue to a Laravel app and publish jobs that Go, Python, Node, Java and .NET services consume natively — without changing your broker.

Your Laravel app already dispatches jobs onto Redis or RabbitMQ. Here's how to make some of those jobs readable by a Go, Python, Node, Java, or .NET service — over the same broker, with no `serialize()` on the wire, and without touching the standard Laravel jobs you already run.

BabelQueue adds a custom queue connection. Dispatch to it and the message goes out as a strict JSON envelope keyed by a URN; dispatch to any other connection and Laravel behaves exactly as before. The only thing that becomes polyglot is the connection you opt in.

## Install

Pull in the Laravel driver and publish its config:

```bash
composer require babelqueue/laravel
php artisan vendor:publish --tag=babelqueue-config
```

That writes `config/babelqueue.php`, which holds the URN → handler map, the unknown-URN policy, and the dead-letter settings. You need PHP 8.2+, Laravel 11 or 12, and a running Redis or RabbitMQ broker.

## Configure a polyglot connection

BabelQueue is a Laravel queue driver, so its connection lives in `config/queue.php` next to your existing ones. Add a `babelqueue` connection on the Redis transport:

```php
// config/queue.php
'connections' => [

    'babelqueue' => [
        'driver'      => 'babelqueue-redis',
        'connection'  => 'default',
        'queue'       => env('BABELQUEUE_QUEUE', 'default'),
        'retry_after' => 90,
    ],

],
```

Prefer RabbitMQ? Swap the driver and point it at the broker:

```php
'babelqueue' => [
    'driver' => 'babelqueue-rabbitmq',
    'host'   => env('RABBITMQ_HOST', '127.0.0.1'),
    'port'   => env('RABBITMQ_PORT', 5672),
    'queue'  => env('BABELQUEUE_QUEUE', 'default'),
],
```

Your `redis`, `sync`, `database`, and `sqs` connections stay as they are. A job dispatched onto one of those is still a normal Laravel job, serialized the normal way. Only `babelqueue` is polyglot.

## Map URNs to handlers

Consumers route on a URN — `urn:babel:<context>:<event>` — not a PHP class name. That's what lets a non-PHP service consume the message: the identity on the wire is a stable string, not a class only PHP can resolve. Map each URN to the handler that processes it in `config/babelqueue.php`:

```php
// config/babelqueue.php
return [

    // Map each incoming URN to the handler that consumes it.
    'handlers' => [
        'urn:babel:orders:created' => App\Babel\Handlers\OrderCreatedHandler::class,
    ],

    // What to do when an incoming URN has no registered handler:
    // 'fail' | 'delete' | 'release' | 'dead_letter'
    'on_unknown_urn' => 'dead_letter',

    // Where exhausted or unroutable messages go. Failures from "orders"
    // are republished to "orders.dlq" as the same envelope plus an
    // additive `dead_letter` block, so any SDK can triage them.
    'dead_letter' => [
        'enabled' => true,
        'suffix'  => '.dlq',
    ],

];
```

`on_unknown_urn` decides the fate of a message whose URN you haven't mapped. Routing it to `dead_letter` keeps it for triage instead of dropping it — useful while a polyglot fleet is still settling on its URNs.

## Produce a message

Two ways to publish, both emitting the identical envelope. For a job with its own shape and behavior, implement `ShouldQueuePolyglot`: `getBabelUrn()` returns the URN, `toPayload()` returns the `data` body.

```php
use BabelQueue\Contracts\ShouldQueuePolyglot;

final class OrderCreated implements ShouldQueuePolyglot
{
    public function __construct(
        private int $orderId,
        private float $amount,
    ) {}

    public function getBabelUrn(): string
    {
        return 'urn:babel:orders:created';
    }

    public function toPayload(): array
    {
        return [
            'order_id' => $this->orderId,
            'amount'   => $this->amount,
        ];
    }
}

dispatch(new OrderCreated(1042, 99.90));
```

For a one-off publish, the `BabelQueue` facade takes the URN and payload directly and returns the message id:

```php
use BabelQueue\Facades\BabelQueue;

$id = BabelQueue::publish('urn:babel:orders:created', [
    'order_id' => 1042,
    'amount'   => 99.90,
]);
```

Either path produces the canonical `schema_version: 1` envelope. `data` is pure JSON, `job` is the URN, `trace_id` is stamped on publish, and `meta` carries the routing context:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042, "amount": 99.90 },
  "meta": { "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567", "queue": "default", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

This is the contract. A Go, Python, Node, Java, or .NET service reading the same queue parses these exact bytes — only `meta.lang` changes to name the consuming SDK. Nothing in here is PHP-specific, so nothing needs PHP to decode it.

## Consume in Laravel

A PHP consumer maps the same URN to a handler. The handler's `handle()` receives the decoded `data`, the `meta` block, and the `trace_id` for cross-service correlation. The optional `failed()` hook runs when processing throws:

```php
namespace App\Babel\Handlers;

final class OrderCreatedHandler
{
    public function handle(array $data, array $meta, string $traceId): void
    {
        logger()->withContext(['trace_id' => $traceId])
            ->info('Order received', $data);

        // ... your business logic
    }

    public function failed(array $data, ?\Throwable $e): void
    {
        report($e);
    }
}
```

`$traceId` is the same value the producer stamped, forwarded unchanged. Log it on every hop and one query reconstructs the whole path a job took, across every language that touched it.

## Run the worker

BabelQueue is a drop-in driver, so you consume with the standard Laravel worker — point it at the polyglot connection:

```bash
php artisan queue:work babelqueue
```

Your other workers keep running against their own connections, untouched. Exhausted retries and unmapped URNs follow the `on_unknown_urn` and `dead_letter` policy from your config.

## What you have now

One connection in your Laravel app speaks the canonical envelope. A job published there is consumable by all six SDKs — PHP, Python, Go, Node, Java, and .NET — at 1.0, over the Redis or RabbitMQ broker you already run. Everything outside that connection is the Laravel you already know.

To consume from another language, register the same URN in that SDK and point its worker at the same queue. Read [consuming messages](/docs/laravel/1.x/consuming-messages/) for the full PHP loop, and the [wire contract](/docs/spec/1.x/envelope/) for the byte-level shape every SDK agrees on.

---

# PHP produces, Go consumes — over Redis

Source: https://babelqueue.com/blog/php-produces-go-consumes-redis/

A PHP producer and a Go worker share one Redis queue, exchanging the identical JSON envelope — no shared class, and no PHP runtime on the Go side.

A PHP producer writes a job to a Redis list, and a Go worker reads the same bytes
off the same list — no shared class, no PHP runtime on the Go side. This works
because both ends speak one wire contract: the canonical BabelQueue envelope at
`schema_version: 1`, routed by a URN instead of a class name.

The code below is the [`redis-orders` example](https://github.com/babelqueue/babelqueue-examples),
running against the published 1.0 SDKs. Start Redis (`docker run -d -p 6379:6379
redis:7`), run the PHP producer once, then run the Go worker.

## The producer: PHP, framework-less

The PHP core ships an `EnvelopeCodec` and an optional `RedisTransport` (a thin
wrapper over predis that does a plain `RPUSH`). A job is anything implementing
`PolyglotJob` — a stable URN plus a pure-JSON payload:

```php
use BabelQueue\Codec\EnvelopeCodec;
use BabelQueue\Transport\RedisTransport;
use Predis\Client;

$queue     = 'orders';
$transport = new RedisTransport(new Client('redis://localhost:6379/0'), $queue);

$messages = [
    new OrderMessage('urn:babel:orders:created', ['order_id' => 1042, 'amount' => 99.90, 'currency' => 'USD']),
    new OrderMessage('urn:babel:orders:created', ['order_id' => 1043, 'amount' => 12.50, 'currency' => 'EUR']),
    new OrderMessage('urn:babel:catalog:item.indexed', ['sku' => 'WIDGET-1', 'title' => 'Café Widget ☕']),
];

foreach ($messages as $message) {
    $envelope = EnvelopeCodec::fromJob($message, $queue);
    $transport->publish(EnvelopeCodec::encode($envelope), $queue);
}
```

`EnvelopeCodec::fromJob($message, $queue)` builds the envelope; `encode(...)`
turns it into the JSON string that lands on the queue. `RedisTransport::publish`
takes that string and runs `RPUSH orders`. Nothing PHP-specific touches the wire —
no `serialize()`, no class name, no object graph.

Here is one of the encoded envelopes, exactly as it sits in Redis:

```json
{
  "job": "urn:babel:catalog:item.indexed",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "sku": "WIDGET-1", "title": "Café Widget ☕" },
  "meta": { "id": "0f8a…", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1750809600000 },
  "attempts": 0
}
```

The identity is the URN in `job`. The payload is plain JSON in `data`. The only
field that announces the producer is `meta.lang: "php"` — and the consumer treats
that as information, not a dependency.

## The consumer: Go, no PHP in sight

The Go SDK is a codec plus an optional `App` runtime. Broker drivers live in
separate modules, so the core stays dependency-free; here we pull in the `/redis`
transport, which uses the same reliable-queue pattern (`RPUSH` to produce, `BLMOVE`
to reserve, `LREM` to ack):

```go
import (
    babelqueue "github.com/babelqueue/babelqueue-go"
    bqredis "github.com/babelqueue/babelqueue-go/redis"
)

transport, _ := bqredis.New("redis://localhost:6379/0")
defer transport.Close()

app := babelqueue.NewApp(transport, babelqueue.WithDefaultQueue("orders"))

app.Handle("urn:babel:orders:created", func(_ context.Context, env babelqueue.Envelope) error {
    fmt.Printf("[go] order created  id=%v amount=%v %v  trace=%s  (produced by %q)\n",
        env.Data["order_id"], env.Data["amount"], env.Data["currency"], env.TraceID, env.Meta.Lang)
    return nil
})
app.Handle("urn:babel:catalog:item.indexed", func(_ context.Context, env babelqueue.Envelope) error {
    fmt.Printf("[go] item indexed   sku=%v title=%q  (produced by %q)\n",
        env.Data["sku"], env.Data["title"], env.Meta.Lang)
    return nil
})

processed, _ := app.Drain(ctx, "orders", 0)
```

`app.Handle(urn, handler)` registers a handler keyed by URN. `app.Drain(ctx,
"orders", 0)` processes everything currently queued and returns the count — handy
for a one-shot run; a long-lived worker calls `app.Consume(ctx)` instead, which
blocks and routes by URN until the context is cancelled.

The handler reads `env.Data` as a Go map, `env.TraceID` as a string, and
`env.Meta.Lang` to see who produced the message. There is no generated stub, no
shared schema package between the two services — just the envelope contract.

## What survived the language boundary

Run the producer, then the consumer, and the Go side prints messages it never
wrote:

```
[go] order created  id=1042 amount=99.9 USD  trace=7b3f9c2a-…  (produced by "php")
[go] order created  id=1043 amount=12.5 EUR  trace=…           (produced by "php")
[go] item indexed   sku=WIDGET-1 title="Café Widget ☕"        (produced by "php")
[go] processed 3 message(s) — same envelope, different language.
```

Three things crossed intact, and they are the point:

- **The `trace_id`** the producer generated arrives unchanged at the consumer.
  Preserving it across every hop is a contract invariant, so an end-to-end trace
  stitches together regardless of which language handled each step.
- **Unicode** in `data` — `Café Widget ☕` — round-trips byte-for-byte. The
  envelope is UTF-8 JSON; no escaping surprises, no mojibake.
- **The structured payload** stays structured. `order_id` is a number, `currency`
  is a string, and the Go map reads them as such — because `data` was JSON the
  whole way, never a PHP-serialized blob.

(The `amount` field uses a JSON number here for a runnable demo; in production,
represent money as minor units or a decimal string rather than a float.)

## Why this holds

The two services never share a type. They agree on one document: the frozen wire
envelope. PHP's `RedisTransport` and Go's `/redis` transport implement the same
reliable-queue convention on a plain Redis list, so either language can sit on
either end — swap the PHP producer for Go, Python, Node, Java, or .NET, and the
Go worker keeps reading. All six SDKs are 1.0 GA and emit byte-compatible
envelopes, verified by a shared conformance suite.

That is the whole trick: stop serializing objects, start exchanging one JSON
contract. The broker you already run carries it.

Read the [wire contract](/docs/spec/1.x/envelope/) for the field-by-field
definition, then wire up your own producer with the PHP
[transports guide](/docs/php-sdk/1.x/transports/) and the Go
[runtime and transports guide](/docs/babelqueue-go/1.x/runtime-and-transports/).

---

# BabelQueue 1.0 is here — polyglot queues, simplified

Source: https://babelqueue.com/blog/babelqueue-1-0/

All six SDKs — PHP, Python, Go, Node.js, Java and .NET — are now published at 1.0.0, with a SemVer-stable API and one frozen wire envelope.

BabelQueue reaches **1.0.0** today — across the **entire ecosystem**, all at once.
A message produced in one language is now consumed natively in another, over the
broker you already run, with a public API you can build on.

## What "1.0" means here

Two independent promises:

- **A SemVer-stable public API.** Every SDK is `1.0.0`. Breaking changes now require
  a MAJOR bump and follow a deprecation policy — so you can pin and upgrade with
  confidence.
- **A frozen wire envelope.** The on-the-wire format stays at `schema_version: 1`.
  A `v1` producer is readable by every `v1` consumer, in any language, indefinitely.

These are deliberately separate: your package versions can move independently while
the bytes on the queue stay stable.

## Six SDKs, one envelope

Every language ships a framework-agnostic **core** plus thin **framework adapters**:

- **PHP** — `babelqueue/php-sdk`, with **Laravel** and **Symfony** adapters
- **Python** — `babelqueue`, with **Celery** and **Django** adapters
- **Go** — `babelqueue-go`, with **Redis** and **RabbitMQ** transport modules
- **Node.js** — `@babelqueue/core`, with **BullMQ** and **NestJS** adapters
- **Java** — `com.babelqueue:babelqueue-core`, with a **Spring Boot** adapter
- **.NET** — `BabelQueue.Core`, with a **MassTransit** adapter

Every one of them vendors and passes the same cross-SDK conformance suite, so two
SDKs producing the same logical message emit byte-comparable envelopes.

## The idea, in one line

Don't change your broker — change only the serialization. BabelQueue replaces
language-specific serialization (like PHP's `serialize()`) with a strict, minimal
JSON envelope and routes by a **URN** instead of a class name:

```json
{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": { "id": "…", "queue": "orders", "lang": "php", "schema_version": 1, "created_at": 1749132727000 },
  "attempts": 0
}
```

A built-in `trace_id` flows across every hop and language, so an end-to-end trace is
one query away.

## Get started

Producing your first polyglot message is a single call:

```php
BabelQueue::publish('urn:babel:orders:created', ['order_id' => 1042]);
```

Pick your stack on the [ecosystem](/#ecosystem), read the
[wire contract](/docs/spec/1.x/envelope/), or jump straight into a driver's docs.

Thanks to everyone who tested the pre-1.0 builds. One envelope, every language.

---

# Frequently asked questions

Source: https://babelqueue.com/faq/

Answers to the questions teams ask before adopting BabelQueue — broker support, wire-format stability, language and framework coverage, serialization overhead and cross-language tracing.

## Do I have to replace my message broker?

No. BabelQueue runs on the broker you already operate — it changes the serialization, not the transport. There is no new broker, sidecar or proxy to deploy.

## Which brokers are supported?

Seven, all GA and conformance-locked: Redis, RabbitMQ, Amazon SQS, Azure Service Bus, Apache Pulsar, Apache Kafka and Apache ActiveMQ/Artemis. The body is byte-identical on every one, so a message produced on one broker — or in one language — is consumed natively on another. Each broker ships across the SDKs (PHP × Azure Service Bus is the one documented exception).

## Will it change my existing (non-BabelQueue) jobs?

No. Standard framework jobs and workers stay exactly as they are. Only connections you opt into as polyglot use the canonical envelope, so adoption is incremental.

## Which languages and frameworks are supported?

All six SDKs are released at 1.0 and SemVer-stable: PHP (Laravel & Symfony), Python (Celery & Django), Go (Asynq & Machinery), Node.js (BullMQ & NestJS), Java (Spring Boot) and .NET (MassTransit) — each a framework-agnostic core plus a framework adapter, with per-broker transport modules on top.

## Is the wire format stable?

Yes. The envelope is frozen at schema_version 1: within that version the existing fields, their types and their meanings do not change — no field is renamed, removed or retyped, and the required fields keep their meaning. A new field may be added only as an optional, ignorable field, and only through an architecture decision record and an explicit version decision. A consumer that does not understand an unknown field must still process the message correctly and should preserve that field when it re-emits the message.

## How is a message identified across languages?

By a URN (urn:babel:<context>:<event>) carried in the envelope, never a class name — so a Go or Python consumer routes on a stable string without sharing any type with the producer.

## Is there a performance cost?

Negligible. The codec adds well under 2% over the plain-JSON serialization a publisher already pays, measured against a conservative broker round-trip — and it talks straight to native drivers, with nothing in between.

## How do I trace a message across services?

Every message carries a trace_id, generated by the first producer and preserved unchanged across every hop and language — so an end-to-end trace is one query away, no manual plumbing.

---

# Glossary of polyglot queue terms

Source: https://babelqueue.com/glossary/

Definitions of the vocabulary BabelQueue’s wire contract uses — polyglot queue, wire envelope, schema_version, URN identity, trace_id, dead-letter queue, idempotency key, transactional outbox, redrive, broker binding and more.

## Polyglot queue

Anchor: https://babelqueue.com/glossary/#polyglot-queue

A polyglot queue is a message queue whose messages are produced in one programming language and consumed natively in another. It is not a kind of broker — the same Redis, RabbitMQ or SQS queue becomes polyglot when the bytes on it are encoded in a language-neutral format instead of a language-native one. BabelQueue makes a queue polyglot by replacing only the serializer: six SDKs (PHP, Python, Go, Node.js, Java and .NET) read and write one frozen JSON envelope over the broker you already run.

## PHP serialize() lock-in

Anchor: https://babelqueue.com/glossary/#php-serialize-lock-in

PHP serialize() lock-in is what happens when a queue’s messages are encoded as a PHP object graph: only PHP can read them, and only PHP that still has a class of exactly that name. Laravel’s native queue serializes jobs this way, so the identity on the wire is a fully-qualified class name like App\Jobs\ProcessOrder — meaningless to a Go or Python worker, and broken by an ordinary rename. Every language has its own version of the trap: Python pickle, Java object serialization. The fix is to change the serialization, not the broker or the worker.

## Wire envelope

Anchor: https://babelqueue.com/glossary/#wire-envelope

The wire envelope is the JSON document BabelQueue puts on the queue: job, trace_id, data, meta and attempts, encoded as UTF-8 JSON. It is the entire contract — every SDK in every language produces and consumes exactly this shape, so a message written by one is read natively by another. meta carries id, queue, lang, schema_version and created_at and is immutable after production; top-level attempts is the one field a broker or worker may change.

## schema_version

Anchor: https://babelqueue.com/glossary/#schema-version

meta.schema_version is the integer that tells a consumer whether it can read an envelope at all. It is currently 1 and frozen there: an additive, optional field keeps the version at 1, while removing, renaming or retyping a field — or changing what one means — requires a bump, an architecture decision record and a compatibility window. A consumer must reject or quarantine a schema_version it does not understand rather than best-effort parse it.

## URN identity

Anchor: https://babelqueue.com/glossary/#urn-identity

A BabelQueue message identifies itself with a URN string in the envelope’s job field, shaped urn:babel:<bounded-context>:<event-or-command> — for example urn:babel:orders:created. That URN is the only thing a consumer reads to decide which handler runs, so routing never depends on a type shared between services. Because it is a plain, application-owned string, the producing class can be renamed freely and a consumer in any language maps the same URN to its own handler; urn is accepted as an inbound alias when job is absent.

## data payload

Anchor: https://babelqueue.com/glossary/#data-payload

data is the business payload inside the envelope, and it must be a pure JSON object — no PHP objects, closures or resources, no language-native types, no raw bytes. Because six runtimes decode the same bytes, the encoding rules are part of the contract: UTF-8 JSON with no NaN or Infinity; integers within signed 64-bit, and under 2^53−1 if a JavaScript consumer is in the chain; currency in integer minor units or strings rather than floats; binary base64-encoded; times as Unix milliseconds or RFC 3339 UTC strings. What shape data takes behind a given URN is the application’s contract, not the envelope’s.

## trace_id

Anchor: https://babelqueue.com/glossary/#trace-id

trace_id is the required UUID that ties every message in one causal chain together. The first producer in the chain mints it, and every SDK then preserves and forwards it unchanged across every hop — including onto the dead-letter queue. It is not meta.id, which identifies one single message: when a consumer publishes a downstream message while handling one, it copies the inbound trace_id and mints a new meta.id.

## Broker binding

Anchor: https://babelqueue.com/glossary/#broker-binding

A broker binding defines how one broker carries the canonical envelope natively. The message body is always the envelope, byte-identical across SDKs; the binding additionally projects parts of it onto the broker’s own metadata — an AMQP type and correlation_id, SQS MessageAttributes, an Azure Service Bus Subject — so a consumer can route by URN and continue a trace without decoding the body. That projection is redundant, never a replacement for the body, which is why adding a broker is purely additive and never changes the wire format.

## Framework adapter vs transport

Anchor: https://babelqueue.com/glossary/#framework-adapter-vs-transport

A framework adapter binds BabelQueue to a framework’s own queue system so you keep that framework’s worker and retry machinery — a drop-in Laravel queue driver, a Symfony Messenger serializer, a Spring AMQP message converter. A transport is the much thinner seam that publishes encoded envelopes over one broker client (and, where a runtime ships one, consumes them). Both sit on a single language core that owns the codec, so the envelope cannot drift between them: in PHP, for instance, the whole Transport contract is one publish(string $payload, ?string $queue) method.

## Dead-letter queue (DLQ)

Anchor: https://babelqueue.com/glossary/#dead-letter-queue

A dead-letter queue is where a message goes when it cannot be processed — retries exhausted, a URN no handler maps, or an undecodable body. BabelQueue’s convention is the source queue name plus a .dlq suffix, so a message that failed on orders lands on orders.dlq as an ordinary canonical envelope that any SDK can read and triage. The dead-lettered message carries one extra optional top-level dead_letter block recording the reason, error, failure time, original queue and attempt count, while the original envelope inside it — trace_id included — is preserved unchanged. Dead-lettering is opt-in per SDK.

## Redrive

Anchor: https://babelqueue.com/glossary/#redrive

Redrive is the operator action of moving dead-lettered messages back onto a live queue for reprocessing. BabelQueue’s redrive removes the dead_letter block and resets attempts to 0 while preserving job, trace_id, data and meta verbatim, so a redriven message is indistinguishable from a fresh one and stays on its original trace. It ships the safe-replay primitives an operator needs: a dry run, a select predicate, a cap on how many messages are pulled, and a target-queue override that routes the replay to a sandbox instead of production.

## Idempotency key

Anchor: https://babelqueue.com/glossary/#idempotency-key

An idempotency key is the value a consumer dedupes on so a redelivered message is not processed twice. In BabelQueue that key is meta.id, verbatim — the unique identity of one specific message, already required by the frozen envelope, so no new field is needed. Each SDK ships an optional seen-set helper that skips a handler whose meta.id was already processed successfully; a message with no usable meta.id runs unchanged rather than being dropped. Unlike an HTTP idempotency key it stores no response to replay, because a queue handler has none.

## Transactional outbox

Anchor: https://babelqueue.com/glossary/#transactional-outbox

The transactional outbox is a producer-side pattern that removes the dual write — the crash window between committing your business row and publishing the message. The encoded envelope is written to an outbox table inside the same database transaction as the business row, so both commit or neither does, and a separate relay reads those rows afterwards and publishes the stored bytes verbatim. The honest guarantee is an exactly-once handoff into the broker and the usual at-least-once on the wire, with the consumer deduping on meta.id. BabelQueue’s helper defines the store contract and binds to no database driver.

## Conformance suite

Anchor: https://babelqueue.com/glossary/#conformance-suite

The conformance suite is the executable form of the wire contract: a language-neutral set of fixture envelopes, the canonical JSON Schema, and a manifest stating what a consumer must derive from each fixture. Every SDK vendors a copy and runs it in CI, so if two SDKs both pass, a message one produces is consumable by the other. It covers deliberately invalid envelopes too — an unknown schema_version, or an envelope with no identity field at all, must be rejected rather than best-effort parsed — plus broker-binding cases such as the SQS attribute projection.

---

# About BabelQueue

Source: https://babelqueue.com/about/

BabelQueue is an open-source, broker-agnostic queue standard and set of SDKs, created and maintained by Muhammet Şafak and released under the MIT licence.

BabelQueue is an open-source standard — plus the SDKs that implement it — for queue messages
that any language can read. It is free software, developed in the open, and maintained by
Muhammet Şafak.

## What it is

A producer in one language writes a job to the broker you already run; a consumer in another
language picks it up and handles it. That works because every SDK emits the same strict JSON
envelope instead of a language-specific serialization format — no PHP `serialize()` on the wire,
no bridge service, no schema registry to operate. Messages are identified by a
[URN](/docs/spec/1.x/urn/) rather than a class name, and a `trace_id` is carried unchanged across
every hop.

The [envelope is frozen](/docs/spec/1.x/envelope/) at `schema_version: 1`: no field is added,
renamed, removed or retyped without a deliberate version decision, so a v1 producer stays readable
by every v1 consumer indefinitely. That guarantee — not any single SDK — is the product.

There are six SDK cores today (PHP, Python, Go, Node.js, Java and .NET), each with framework
adapters and per-broker transport modules, and all of them are locked to a shared cross-SDK
conformance suite of golden envelopes. The [wire contract](/docs/spec/1.x/introduction/) is the
normative reference; the SDKs follow it, never the other way round. The vocabulary those pages
use is defined term by term in the [glossary](/glossary/).

## The problem it solves

Teams that run more than one language end up sharing a queue, and the default message formats do
not travel. A Laravel job is a PHP `serialize()` string that only PHP can decode, a Celery task
follows Celery's own message protocol, and each framework names jobs after its own classes. The usual
workarounds — a translation service in the middle, a second queue per language, or rewriting the
producer — add moving parts and drift over time.

BabelQueue removes the translation step. Every producer writes the same envelope, every consumer
reads it, and the broker in between stays the one you already operate.

## Status and maturity

BabelQueue is **generally available**. All six SDK cores reached
[1.0.0 on 7 June 2026](/blog/babelqueue-1-0/) together and have shipped compatible minor releases
since; each one follows SemVer, so breaking API changes wait for a major version. The wire envelope
is frozen at `schema_version: 1`, and every opt-in feature added after 1.0 — idempotency, the
transactional outbox, schema validation, tracing, DLQ redrive, field-level encryption — has kept it
there. Redis and RabbitMQ are supported everywhere; Amazon SQS, Azure Service Bus, Apache Pulsar,
Apache Kafka and ActiveMQ Artemis are supported per SDK as listed in the
[documentation](/docs/). The current version of each core is shown on the [home page](/#ecosystem).

## Who maintains it

BabelQueue was created and is maintained by **Muhammet Şafak**
([@muhammetsafak](https://github.com/muhammetsafak)), who also writes the specification, the SDKs
and the articles on this site. Work happens in public: each SDK, the shared conformance suite and
this website live in their own repositories (see below).

## Licence and source code

The standard and every SDK are released under the **MIT licence**. You can use BabelQueue
commercially, fork it, and vendor it into a closed-source product; every SDK repository carries
the licence text (© 2026 Muhammet Şafak). The source of every SDK, the conformance suite and this
website is public in the [BabelQueue organisation on GitHub](https://github.com/BabelQueue).

The content of this website is licensed under
[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/); the BabelQueue and Tignex names and
logos are excluded. See the [terms of use](/terms-of-use/) for details.

## Getting involved

Bug reports, questions and pull requests are welcome on the repository for the SDK you are using —
open an issue in the [GitHub organisation](https://github.com/BabelQueue) and it will land with the
maintainer. Two things are worth knowing before you file:

- Anything that touches the wire format is a contract change, not a bug fix. It needs a recorded
  decision and a version choice, so expect that conversation to start in the spec rather than in a
  patch.
- Behaviour that differs between two SDKs is the highest-value report there is — parity is the
  whole promise. Include the envelope both sides produced.

New to the project? Start with the [documentation](/docs/), the [FAQ](/faq/), the
[glossary](/glossary/), or the [blog](/blog/).

## About Tignex

Tignex is an independent open-source engineering organization that builds tools, infrastructure, protocols and standards for developers and software teams. Its aim is software that gives developers more control, takes interoperability seriously, avoids unnecessary vendor lock-in and can be self-hosted. Projects published under Tignex keep their own names, identities and domains; Tignex is the engineering organization they share.

[tignex.com](https://tignex.com)

## About the founder

Muhammet Şafak is the founder of Tignex — a software engineer, system architect and engineering researcher who has been building software since 2008. Today he works on the architecture, scalability and reliability of high-traffic distributed systems. He publishes independent research that examines system behaviour through reproducible measurements, and chooses technology by the problem rather than the language. In the rest of his time he builds open-source tools and infrastructure software under Tignex.

[muhammetsafak.com](https://muhammetsafak.com)

---

# Contact

Source: https://babelqueue.com/contact/

How to reach the BabelQueue maintainer: email, the GitHub issue tracker, and how to report a security vulnerability privately.

BabelQueue is maintained by Muhammet Şafak (Tignex).

## Email

For general questions, feedback or privacy requests, write to
[info@muhammetsafak.com](mailto:info@muhammetsafak.com).

## Issue tracker

Bugs, feature requests and documentation fixes belong in the issue tracker of the relevant
repository at [github.com/BabelQueue](https://github.com/BabelQueue). Pick the repository for the
SDK or the site the issue is about. Please search existing issues first.

## Reporting a security vulnerability

**Please do not open a public issue for a security vulnerability.** Report it privately, either:

- through GitHub Private Vulnerability Reporting: open the affected repository, go to its
  **Security** tab and choose **Report a vulnerability**, or
- by email to [info@muhammetsafak.com](mailto:info@muhammetsafak.com) with the subject
  `BabelQueue Security`.

Include the affected package and version, the broker (Redis, RabbitMQ, Amazon SQS, Azure Service
Bus, Apache Pulsar, Apache Kafka or ActiveMQ/Artemis), a description, and a reproduction or
proof-of-concept if you have one. We aim to acknowledge your report within 72 hours, agree on a
disclosure timeline with you, and credit reporters who wish to be named once a fix is released.