DLQ Redrive & Replay-Bypass

Last updated: October 13, 2026

A message that exhausts its retries lands on a dead-letter queue carrying the optional dead_letter block. The wire contract defines how a message gets there but deliberately leaves getting it back to tooling. This page is that tooling: the cross-SDK Redrive operation (ADR-0026) and the Replay-Bypass guard (ADR-0027) that makes a replay safe to run.

Status: Authoritative · Operator tooling · envelope frozen at schema_version: 1

Both are tooling layers. Redrive reads and re-publishes the existing frozen envelope; Replay-Bypass rides an out-of-band transport header, never a new envelope field. The wire stays at schema_version: 1.

Redrive: move messages off the DLQ, reset

Redrive drains dead-lettered messages from a DLQ and re-publishes each, reset for reprocessing:

• the dead_letter block is removed and top-level attempts reset to 0;
• job, trace_id, data and meta are preserved verbatim — a redriven message is indistinguishable from a fresh one, and its trace_id keeps it on the original trace.

By default each message goes back to its own dead_letter.original_queue; ToQueue overrides that to a sandbox queue so you can replay safely. Messages are drained from the DLQ first and then processed, so restored messages are never re-encountered in the same run; a DLQ message is acked only after its re-publish succeeds, and an undecodable (“poison”) body is restored, not lost.

The options are the safe-replay primitives:

Kafka: the record key survives. On a keyed topic the DLQ record’s key travels with the message. Both the write back to the work topic and every restore write back to the DLQ (dry run, unselected, undecodable, failed redrive write) carry the key byte for byte: a null key stays null, an empty key stays empty, and the SDK never invents one. The key sits beside the envelope, never inside it, so the envelope is unchanged. Partition equality across SDKs is not guaranteed, because client partitioners differ. Go, Java, .NET and Python do this out of the box. Node and PHP ship a KafkaRedriveIO (Node, PHP) that carries the key, provided the pop or client you supply surfaces it; without it the redrive writes keyless. See the Kafka binding, §6.5. The replay-bypass header write path carries no key.

Kafka: a restore is verbatim. A restore write back to the DLQ re-writes the consumed record as it was: its original value bytes, headers and key, with a fresh timestamp so the restored copy does not age out of the DLQ early. A record that is not valid UTF-8 is never altered, and a poison record (already dead-lettered wrapped, with dead_letter.reason: "poison") stays byte for byte.

Kafka: a failed restore rewinds the consumer. Kafka offset commits are cumulative, so if a restore write fails and the run stops, a later commit by the same consumer group could move it past the unsettled record and lose it. Every Kafka SDK therefore rewinds the consumer to the first unacknowledged record of each affected partition before the run ends (the whole batch is drained before it is processed, so more than one partition can be involved). The failed record is never acknowledged, an acknowledged record is never rewound, and if the rewind fails too both errors are reported together.

The run returns a result summarizing what happened — counts of redriven vs skipped, and a per-message item recording its meta.id, trace_id, URN, dead_letter.reason, target queue, and whether it was actually redriven.

Replay-Bypass: don’t re-fire effects that already ran

Redrive solves getting a message back; it does not solve what its handler does the second time. A deliberate replay re-runs the handler — and its external side-effects re-fire: a second charge, a duplicate email. Idempotency stops an accidental duplicate (same meta.id delivered twice); Replay-Bypass stops an intended reprocess from re-firing effects that already happened. They are complementary.

The mechanism (ADR-0027): when Redrive runs with Bypass, it stamps the bq-replay-bypass header on each redriven message. This is an out-of-band transport header, not an envelope field — so the wire stays frozen, and it only propagates over a broker whose transport implements the optional header capability. It rides the same seam as the traceparent propagation: the in-memory transport carries it, header-capable bindings carry it, and a binding that does not yet carry headers makes Bypass a no-op there. The runtime surfaces the marker to the handler, which wraps its external, non-idempotent side so a replay re-runs the idempotent core but skips the effects that already fired:

• IsReplay(ctx) — is the message currently being handled a deliberate replay?
• BypassExternalEffects(ctx, fn) — run fn unless this is a replay, in which case skip it.

Go

Redrive(ctx, transport, dlq, RedriveOptions{...}) and the replay guard (HeaderReplayBypass, IsReplay, BypassExternalEffects) are in babelqueue-go:

import babelqueue "github.com/babelqueue/babelqueue-go"

// dry run: report the plan, change nothing
plan, err := babelqueue.Redrive(ctx, transport, "orders.dlq",
	babelqueue.RedriveOptions{DryRun: true})

// redrive to a sandbox, stamping bq-replay-bypass, only messages that failed on timeout
res, err := babelqueue.Redrive(ctx, transport, "orders.dlq", babelqueue.RedriveOptions{
	ToQueue: "orders.sandbox",
	Bypass:  true,
	Select:  func(env babelqueue.Envelope) bool { return env.DeadLetter != nil && env.DeadLetter.Reason == "failed" },
})

The handler guards its external effects so a replay is safe:

app.Handle("urn:babel:orders:created", func(ctx context.Context, env babelqueue.Envelope) error {
	saveOrder(env) // idempotent core — always runs
	return babelqueue.BypassExternalEffects(ctx, func() error {
		return sendConfirmationEmail(env) // external effect — skipped on replay
	})
})

Python

babelqueue.redrive.redrive(transport, dlq, ...) mirrors the Go API; the guard is is_replay / bypass_external_effects:

from babelqueue import BabelQueue, is_replay, bypass_external_effects
from babelqueue.redrive import redrive

app = BabelQueue("redis://localhost:6379/0")

# inspect, change nothing
plan = redrive(app.transport, "orders.dlq", dry_run=True)

# redrive to a sandbox, stamping bq-replay-bypass
res = redrive(app.transport, "orders.dlq", to_queue="sandbox", bypass=True)

@app.handler("urn:babel:orders:created")
def on_order_created(data, meta):
    save_order(data)                                    # idempotent core — always runs
    bypass_external_effects(lambda: send_email(data))   # external effect — skipped on replay

PHP

PHP’s Transport is publish-only, so Redrive::run($io, $dlq, $options) drives a RedriveIO (reserve / ack / publish) that you bind to your broker. The same reset contract holds — dead_letter removed, attempts to 0, everything else preserved:

use BabelQueue\Redrive\Redrive;
use BabelQueue\Redrive\RedriveOptions;

$result = Redrive::run($io, 'orders.dlq', new RedriveOptions(
    toQueue: 'orders.sandbox', // safe sandbox replay
    dryRun: false,
    bypass: true,              // stamp bq-replay-bypass (needs a HeaderRedriveIO)
    select: fn (array $env) => ($env['dead_letter']['reason'] ?? '') === 'failed',
));

Redrive::reset($envelope) is exposed on its own for callers that re-publish through their own machinery.

The replay-bypass guard reads the marker off the delivered message (a HasHeaders), so the handler skips effects that already fired — the PHP mirror of Go’s BypassExternalEffects:

use BabelQueue\Redrive\ReplayBypass;

$handler = ReplayBypass::wrap(function (\BabelQueue\Contracts\ConsumedMessage $m): void {
    saveOrder($m);                                   // idempotent core — always runs
    ReplayBypass::bypassExternalEffects($m, fn () => sendEmail($m)); // skipped on replay
});

Per-SDK

Redrive ships across the SDKs with the same reset contract and safe-replay options, and the Replay-Bypass core guard is now in all six — the bq-replay-bypass marker is identical across SDKs, so it is cross-SDK. The behaviour is the same everywhere; only the surfacing differs by language: Go/Python carry the replay flag on the handler’s context, Java on a ThreadLocal scope, and Node/.NET/PHP pass the delivered headers explicitly (PHP’s core Transport is publish-only, so its guard is a header-reading helper like Idempotent::wrap, not a context-based one).

The one honest limit is the transport layer, not the SDK: the marker only propagates over a broker whose transport carries out-of-band headers (the in-memory transport does; HeaderPublisher-capable bindings carry it, others fall back). Until a given broker binding wires the header, Bypass is a best-effort no-op on it and sandbox ToQueue routing remains the fallback — exactly the per-transport rollout shape the traceparent propagation follows, since both ride the same header seam.

Security considerations

The bq-replay-bypass marker is not authenticated. Every SDK treats any non-empty value as a replay, and nothing proves the header came from Redrive — anyone who can publish to the source queue can set it. A forged marker makes the consumer run its idempotent core while silently skipping the guarded external effects (no charge, no email, no third-party call). If you enable replay-bypass on a handler:

1. Restrict who can publish to the source queue with broker authorization (RabbitMQ user permissions, SQS queue policy / IAM, Kafka or Pulsar ACLs, Azure Service Bus send rights, Artemis security settings).
2. Strip bq-replay-bypass at every trust boundary — bridges, relays, shovels, gateways or webhook ingress that forward messages from less-trusted producers must drop it.
3. Prefer sandbox replay (ToQueue into a queue only the operator can write to) when producers are not fully trusted, and enable bypass only on that queue’s consumer.
4. Wrap side effects only, never controls. Authorization, validation, limit or fraud checks and audit writes must stay outside BypassExternalEffects, so a forged marker can at worst suppress an effect.
5. Observe it — log or count replay-marked deliveries; one outside an operator-initiated redrive is an incident.

A signed replay marker that consumers can verify is planned; until then the marker is only as trustworthy as the publishers allowed on the queue.

For a runnable walkthrough, see the DLQ redrive example, and for the related de-duplication primitive, Idempotency.

See each SDK’s reference: Go, Python, PHP, Java, Node, .NET.