Configure the Python SDK

Last updated: October 25, 2026

Create a BabelQueue runtime and point it at your broker with a URL. The transport is chosen from the URL scheme — no separate “broker” flag.

Broker URL

from babelqueue import BabelQueue

# Redis (extra: babelqueue[redis])
app = BabelQueue("redis://localhost:6379/0", queue="orders")

# RabbitMQ (extra: babelqueue[amqp])
# app = BabelQueue("amqp://guest:guest@localhost:5672/", queue="orders")

# Amazon SQS (extra: babelqueue[sqs])
# app = BabelQueue("sqs://eu-central-1", queue="orders")

# Azure Service Bus (extra: babelqueue[azureservicebus])
# app = BabelQueue("sb://<namespace>.servicebus.windows.net", queue="orders")

# Apache Pulsar (extra: babelqueue[pulsar])
# app = BabelQueue("pulsar://localhost:6650", queue="orders")

# Apache Kafka (extra: babelqueue[kafka])
# app = BabelQueue("kafka://localhost:9092", queue="orders")

# Apache ActiveMQ Artemis (extra: babelqueue[artemis])
# app = BabelQueue("artemis://localhost:5672", queue="orders")

# In-process, great for tests/local — no broker required
# app = BabelQueue("memory://", queue="orders")

The sqs:// URL is sqs://<region> with optional query params: endpoint (LocalStack/ElasticMQ), prefix (a queue-URL prefix that skips GetQueueUrl), fifo=1, group_id, wait_time. The transport sends the canonical envelope with the §3 MessageAttributes and reconciles attempts to max(body.attempts, ApproximateReceiveCount − 1). With dead_letter=True on SQS the dead-letter queue (since 1.24.0) defaults to <queue>-dlq (FIFO: <queue>-dlq.fifo), because SQS does not accept a dot in a queue name; dead_letter_queue or dead_letter_suffix override it (on a FIFO queue a suffix goes before .fifo: orders.fifo + -dlq is orders-dlq.fifo).

The sb:// URL is sb://<namespace>.servicebus.windows.net and authenticates with DefaultAzureCredential (Azure AD); pass connection_string=... to AsbTransport for SAS auth, or BabelQueue(transport=...) with a built client. It maps the envelope onto native Service Bus fields (§4) and reconciles attempts to max(body, DeliveryCount − 1).

The pulsar:// URL is pulsar://<host>:<port> (or pulsar+ssl://); pass a built client via BabelQueue(transport=...) for TLS/token auth. It projects the envelope onto native Pulsar message properties (bq-job = URN, plus the bq- fields, §5), defaults to a Shared subscription named babelqueue, and reconciles attempts to max(body, redelivery_count) — the redelivery count is 0-based, so no −1. A retry re-publishes the message to the same topic with bq-attempts + 1 and acknowledges the original, so max_attempts (default 3) bounds it on every subscription type; when the attempts are used up the runtime writes <queue>.dlq with dead_letter.attempts set to the counted value (with the default, 3), or drops the message and logs it when dead-lettering is off. A failed DLQ write or retry publish negatively acknowledges the original, so the broker redelivers it.

The kafka:// URL is kafka://<host>:<port>[,<host>:<port>]; pass an injected producer + consumer_factory for custom auth. The record value is the envelope, projected onto bq- headers (§6); consume is process-then-commit (enable.auto.commit=false) and attempts reconciles to the authoritative bq-attempts header (else the body). Kafka has no native delay/DLQ, so the runtime owns retry by republishing with attempts+1 and dead-letters to <queue>.dlq. The transport keeps the record it last handed out until it is acknowledged or rewound.

The runtime’s own settle writes (the retry republish, the release, and the envelope, poison and schema-version-rejection dead-letter writes) go through publish_settle(), which waits for the broker to confirm that write: it uses the write’s own delivery report and waits up to delivery_timeout seconds (the optional KafkaTransport(delivery_timeout=30.0) argument). On a delivery error or a timeout it raises KafkaDeliveryError, nothing is committed, and the next poll seeks the partition back to the record, so it is redelivered instead of being skipped by the next record’s commit (the handler may run again; keep handlers idempotent). A failed seek is logged and does not block a rebalance. A missing <queue>.dlq topic or an unavailable partition leader therefore no longer loses the record. Your own app.publish() calls, also from inside a handler, are unchanged: fire-and-forget, with no wait for the broker.

A poison record (a value that is not valid UTF-8, not valid JSON, not a JSON object, or that has a member of the wrong type) never raises in pop(). With a DLQ on it is never retried; it is dead-lettered like the Go reference (ADR-0035). A body that is a strict RFC 8259 JSON object (valid UTF-8, no top-level dead_letter member, no NaN / Infinity literal, no unpaired surrogate escape, nested within the interpreter’s recursion limit) is spliced byte for byte: every original character is kept, numbers such as 1e400 or an integer beyond 2^53 stay as written, and only a dead_letter member is appended. Anything else becomes {"raw": "<body>"} (bytes that are not UTF-8 are carried as standard base64 with "raw_encoding": "base64", never with U+FFFD substitutions). Either way the dead_letter block carries reason: "poison", original_queue, attempts (from the bq-attempts header) and lang. The record key is preserved, and the offset is committed only once the DLQ write is confirmed (a failed write leaves it uncommitted and rewinds). Poison follows the rules the Go reference applies, with two intentional differences: a value that is not valid UTF-8 is poison here (Go substitutes U+FFFD and accepts it), and JSON nested deeper than the interpreter’s recursion limit (about 1000 levels, where Go allows 10000) is poison. Member names are matched as spelled (a JOB or Meta member is an unknown member, not a typed one), and duplicate keys are each checked. Whatever a decode raises after the shape scan is poison too, so a decode error never stops the consumer.

How a poison record is settled depends on on_unknown_urn and on whether a DLQ is configured, like the Go reference’s routePoison:

• UnknownUrnStrategy.DELETE drops it (and commits), even when a DLQ is configured.
• Otherwise, with a DLQ on (dead_letter enabled, or UnknownUrnStrategy.DEAD_LETTER), it is written to the DLQ without being retried.
• Otherwise (no DLQ) it takes the bounded retry path through the retry tiers and is dropped once max_attempts is reached. Without retry_tiers_ms there is no tier to retry on, so the record is committed, dropped and logged and the partition advances (it used to be left uncommitted and block the partition). The runtime’s start-up warning for that configuration says the same. Enable dead_letter to keep such records in <topic>.dlq, or configure retry_tiers_ms for the bounded retry. This applies to poison only: a failing handler, or release of a valid envelope, still raises BabelQueueException without tiers.

An identity-less JSON object (one with neither a valid string job nor a valid string urn) is poison too, not an unknown URN: it is never handed to a handler and never takes the on_unknown_urn strategies, which are for a valid envelope whose URN has no handler. On Redis it is dead-lettered unconditionally, like every Redis poison body, in both key layouts.

On the other transports (sqs://, amqp://, pulsar://, sb://, artemis:// and memory://) a body that does not decode is no longer decoded into an empty envelope and retried as an unknown URN. It takes the same poison route as above: with a DLQ on it is written to <queue>.dlq at once, without retry, as the poison record with dead_letter.reason: "poison" and the original body kept (it used to be reason: "unknown_urn" with no original body); UnknownUrnStrategy.DELETE drops it; with the DLQ off RELEASE is replaced by the bounded FAIL path (re-published with attempts + 1 up to max_attempts, then dropped and logged), except on sqs:// and sb://, where a poison body with the DLQ off is not retried and not dropped but handed to the broker’s own mechanism on its first delivery (SQS: left undeleted, so visibility expiry redelivers it and the queue’s RedrivePolicy takes it; Azure Service Bus: abandoned until MaxDeliveryCount, or dead-lettered natively; ADR-0037, since 1.24.0). The count the bound uses is the larger of the transport’s native delivery count (Pulsar, SQS, Azure Service Bus and Artemis report one) and the attempts of the re-published body, so a poison body is bounded on every transport.

The bq-schema-version gate rejection is never retried: with a DLQ on it dead-letters the incoming body raw and unchanged (no dead_letter block), with no DLQ it is dropped with a logged warning, and with UnknownUrnStrategy.DELETE it is acknowledged and dropped before the gate runs. A bq-attempts header that is not valid UTF-8 does not raise either. Consumers of the DLQ that expected the raw bytes of a malformed record must read the wrapped body.

The retry republish, the envelope dead-letter write, the raw dead-letter write, the redrive write back to the work topic and a redrive restore all keep the consumed record’s key (ReceivedMessage.key, None for a null key): a null key stays null, an empty key stays empty, and a key is never invented. Partition equality across SDKs is not guaranteed.

The artemis:// URL is artemis://<host>:<port> (or artemis+ssl://) — Apache ActiveMQ Artemis over AMQP 1.0 (python-qpid-proton, not RabbitMQ’s 0-9-1 pika); pass a built connection via BabelQueue(transport=...) for auth/TLS. It projects the envelope onto the AMQP a JMS peer reads — x-opt-jms-type = URN, correlation-id = trace_id, plus the string bq_ application properties (underscored — JMS property names must be valid Java identifiers; §7) — and reconciles attempts to max(body, delivery_count); the AMQP delivery-count is 0-based, so no −1.

Schema-version gate

The sqs://, sb://, kafka:// and artemis:// transports check the bq-schema-version message attribute / property / header (Artemis: bq_schema_version) before the body is decoded. A missing or blank value (the empty string, or only ASCII whitespace: space, tab, LF, VT, FF, CR) changes nothing: the body is decoded as usual. A header or property present with a null value counts as missing too. Any other value is compared with "1" exactly as sent, with no trimming, so "2", "x", "01" and " 1" are all rejected without decoding the body. On Azure Service Bus an integer 1 is accepted as well as the string "1"; a float (including 1.0) or a boolean is rejected. A "1" value does not skip the body check: the decoded envelope still goes through the schema_version validation.

A rejected delivery reaches the app on a reserved route, babelqueue:unsupported-schema-version:<version>. The transport gates without decoding the body, and the app does not decode it either. No handler can match the route and a retry cannot change the outcome, so it is never retried. on_unknown_urn="delete" wins over dead-lettering: the delivery is acknowledged and dropped. Otherwise, with dead_letter=True (or on_unknown_urn="dead_letter") the incoming body goes straight to the dead-letter queue byte-identical, with no dead_letter block and no unknown_urn reason. With dead-lettering off it is acknowledged and dropped on kafka:// and artemis://, while sqs:// and sb:// hand it to the broker’s own mechanism instead (SQS: left undeleted for the RedrivePolicy; Azure Service Bus: abandoned or dead-lettered natively), so nothing is dropped there (ADR-0037, since 1.24.0).

Options

app = BabelQueue(
    "redis://localhost:6379/0",
    queue="orders",
    max_attempts=3,
    dead_letter=True,
    on_unknown_urn="dead_letter",
)

Need just the wire codec, with no broker wiring? Import EnvelopeCodec directly — see Producing messages.

Next: Producing messages.