What is a polyglot queue?

A polyglot queue is a message queue whose jobs are produced in one programming language and consumed natively in another — a PHP app enqueues the work, a Go service runs it, and neither needs the other’s runtime installed. It is not a kind of broker or a product you buy: any Redis, RabbitMQ or SQS queue becomes polyglot the moment the bytes on it are encoded in a format every consumer can read.

That distinction is worth stating plainly, because the thing that stops most queues from being polyglot is not the broker. It is the serializer.

Why an ordinary queue is not polyglot

Your broker is already language-neutral. Redis moves byte strings; it has no opinion about what is inside them. What binds a queue to one runtime is the framework’s default serializer, and nearly every framework defaults to a language-native one.

Laravel’s native queue serializes the job object with PHP’s serialize(). The bytes describing the work look roughly like this:

O:21:"App\Jobs\ProcessOrder":1:{s:8:"order_id";i:1042;}

Three separate problems are packed into that one line:

• The identity is a class name. App\Jobs\ProcessOrder is a path in a PHP source tree.
• The types are PHP’s. O: means “a PHP object”; the payload is an object graph, not data.
• The format is PHP’s. Only a PHP parser reconstructs it.

Every ecosystem has its own version. Python pickle writes a byte stream that references a module path and rebuilds the object by importing it. Java’s ObjectOutputStream writes a class name plus a serialVersionUID. .NET had BinaryFormatter, which Microsoft has since deprecated and removed from the runtime, partly over the risks below.

They differ in encoding and agree on the fatal part: they serialize an object rather than data, so identity is a class, the payload is a runtime’s type system, and the format is a runtime’s private business.

The failure modes teams actually hit

This stays invisible while there is one language in the system. It surfaces all at once when a second one shows up.

The second service cannot decode the message at all. This is not a bug in the consumer and no amount of work on the consumer side fixes it — the bytes genuinely do not describe anything a Go program can build. The usual first response is to write a bridge service that reads one format and republishes another, which means a new deployable, a new single point of failure, and a bespoke format maintained by one team.

Your source tree becomes the wire contract. Because identity is a class name, renaming App\Jobs\ProcessOrder or moving it to another namespace breaks every message already sitting on the queue and every message another service enqueued before the deploy. Ordinary refactors turn into coordinated releases, and deploy windows become breakage windows.

Deserializing untrusted bytes is a code-execution surface. Python’s own documentation warns that unpickling data can execute arbitrary code; Java deserialization gadget chains have been a recurring vulnerability class for a decade; PHP unserialize() on attacker-influenced input has the same shape. When queue payloads are object graphs, anything that can write to your queue is close to being able to run code in your workers.

Nothing correlates across the boundary. A language-native format carries whatever the runtime happened to embed, so there is no agreed field tying a message to the request that caused it. When a job fails two services downstream, you have a stack trace and no way back to the origin.

You cannot read your own queue. redis-cli LRANGE returns a blob only one runtime can open. The broker’s management UI shows nothing useful, and a dead-letter queue full of those payloads is a pile of bytes rather than a triage tool.

The workarounds compound. Teams end up either owning a translation service forever, or adopting a whole new streaming platform to solve what was a serialization problem — new infrastructure, a new dispatch model, and the framework’s worker left behind.

What a solution actually has to do

Make the queue polyglot and every one of those problems goes away at the source. That takes surprisingly little, but the little it takes must be strict.

1. A language-neutral payload format. JSON is the pragmatic default: every language parses it, and a human can read it in a broker UI. The format matters less than the strictness — the encoding rules have to be pinned, or you trade an obvious interop failure for a subtle one. Fix UTF-8, ban NaN/Infinity, state the integer range (and what happens when a JavaScript consumer is in the chain, where 2^53 − 1 is the real ceiling), forbid floats for currency, decide how binary and timestamps are represented. A payload that “is JSON” but silently rounds a float or truncates a 64-bit id is not interoperable.

2. Identity as a string, not a type. The message has to name what it is with a stable string that each consumer maps to its own handler, in its own language, through its own registry. Decoupling identity from the source tree is what makes both cross-language routing and ordinary refactoring possible — the string is the contract, the class is an implementation detail.

3. A correlation id preserved across every hop. Minted once by the first producer and forwarded unchanged by everything that touches the chain. Without it, polyglot means “another language can decode it” but not “another language can debug it.”

4. A version field on the format itself, so a consumer that meets bytes it does not understand rejects or quarantines them loudly instead of best-effort parsing them into corrupt state.

5. An agreed destination for failures — a dead-letter queue holding the same neutral format, so whichever team is on call can read a failure that another language produced.

None of this requires a new broker, a sidecar, or a rewrite of your workers. The queue you already run moves bytes; what changes is the encoder and the routing key.

One implementation

BabelQueue is one way to get those five properties, and a useful way to see them concretely. Every SDK — PHP, Python, Go, Node.js, Java and .NET — produces and consumes exactly this document:

{
  "job": "urn:babel:orders:created",
  "trace_id": "7b3f9c2a-e41d-4f88-9b2a-1c0d5e6f7a8b",
  "data": { "order_id": 1042 },
  "meta": {
    "id": "f1e2d3c4-b5a6-4789-90ab-cdef01234567",
    "queue": "orders",
    "lang": "php",
    "schema_version": 1,
    "created_at": 1749132727000
  },
  "attempts": 0
}

Map that back to the list. data is the neutral payload — a pure JSON object, with the number, time and binary rules written into the contract rather than left to each SDK. job is the identity: a URN such as urn:babel:orders:created, never a class name, which each consumer maps to its own handler. trace_id is the correlation id, minted by the first producer and forwarded unchanged across every hop. meta.schema_version is the version field, frozen at 1, which consumers must reject or quarantine when they do not recognize it. Failures go to <queue>.dlq carrying the same canonical envelope plus an additive dead_letter block. The one extra field, top-level attempts, is the transport’s retry counter — the only value a broker or worker is allowed to change.

On the producing side that is a one-liner, and the broker connection does not change:

BabelQueue::publish('urn:babel:orders:created', ['order_id' => 1042]);

A Go, Python, Java, .NET or Node worker reads the identical bytes off the same queue and routes on the string urn:babel:orders:created, sharing no type with the producer. Rename the PHP class that afternoon and nothing downstream notices.

Two things make that hold in practice rather than on paper. The envelope is frozen at schema_version: 1 — additive optional fields keep the version, while removing or retyping anything requires a deliberate bump and a migration — and every SDK runs a shared conformance suite of golden envelopes in CI, so two SDKs that pass it can read each other’s messages. A standard that is only asserted drifts; one that is executed in every repository’s CI does not.

What a polyglot queue does not give you

Being honest about the boundary is part of the definition:

• It is not a streaming platform. No replayable log, no partitioned ordering, no consumer groups. If you need those, you need Kafka, and a neutral envelope is not a substitute.
• It does not make your domain neutral. Producer and consumer still have to agree on what data means behind a given identity. The envelope makes the transport language-agnostic; the payload’s meaning is a contract your teams own, ideally with a per-identity schema.
• It buys a single-language system nothing. If PHP produces and PHP consumes, serialize() is fine and switching costs you effort for no gain. The value appears exactly when a second runtime does.
• Delivery is still at-least-once. A neutral format does not change your broker’s semantics, so handlers must stay idempotent.

Takeaway

A polyglot queue is an ordinary queue whose messages are encoded so that any language can read them: strict neutral data instead of an object graph, a stable string identity instead of a class name, a correlation id that survives every hop, a version field, and a failure destination in the same format. The broker does not change and neither does your worker — only the serializer does.

Definitions for the vocabulary in this post are in the glossary; the exact bytes are in the wire contract. For a worked example, see PHP produces, Go consumes — over Redis.