Per-URN Schema Validation

Last updated: November 11, 2026

The wire contract freezes the envelope, but deliberately keeps data as “pure JSON the caller validates” — the business payload differs per URN, so the contract cannot fix its shape. URN naming §6 recommends that teams keep a checked-in, per-URN registry with a JSON Schema for each message’s data. This page is the cross-SDK contract for enforcing those schemas (ADR-0024) and the babelqueue-registry / bqschema tooling that governs them.

Status: Authoritative · Opt-in helper + tooling · envelope frozen at schema_version: 1

This is a tooling layer over data, not a wire change. The envelope is untouched; what is validated is the data block, against a schema you register for a URN. It is opt-in: a URN with no registered schema is never validated.

The model: a Provider per URN

Every SDK exposes a Provider (interface / protocol) that answers one question — what is the JSON Schema for this URN’s data? — returning the schema, or “none registered” so the caller skips validation. Two reference providers ship in each SDK:

• MapProvider — in-memory, for tests and for embedding schemas in code.
• DirProvider — the registry bridge: it reads a babelqueue-registry registry.json manifest (a list of {urn, schema} entries) and loads each URN’s schema file lazily. This is what makes the registry’s git-governed schemas enforceable at runtime.

The validator is an intentionally small subset of JSON Schema draft-07 — ten keywords, enough for real data shapes. Zero dependencies (stdlib only). The verdicts are the same in all six SDKs (Go, PHP, Python, Node, Java, .NET) and in the bqschema CLI (see below); the conformance suite locks them.

minLength and maxLength are meant to be integers, as Draft-07 requires. If a schema gives a fractional bound, every SDK (and bqschema) truncates it toward zero ("minLength": 2.5 behaves as 2, "maxLength": -0.5 as 0). A non-finite bound (NaN, Infinity) is ignored, as if the keyword were absent; it can only come from a programmatically built schema, or in Python from the NaN / Infinity literals its json module accepts (standard JSON has none). Any other keyword is unsupported: it never makes a value valid or invalid, and registering the schema warns about it.

Numeric equality in const and enum

Numbers compare by mathematical value, so 1, 1.0 and 1e0 are equal and 2.5 equals 2.50. The rule is recursive: arrays compare element by element, in order, and objects compare by key set and values, with key order irrelevant. Only numbers are loosened: true is not 1, false is not 0, "1" is not 1, and null equals only null. A schema with {"const": 1} therefore accepts a producer’s 1.0. (Node’s JSON.parse does not keep the difference between 1 and 1.0, so the rule holds there trivially.)

Unsupported keywords

A keyword outside the ten above (pattern, format, maximum, maxItems, multipleOf, $ref, definitions, allOf, …) is not enforced. It does not change a verdict — the schema is evaluated as if the keyword were absent — but so that a constraint you wrote is never silently ignored, the SDK emits one warning per occurrence when the schema is registered. A warning is a {pointer, keyword} pair: pointer is the RFC 6901 JSON Pointer of the schema object that carries the keyword ("" for the root, then /properties/<name>, /items, /additionalProperties), keyword is its name. The walk descends only into properties, items and additionalProperties; the value of an unsupported keyword counts as one warning and is not searched, and a property that happens to be called title or pattern does not warn.

Annotations never warn: $schema, $id, title, description, examples, default, $comment, and every keyword that starts with x- (so the x-gdpr-sensitive marker stays silent).

Where the warning goes is each SDK’s own channel; the text is not part of the contract, the {pointer, keyword} set is:

Node, Python, Java and .NET name the URN whose schema produced the warning; PHP’s message does too. Node’s default process.emitWarning is emitted once per distinct message per process, so a second provider built from the same schema does not repeat it (MapProvider.warnings is always complete). DirProvider (Go, PHP, Python) loads a schema file lazily, so its warning appears the first time that URN is used. To enforce a constraint the subset cannot express, validate it in your own code.

The bqschema validate command below uses the same rules and prints the same warnings (one line per occurrence, once per URN) to stderr; they never change its output on stdout or its exit code.

Producer-side Check is preferred

Validation can run on either side, but the sides are not equivalent:

• Producer-side (Check / assert / validate) — preferred. Call it before publishing so invalid data never enters the queue. A violation is caught at the source, where the caller can fix it, and the bad message is never written.
• Consumer-side (Wrap) — a safety net. Wrapping a handler validates each message before it runs. But invalid data will not become valid on retry, so a poison message exhausts its attempts and is dead-lettered. That is correct fail-safe behaviour, but it spends retries on a message that can never succeed — hence producer-side Check is the recommended primary gate, with Wrap as defence in depth.

Go

schema.Check(provider, urn, data) is the producer guard; schema.Wrap(provider, handler) is the consumer safety net. schema.Validate(provider, env) is the envelope form of Check. The package is in the core module (stdlib only):

import "github.com/babelqueue/babelqueue-go/schema"

// DirProvider bridges a babelqueue-registry manifest to runtime validation
provider, err := schema.NewDirProvider("registry.json")
if err != nil {
	return err
}

// producer-side (preferred): keep invalid data out of the queue
if err := schema.Check(provider, "urn:babel:orders:created",
	map[string]any{"order_id": 1042, "amount": 99.90}); err != nil {
	return err // errors.Is(err, schema.ErrInvalidPayload)
}

// consumer-side safety net
app.Handle("urn:babel:orders:created", schema.Wrap(provider, handler))

schema.NewMapProvider(map[string][]byte{...}) builds an in-memory provider for tests.

PHP

BabelQueue\Schema\SchemaValidated mirrors the Go helper — assert (throwing producer guard), check (non-throwing, for branching), and wrap (consumer safety net):

use BabelQueue\Schema\DirProvider;
use BabelQueue\Schema\SchemaValidated;

$provider = new DirProvider('registry.json'); // registry bridge

// producer-side (preferred)
SchemaValidated::assert($provider, 'urn:babel:orders:created', [
    'order_id' => 1042,
    'amount'   => 99.90,
]); // throws InvalidPayloadException on mismatch

// consumer-side safety net
$dispatch->on('urn:babel:orders:created', SchemaValidated::wrap($provider, $handler));

new BabelQueue\Schema\MapProvider([...]) is the in-memory provider.

Python

babelqueue.schema mirrors the same surface — validate (raising producer guard), check (non-raising), and wrap (consumer safety net, taking the URN explicitly since a Python handler receives data positionally):

from babelqueue.schema import DirProvider, validate, wrap

provider = DirProvider("registry.json")  # registry bridge

# producer-side (preferred)
validate(provider, "urn:babel:orders:created",
         {"order_id": 1042, "amount": 99.90})  # raises InvalidPayloadError on mismatch

# consumer-side safety net
app.register("urn:babel:orders:created", wrap(provider, "urn:babel:orders:created", handler))

MapProvider.from_json({...}) builds the in-memory provider from raw schema strings.

Governing schemas: babelqueue-registry and bqschema

The runtime helpers enforce schemas; the babelqueue-registry project governs them. bqschema is a CLI you run in CI — no Kafka, no service, no database. Schemas live in your git repo as files, referenced from a registry.json manifest:

{
  "schemas": [
    { "urn": "urn:babel:orders:created", "schema": "schemas/orders-created.json", "owner": "orders" }
  ]
}

go install github.com/babelqueue/babelqueue-registry/cmd/bqschema@latest

# does this message's data match the schema registered for its URN?
bqschema validate --registry registry.json messages/order-created.json

# is this schema change backward-compatible, or does it break consumers?
bqschema compat schemas/orders-created.json schemas/orders-created.v2.json

# sanity-check the registry itself (every schema parses)
bqschema check --registry registry.json

# generate an AsyncAPI 3.0 event catalog from the registry
bqschema export-asyncapi --registry registry.json -o asyncapi.json

# inventory / audit / mask the GDPR-sensitive fields declared in the registry
bqschema gdpr --registry registry.json                 # inventory the x-gdpr-sensitive paths
bqschema gdpr --registry registry.json --require       # CI gate: fail on un-annotated PII
bqschema gdpr --registry registry.json --mask msg.json # redact PII for safe logging

compat enforces the versioning policy at the data level, the same way the envelope rules work for the wire: adding an optional field is compatible; removing / renaming / retyping a field, or making an optional field required, is breaking — the signal to mint a new URN (…:created.v2) rather than mutate the existing one, so consumers can upgrade before producers. Unlike a broker-coupled schema registry, the schemas are plain git files, so there is no cold-start circular dependency and the gate works identically across Redis, RabbitMQ, SQS, Kafka, Pulsar and the rest.

validate applies the same ten keywords as the SDKs — union type, maxLength, numeric const / enum equality — so a message that passes in CI passes at runtime. compat understands them too: type is compared as a set, so adding a type to a union is compatible and removing one is breaking; lowering maxLength, or adding it to a field that had none, is breaking (the same way raising minLength is). A schema that already declared maxLength or an array type was not enforced by bqschema before v0.5.0; it is now, so data that passed validate under 0.4.x may fail.

GDPR-sensitive fields: declare and audit

The same registry is where PII is declared. An x-gdpr-sensitive keyword on a property (value true or a category string like "email") marks a data field as personal data. It is validation-neutral — parsed but ignored by validate and compat, so annotating an existing schema is never a breaking change — and it flows through into the AsyncAPI catalog automatically. bqschema gdpr inventories the marked paths, --require is a CI gate that fails when a PII-named field is left un-annotated, and --mask redacts a message for safe logging (one-way, registry-side — not a crypto primitive). The matching runtime encryption that reads the same keyword to protect those fields on the wire is the SDK’s job — see GDPR field encryption (ADR-0030).

Per-SDK

Validation is optional and opt-in everywhere — same Provider model, same draft-07 subset, same producer-preferred / consumer-safety-net split. The registry bridge (DirProvider) ships in Go, PHP and Python; the in-memory MapProvider is the path in Node, Java and .NET.

See each SDK’s reference and the babelqueue-registry README for bqschema details: Go, PHP, Python, Node, Java.

Continue to GDPR field encryption.