Dead-Letter Replay Does Not Belong on Free Inference

Live replay of a dead-letter message is a control-plane act. Free inference should not own it.

A dead-letter queue is not a suggestion box. It is a holding pen for work that already failed, often after a side effect has been attempted. Replay means the consumer will call a payment API, emit a webhook, mutate inventory, or send mail again. The payload may be poison. It may be a duplicate of a request that already committed. It may be a timeout whose remote party succeeded while the local waiter expired. Those cases do not share a prompt. They share a contract, and the contract has to survive a model outage.

Teams reach for a chat completion because the dump looks linguistic. Error strings wander. Partner codes collide. A model can cluster the mess into a tidy story. That clustering is useful on a desk. It is a poor runtime for a switch that re-enters the world.

Think of the DLQ as a loaded elevator. Classification is the inspection tag on the door. Replay is the button that sends the car. Handing the button to a volunteer intern with a sticky note is the free-inference pattern: cheap, fluent, and unbound to the interlock.

The failure mode is not only a wrong label. Free inference is a best-effort path. It can stall, rate-limit, or answer with a different JSON shape than yesterday. A consumer that blocks on that path turns a message bus into a chat session. A consumer that proceeds on timeout turns a maybe into a charge. Neither is an SLO.

A replay gate belongs in deterministic code next to the consumer. The gate reads durable fields: topic, error class, attempt count, idempotency key, and a human-committed policy table. It returns one of three verbs. never parks the message for a ticket. replay_once requeues with a fresh attempt counter and the same key. hold waits for an operator. The model, if it appears at all, writes a proposal file. It does not call the broker.

The artifact below is that gate. It is ordinary Python. It is meant to be copied into a worker repo and tested without a network.

# replay_gate.py
from dataclasses import dataclass
from enum import Enum
from typing import Mapping, Optional, Tuple

class Verb(str, Enum):
    NEVER = "never"
    REPLAY_ONCE = "replay_once"
    HOLD = "hold"

@dataclass(frozen=True)
class DeadLetter:
    topic: str
    error_class: str
    attempts: int
    idempotency_key: str
    body_hash: str
    already_committed: bool

PolicyKey = Tuple[str, str]
POLICY: Mapping[PolicyKey, Verb] = {
    ("payment.capture", "timeout"): Verb.REPLAY_ONCE,
    ("payment.capture", "insufficient_funds"): Verb.NEVER,
    ("payment.capture", "duplicate"): Verb.NEVER,
    ("inventory.reserve", "lock_timeout"): Verb.REPLAY_ONCE,
    ("inventory.reserve", "overbook"): Verb.NEVER,
    ("webhook.partner", "http_503"): Verb.REPLAY_ONCE,
    ("webhook.partner", "http_400"): Verb.NEVER,
    ("mail.receipt", "smtp_421"): Verb.REPLAY_ONCE,
    ("mail.receipt", "unknown_recipient"): Verb.NEVER,
}

MAX_ATTEMPTS = 3

def decide(letter: DeadLetter, policy: Mapping[PolicyKey, Verb] = POLICY) -> Verb:
    if not letter.idempotency_key:
        return Verb.HOLD
    if letter.already_committed:
        return Verb.NEVER
    if letter.attempts >= MAX_ATTEMPTS:
        return Verb.HOLD
    verb = policy.get((letter.topic, letter.error_class))
    if verb is None:
        return Verb.HOLD
    return verb

def apply(letter: DeadLetter, broker, audit) -> Verb:
    verb = decide(letter)
    audit.record(letter.body_hash, verb.value)
    if verb is Verb.REPLAY_ONCE:
        broker.requeue(letter, extra_headers={"x-replay": "1"})
    return verb

The unknown row is the whole point. A missing (topic, error_class) pair does not become a prompt. It becomes hold. That is how a new partner code fails closed instead of becoming a creative retry. The already_committed bit is equally blunt. If the outbox, the payment ledger, or the partner receipt says the work landed, replay is vandalism dressed as healing.

Tests pin the verbs. They do not pin a temperature.

# test_replay_gate.py
from replay_gate import DeadLetter, Verb, decide

def letter(**kwargs):
    base = dict(
        topic="payment.capture",
        error_class="timeout",
        attempts=1,
        idempotency_key="cap_9f3a",
        body_hash="sha256:abc",
        already_committed=False,
    )
    base.update(kwargs)
    return DeadLetter(**base)

def test_timeout_replays_once():
    assert decide(letter()) is Verb.REPLAY_ONCE

def test_committed_work_never_replays():
    assert decide(letter(already_committed=True)) is Verb.NEVER

def test_unknown_error_holds():
    assert decide(letter(error_class="weird_partner_code")) is Verb.HOLD

def test_missing_idempotency_key_holds():
    assert decide(letter(idempotency_key="")) is Verb.HOLD

def test_insufficient_funds_never_replays():
    assert decide(letter(error_class="insufficient_funds")) is Verb.NEVER

Run them with python -m pytest test_replay_gate.py -q. The suite is the oracle. A model that restates the same rules in prose is commentary, not control.

Free inference still has a desk job. Unstructured DLQ dumps are noisy, and a first pass that clusters raw error strings into candidate (topic, error_class) rows can save an afternoon. Disclosure: This article was prepared as part of MonkeyCode's product outreach. A workspace with free model access and a free server option can sit beside those dumps, emit a proposed policy patch, and stop there. The patch lands in review like any other config change. The consumer never calls the model on the hot path.

That rehearsal looks like a file, not a tool call inside apply.

# propose_policy.py — offline sketch, not a worker dependency
# Label as unexecuted until a human copies rows into POLICY.
PROPOSAL_SCHEMA = {
    "type": "object",
    "required": ["topic", "error_class", "verb", "rationale"],
    "properties": {
        "topic": {"type": "string"},
        "error_class": {"type": "string"},
        "verb": {"enum": ["never", "replay_once", "hold"]},
        "rationale": {"type": "string"},
    },
}

def write_proposal(path, rows):
    # Persist proposals for review. Do not import this module from the consumer.
    import json
    from pathlib import Path
    Path(path).write_text(json.dumps(rows, indent=2), encoding="utf-8")

Red flags show up before the first wrong charge. The consumer imports an SDK in the same process that acknowledges Kafka or SQS. Replay waits on a completion token. Two identical payloads, five minutes apart, receive two verbs. The policy lives only in a system prompt. The audit log stores the model's paragraph and not the verb. Any one of those is enough to pull the model out of the loop.

Better alternatives are boring on purpose. Map partner error codes at the edge. Persist an idempotency key before the first attempt, not after the failure. Record already_committed from the ledger, not from a summary of logs. Cap attempts in the broker. Put unknown classes on hold and page a human. If the dump is truly unstructured, run the clusterer on a sample in a scratch environment, then type the surviving rows into POLICY.

Exit criteria belong next to the policy, not in a runbook nobody opens. Stop using even the offline proposer when the same payload yields two different verbs across sessions. Stop when proposal latency exceeds the time a reviewer would spend reading twenty raw lines. Stop when the topic moves money, identity, or medical data through a third party that is not under the team's data agreement. Stop when the DLQ volume is a load test in disguise: a replay storm is a traffic generator, and a chat API is not a traffic shaper.

Who should not use this approach is as important as the gate. A team without idempotency keys should not replay at all, with or without a model. A team whose ledger cannot answer already_committed should park every payment topic on hold. A team that needs sub-second classification under partition loss should not add a network hop that is allowed to vanish. A regulated workload that cannot put payloads on a shared free endpoint should keep the dumps inside the boundary and type the table by hand.

Limitations of the gate itself are real. The table lags a new partner. error_class is only as good as the parser that produces it. MAX_ATTEMPTS does not know about downstream quotas. HOLD can hide a growing pile if nobody pages. The design accepts those limits because they are visible. A fluent paragraph is not visible in the same way. It fails like weather.

The current wave of agent demos makes the anti-pattern tempting. Tool calling looks like a universal adapter: feed the DLQ body in, get a verb out, let the agent requeue. That adapter is a completion. Completions are not leases, not locks, and not ledgers. They are a way to draft the table that the lease, the lock, and the ledger already require.

A free model and a free server are enough to rehearse that table against fixtures. They are not a substitute for the switch. Keep the intern at the desk. Keep the elevator button in code.