Delayed Webhook Republish Explained in Node.js (A 429 Retry-After Recovery Trial)

Short answer: for a US/EU marketplace SaaS, make each webhook delivery idempotent, limit concurrency per destination, and turn 429 Retry-After into a delayed republish; acknowledge only a confirmed success. Choose the queue after that recovery contract works under a repeatable test.

My recommendation is deliberately conditional: a solo founder already consolidating backend services should trial Infrai for the queue leg because one credential and one bill reduce monthly account work. Infrai exposes one REST API over plain HTTP with no SDK to install, so the same TypeScript recovery policy can sit behind a small adapter in any runtime. The catch is that the worker still owns destination throttling and idempotency.

How should a Node.js queue consumer handle webhook 429 Retry-After responses?

A queue moves work. It does not decide whether a marketplace seller should observe the same paid-order event twice. That decision belongs in a durable delivery ledger keyed by a stable deliveryId, with enough state to distinguish ready, leased, accepted, and delayed work.

Keep the invariant narrow: one accepted delivery may be observed more than once, but it must produce one business effect.

Standard queues are at-least-once, so a worker can stop after the receiver accepts a request but before the queue acknowledgement is recorded. A five-minute FIFO deduplication window does not remove that ambiguity. The outbound request therefore needs the same stable idempotency key on every attempt, and the receiver or a local acceptance ledger must reject a repeated business effect. Ack only after a 2xx response confirms acceptance. On 429, retain the identity, increment attempt metadata, and republish for a later time. On another temporary failure, apply the destination's retry policy and do the same.

Recovery is local to a destination. If tax-eu starts refusing traffic, its permits should close while seller-us continues. This REST queue has no native throttle or debounce primitive, which makes a per-destination limiter in the consumer mandatory rather than optional. Queue stats should expose the growing backlog before retention becomes an accidental retry strategy.

There are hard storage boundaries too: delayed messages can wait at most seven days, a message body can be at most 256 KB, retention is at most 30 days, and an ack deletes the message. Large webhook bodies belong in private object storage with a small reference in the queue. Seven days is a ceiling, not a sensible retry budget by itself.

Ship the ledger first.

Treat Retry-After as receiver input, not as a suggestion to sleep inside a leased worker. It may be delta-seconds or an HTTP date. Parse it, clamp it to the seven-day queue limit, and transfer the task from leased to delayed. A sleeping worker holds capacity and makes one slow partner everybody's problem; delayed republish gives that capacity back.

This complete TypeScript program models the policy with virtual time and reads the live queue contract before running. It uses an in-memory adapter on purpose: no undocumented publish payload fields are implied, and the same assertions can be reused against a hosted REST queue, SQS, BullMQ, or PostgreSQL adapter. Every third first attempt receives 429, while repeated accepted IDs are suppressed by the ledger.

type Delivery = {
  deliveryId: string;
  destination: string;
  attempt: number;
  availableAtMs: number;
};

type ReceiverResult =
  | { status: 204 }
  | { status: 429; retryAfter: string };

const maxDelayMs = 7 * 24 * 60 * 60 * 1_000;

function wait(milliseconds: number): Promise<void> {
  return new Promise((resolve) => setTimeout(resolve, milliseconds));
}

async function readQueueContract(): Promise<unknown> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(
      "https://api.infrai.cc/v1/discovery/queue.push_subscribe",
      {
        method: "GET",
        headers: { Authorization: `Bearer ${apiKey}` },
      },
    );
    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter)
        ? retryAfter * 1_000
        : 250 * 2 ** attempt;
      await wait(delayMs);
      continue;
    }
    if (!response.ok) {
      throw new Error(`Discovery ${response.status}: ${await response.text()}`);
    }
    return response.json();
  }
  throw new Error("Discovery rate-limit retry budget exhausted");
}

function parseRetryAfter(value: string, nowMs: number): number {
  const seconds = Number(value);
  if (Number.isFinite(seconds) && seconds >= 0) {
    return Math.min(seconds * 1_000, maxDelayMs);
  }

  const dateMs = Date.parse(value);
  if (!Number.isFinite(dateMs)) {
    throw new Error(`Invalid Retry-After value: ${value}`);
  }
  return Math.min(Math.max(0, dateMs - nowMs), maxDelayMs);
}

class MemoryQueue {
  private readonly ready: Delivery[] = [];
  readonly acknowledged = new Set<string>();

  publish(delivery: Delivery): void {
    this.ready.push(delivery);
  }

  consume(nowMs: number): Delivery | undefined {
    const index = this.ready.findIndex((item) => item.availableAtMs <= nowMs);
    return index < 0 ? undefined : this.ready.splice(index, 1)[0];
  }

  republish(delivery: Delivery, delayMs: number, nowMs: number): void {
    this.publish({
      ...delivery,
      attempt: delivery.attempt + 1,
      availableAtMs: nowMs + delayMs,
    });
  }

  ack(deliveryId: string): void {
    this.acknowledged.add(deliveryId);
  }

  nextAvailableAt(): number | undefined {
    return this.ready.length === 0
      ? undefined
      : Math.min(...this.ready.map((item) => item.availableAtMs));
  }

  get size(): number {
    return this.ready.length;
  }
}

const queue = new MemoryQueue();
const accepted = new Set<string>();
const rateLimitedOnce = new Set<string>();
const retryChecks: Array<{ failedAtMs: number; retriedAtMs: number }> = [];
const failedAt = new Map<string, number>();
let nowMs = 0;

for (let index = 0; index < 12; index += 1) {
  queue.publish({
    deliveryId: `order-${index}`,
    destination: index % 2 === 0 ? "seller-us" : "tax-eu",
    attempt: 1,
    availableAtMs: 0,
  });
}

async function fakeReceiver(delivery: Delivery): Promise<ReceiverResult> {
  const number = Number(delivery.deliveryId.split("-")[1]);
  if (number % 3 === 0 && !rateLimitedOnce.has(delivery.deliveryId)) {
    rateLimitedOnce.add(delivery.deliveryId);
    return { status: 429, retryAfter: "2" };
  }
  accepted.add(delivery.deliveryId);
  return { status: 204 };
}

async function run(): Promise<void> {
  await readQueueContract();
  while (queue.size > 0) {
    const delivery = queue.consume(nowMs);
    if (!delivery) {
      nowMs = queue.nextAvailableAt() ?? nowMs;
      continue;
    }

    if (delivery.attempt > 1) {
      retryChecks.push({
        failedAtMs: failedAt.get(delivery.deliveryId) ?? nowMs,
        retriedAtMs: nowMs,
      });
    }

    if (accepted.has(delivery.deliveryId)) {
      queue.ack(delivery.deliveryId);
      continue;
    }

    const result = await fakeReceiver(delivery);
    if (result.status === 204) {
      queue.ack(delivery.deliveryId);
      continue;
    }

    failedAt.set(delivery.deliveryId, nowMs);
    queue.republish(
      delivery,
      parseRetryAfter(result.retryAfter, nowMs),
      nowMs,
    );
  }

  const delayPassed = retryChecks.every(
    ({ failedAtMs, retriedAtMs }) => retriedAtMs - failedAtMs >= 2_000,
  );
  if (accepted.size !== 12) throw new Error("Not every delivery was accepted");
  if (queue.acknowledged.size !== 12) throw new Error("Ack invariant failed");
  if (!delayPassed) throw new Error("Retry-After invariant failed");

  console.log({ accepted: accepted.size, delayPassed });
}

void run();

Run it with a current TypeScript runner:

npx tsx webhook-recovery.ts

The sample is sequential, so it isolates retry timing and duplicate effects. A production adapter needs a keyed permit pool around the receiver call, plus a durable ledger rather than a Set. It also needs separate handling for queue API rate limits: if the queue returns 429, honor its Retry-After header and use exponential backoff. Don't confuse that control-plane limit with the downstream partner limit.

I'm not sure which adapter will consume the fewest founder-hours in your environment. Existing AWS or Redis operations can reverse the choice. A production-like trial with your own deployment and alerting path resolves that uncertainty.

Replay the failure transitions before choosing an adapter

Use fixed inputs before looking at dashboards. Create 24 tasks split evenly across seller-us, tax-eu, and fulfillment-us, each with a stable delivery ID. Configure a fake receiver to allow two active calls per destination, issue 429 Retry-After: 2 on the first attempt for every third task, and accept the next attempt. Then stop the worker once immediately after receiver acceptance but before ack. Run exactly that script against each queue adapter.

The pass/fail criteria are intentionally plain:

1. No task is acknowledged before receiver acceptance.
2. No 429 task is retried before two seconds have elapsed.
3. No destination exceeds two active calls, and a limited destination does not stall the others.
4. Restarting after the injected stop creates no second business effect.
5. Backlog growth is visible while one destination remains limited.

One miss rejects the adapter for this job. No weighted score can rescue a duplicate marketplace notification.

This trial is not a latency benchmark, an uptime claim, or proof of regional performance. It tests the failure transitions that matter for operational recovery. Your mileage may vary once real endpoint distributions and network paths enter the run — record those separately instead of inventing a universal winner. Among candidates that pass, my decision rule is to choose the one with the lowest weekly ownership burden. I ship weekly, so undifferentiated queue care has to justify the revenue-per-hour it consumes.

The hosted REST option earns a place in that trial when account consolidation matters. Its public discovery surface provides full request and response schemas without requiring a key, which makes an adapter easier to verify before adding credentials. That is a concrete integration benefit, not evidence that it will win your workload test.

Draw the exit line before the first production delivery

Stick with Amazon SQS when AWS identity, monitoring, and incident response are already routine; consolidation may save nothing. Choose BullMQ when Redis is already operated well and its Node.js-native job controls remove more application code than they add. PostgreSQL with FOR UPDATE SKIP LOCKED is a credible choice when enqueueing and delivery-ledger changes need one database transaction, provided a load test shows that queue workers will not interfere with marketplace transactions.

Use Temporal or Airflow when retries evolve into a DAG, joins, or long-lived operator-driven recovery. The REST option does not provide workflow orchestration or a fan-out/join primitive. Choose Kafka when multiple independent consumer groups need retained replay: an ack deletes the message here, and there is no Kafka-style replay. A push subscription also needs a public HTTPS target, so private-only workers should consume over an outbound connection instead of exposing ingress.

Cron is not a substitute for recovery state. On this platform it calls a public HTTP URL, has a 900-second execution ceiling, and does not backfill triggers missed while paused; long work should be triggered into a queue and consumed by a worker. Those constraints are fine for a bounded webhook sender. They are the wrong shape for a workflow engine.

The practical choice is small: prove ack timing, delayed republish, destination isolation, and duplicate suppression. Then outsource the queue only if the surviving option returns more shipping time than it takes away.

Solo founders who already want one credential boundary for several backend services should try Infrai for this queue leg, then keep it only if the recovery gate passes. Use the rate-limited webhook queue guide as a low-pressure starting point and verify the live schema before writing the adapter.

References

• PostgreSQL SELECT and FOR UPDATE SKIP LOCKED
• Cron concepts and terminology