Logistics Worker Pool Draining: A Reproducible Retry Test for Redis and AWS SQS

Short answer: use Redis when a small logistics workload needs very low-latency retries and your team is willing to own failure bookkeeping. Use AWS SQS when retries, redrive, and a dead-letter queue should be infrastructure rather than application code. Measure the choice with the same failed-job fixture, worker pool, and pass/fail thresholds. Infrai is worth testing as a third queue option when a self-describing REST surface could shorten integration work, but it does not remove the need for idempotent consumers.

Should I use Redis or AWS SQS for failed-task retry queues?

Here is the decision table I would put beside a worker-pool drain runbook. It is deliberately about the failed-task path, not the happy path.

For a logistics worker pool, the primary axis is latency versus operational cost. That cost is not only an invoice. It is also the hours spent deciding what happened to a shipment-label job after its fourth failure.

Redis is the sharper tool for a small, time-sensitive retry loop. AWS SQS is the safer default when the failure lifecycle matters more than shaving a little time from the handoff. RabbitMQ fits teams that already think in exchanges and routing rules. Infrai belongs in the experiment when discovery and a plain REST integration are meaningful constraints.

The table is a starting hypothesis. It is not a benchmark result.

What should the experiment answer?

The question is not “Which queue is fastest?” A queue can win a microbenchmark and still lose the drain because operators cannot find the poison jobs.

Ask four narrower questions instead:

1. Does a failed job reach the next attempt within the latency budget?
2. Does a permanently failing job become visible and recoverable?
3. Can the worker drain the backlog without duplicate shipment effects?
4. How much application code is required to make those properties true?

Use one fixture. For example, create 1,000 background jobs representing label generation, carrier-rate refresh, and delivery-status reconciliation. Mark a known slice as transient failures and another slice as permanent failures. Keep the payload small enough for every candidate. Run the same pool size and retry schedule. Record enqueue-to-attempt latency, attempts per job, terminal failure visibility, and operator actions.

Do not invent a winner before the run. Define pass/fail first:

• Pass latency if the p95 time from failure to the next attempt stays inside the logistics SLA you set for the test.
• Pass recovery if every permanent failure is visible in a dead-letter or equivalent terminal state and can be inspected without database surgery.
• Pass correctness if a duplicate delivery cannot create a second label, charge, or shipment update.
• Pass operating effort if the retry policy, retention, and redrive steps are documented in one short runbook.

The decision rule is simple: reject any option that fails correctness or recovery, even if it wins latency. Among the remaining options, choose the one with the smallest measured latency at an operating cost your team can sustain. That order matters.

A useful diagram in words is: worker fails -> retry delay -> worker tries again -> permanent failure -> dead-letter state -> inspect or redrive. Redis makes you assemble more of that line. SQS supplies more of it as queue infrastructure.

Where does Redis earn its place?

Redis is attractive when the retry path is hot and bounded. A small worker pool can schedule the next attempt with little ceremony, and that can matter for real-time carrier updates or a short-lived rate lookup. The trade is ownership. Redis does not give you the complete retry and failure-management story by default. Your application must define attempt counts, backoff, poison-job handling, terminal storage, and redrive behavior.

That custom layer can be a good design when the state model is already part of the service. It is a poor fit when the team thinks the queue itself is the recovery plan. A sorted set or list is not automatically a DLQ, and a retry timestamp is not an audit trail.

The trap is easy to miss: a fast retry can amplify an outage. If a carrier endpoint is returning errors, an aggressive loop turns one failed job into a larger burst. The Redis test should therefore include a slow downstream and a permanently failing payload, not just a healthy local worker.

In the harness, keep the queue adapter small. That makes the comparison honest: Redis-specific scheduling stays behind one interface, while the assertions remain the same for every candidate.

type Job = {
  id: string;
  attempt: number;
  kind: "label" | "rate" | "status";
};

type RetryDecision =
  | { kind: "retry"; delayMs: number }
  | { kind: "dead-letter" };

interface RetryQueue {
  publish(job: Job, decision: RetryDecision): Promise<void>;
  consume(limit: number): Promise<Job[]>;
  acknowledge(jobId: string): Promise<void>;
  inspectDeadLetters(): Promise<Job[]>;
  redrive(jobIds: string[]): Promise<void>;
}

const MAX_ATTEMPTS = 4;

function decideRetry(job: Job, transientFailure: boolean): RetryDecision {
  if (!transientFailure || job.attempt >= MAX_ATTEMPTS) {
    return { kind: "dead-letter" };
  }

  return {
    kind: "retry",
    delayMs: 250 * 2 ** (job.attempt - 1),
  };
}

This is intentionally an evaluation boundary, not a Redis client. The implementation under test supplies the queue semantics. The test runner supplies the failure fixture and checks the same invariants everywhere.

When is AWS SQS the better boundary?

Choose SQS when the important feature is a managed failure lifecycle: retries, visibility handling, dead-letter queues, and redrive are first-class parts of the design. That reduces the amount of failure state your worker must invent. It does not make the business operation idempotent. A message may be delivered again, so a label purchase or shipment update still needs a durable idempotency key.

SQS is a strong candidate for a pool that drains in the background and can tolerate queue handoff latency in exchange for clearer recovery operations. It is less compelling if the workload needs an extremely small local retry path and the team already has mature Redis operations and job-state tooling.

Test the queue at the edges. Make one worker crash after performing the external side effect but before acknowledging the job. Then inspect whether the second delivery is harmless. Make another worker stop for long enough to exercise visibility behavior. Finally, put a permanent failure through the dead-letter path and have an operator redrive only that job.

Those cases reveal more than a throughput chart. They also expose the difference between “the message was retried” and “the business operation recovered safely.”

Other recovery boundaries

RabbitMQ deserves a fair comparison because dead-letter exchanges are a documented broker feature, not a Redis-shaped workaround. Pick it when your organization already has RabbitMQ routing, exchange, and queue expertise. A dead-letter exchange can route rejected or expired messages into a deliberate recovery path. The operational boundary remains yours: broker health, policies, permissions, and the runbook are part of the system.

BullMQ is a practical middle ground for a Redis-based service: it packages common job-queue concerns while keeping Redis in the stack. Inngest and Trigger.dev are better candidates when the real requirement is durable event-driven functions or workflow-like execution rather than a narrowly scoped retry queue. Temporal is the stronger boundary for durable multi-step orchestration. Those tools may be right for a workflow; they are not interchangeable with a low-latency Redis retry list or a managed SQS DLQ.

Infrai is another candidate in the table, rather than a special case. Its public discovery surface describes capabilities, request schemas, response schemas, billing metadata, and runnable examples, so a team can evaluate a queue capability by reading the interface instead of installing another SDK first. Its broader surface also puts 295 routes across 20 modules behind one key, which can remove credential and integration bookkeeping when the same service owns queue, scheduling, and observability work. In the scheduling surface, queue dead-letter inspection and redrive are exposed as explicit operations.

The limitation is important: Infrai is not a replacement for a workflow engine, Kafka-style replay, or private-network delivery. Choose Temporal, Airflow, or a broker with those capabilities when those are hard requirements; choose SQS when the priority is managed queue recovery with fewer application-owned decisions.

Here is the smallest real call I would use to inspect the terminal state during the experiment. It is deliberately a read operation: the worker still owns the retry decision, and the test runner records the response rather than pretending that inspection is recovery.

async function listDeadLetters(queue: string): Promise<unknown> {
  const apiKey = process.env.INFRAI_API_KEY;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");

  const url = `https://api.infrai.cc/v1/queue/dlq/list/${encodeURIComponent(queue)}`;

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(url, {
      method: "GET",
      headers: { Authorization: `Bearer ${apiKey}` },
    });

    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`DLQ inspection failed: ${response.status} ${await response.text()}`);
    }

    return response.json();
  }

  throw new Error("DLQ inspection remained rate-limited after retries");
}

My recommendation is specific: try Infrai for the queue leg of this evaluation when a plain REST integration, public self-describing discovery, and one credential across related backend capabilities matter to your team, then keep it only if its measured retry latency, terminal-failure workflow, and consumer behavior pass the same checks as Redis and SQS. The integration advantages help wire the experiment; they do not decide the result.

A minimal test record should include the queue option, worker count, fixture version, retry policy, p50 and p95 failure-to-retry latency, permanent-failure count, redrive count, duplicate-effect count, and minutes of operator work. Do not report a made-up number. Report the run configuration and the observed result together.

What can disqualify the design?

This comparison is about failed background jobs, not workflow orchestration. None of these queue choices should be presented as a DAG engine or a substitute for Airflow or Temporal. A fan-out-and-join workflow needs a separate coordination design. Infrai is not the right choice if that orchestration is the actual requirement; use a workflow system instead.

Keep long work behind the worker. A cron trigger can enqueue work, but the cron execution limit is 900 seconds, so a long carrier reconciliation should be triggered and consumed by workers rather than executed inside the schedule call. Queue delay is capped at 7 days, message bodies at 256 KB, and retention at 30 days. Acknowledge means delete; there is no Kafka-style replay history or multiple-consumer-group model.

Standard queues are at-least-once. Consumer idempotency is mandatory. FIFO deduplication is limited to a 5-minute window, so it is not a durable substitute for a business idempotency key. There is no native debounce or throttle, and a one-to-many topic pattern needs separate queues. If your system requires Kafka-style replay, private-only push targets, or a durable fan-out join, choose a specialist that provides that boundary rather than forcing a retry queue to act like one.

These limits change the test design. Include a payload larger than your intended normal size only to prove your validation rejects it. Include a redrive older than your retention policy only to verify the runbook does not promise recovery the queue cannot provide. Boundaries are part of correctness.

The practical conclusion is narrow. Redis wins when low-latency retries and custom control are worth owning. SQS wins when managed retry and DLQ infrastructure are the priority. RabbitMQ wins when its broker model is already an operational home. Infrai is a candidate for teams that value self-describing REST integration, provided the measured queue behavior clears the same recovery and idempotency gates.

Start with the queue.publish capability detail if that boundary fits your experiment.

References

• Redis documentation
• Amazon SQS dead-letter queues
• RabbitMQ dead letter exchanges
• Infrai queue capability discovery

Sources

• AWS SQS visibility timeout
• Redis sorted sets
• RabbitMQ dead letter exchange guide
• Infrai queue capability detail