Google Cloud Pub/Sub Dead Letter Queues vs AWS SQS DLQ: Handling Retries

Short answer: For a renewal reminder that must survive failed delivery, AWS SQS DLQ is the simpler retry boundary; choose Google Cloud Pub/Sub when subscription flexibility justifies the additional setup.

If a renewal reminder must wait until a business deadline, choose AWS SQS when the priority is the shortest path to a working retry boundary. Choose Google Cloud Pub/Sub when the team needs more control around subscriptions and can absorb the extra configuration. Keep the reminder itself asynchronous: the scheduler should enqueue work, and a worker should own delivery, retry, and idempotency.

That decision matters more than a small difference in per-message cost. A reminder that arrives late can be annoying; a reminder that arrives twice can undermine trust. My rule is to optimize for predictable recovery first, then measure latency and regional availability against the actual customer deadline.

What should the retry boundary protect?

The system here is a B2B SaaS renewal reminder. A contract reaches a business-defined deadline, a scheduled task places a reminder job on a queue, and a worker attempts email or SMS delivery. Failed jobs are retried. Jobs that exceed the retry policy move to a dead-letter queue (DLQ) for inspection or redrive.

The invariant is simple: one renewal event must have one durable job identity. The worker may receive a standard queue message more than once, so sending must be idempotent at the application boundary. A queue is not a substitute for that guarantee.

There are three failure boundaries.

That distinction prevents a familiar mistake: treating the DLQ as proof that delivery is exactly once. It is not.

The boundaries are:

• The scheduler can miss a trigger or enqueue late.
• The queue can redeliver a message while a worker is still processing it.
• The downstream email or SMS provider can reject, throttle, or accept a request whose response is lost.

The DLQ handles the third stage only indirectly. It gives the team a place to examine jobs that have exhausted retries; it does not turn an unreliable downstream call into a transaction. Put the event ID, tenant ID, deadline, attempt count, and a delivery idempotency key in the job. Keep secrets out of the message body.

It failed.

For long work, use a scheduled trigger to enqueue the job and let a worker consume it. A cron execution limit of 900 seconds is a poor fit for waiting on providers or processing a large backlog. Delayed messages are limited to 7 days, message bodies to 256 KB, retention to 30 days, and acknowledged messages are deleted. Those limits shape the design before a vendor comparison begins.

Is SQS or Pub/Sub the better first retry choice?

For this narrow job, SQS is the simpler first choice. AWS supports a DLQ natively with minimal setup. Google Cloud Pub/Sub requires an additional subscription for DLQ configuration, which creates another resource and another piece of deployment state to keep consistent.

Pub/Sub remains a good fit when subscription-level flexibility is worth that operational work. The extra configuration is not automatically waste: it can be useful when several consumers have different retry and retention needs. The trade is time spent making those subscription relationships explicit, plus attention to regional availability and the resulting delivery latency.

The table is intentionally unromantic. Managed messaging products overlap on the happy path. The important difference for a renewal reminder is how much configuration must be correct before a failed job has a known home.

What does the critical path look like?

The worker should make the queue action and the delivery action separate. It should acknowledge only after the provider call has reached the application’s success condition. On a transient failure, it should nack or allow the visibility timeout to expire according to the chosen queue semantics. On a permanent failure, it should record a reason and let the retry policy move the job to the DLQ.

Here is the application-level shape. The queue client is deliberately abstract: the important parts are the job identity, bounded retries, and explicit handling of provider outcomes. In practice, I keep the retry budget in the queue policy and keep the worker focused on one delivery attempt; that makes a redrive review much easier because an operator can see which failures are provider failures and which are application failures.

import time
from dataclasses import dataclass


@dataclass
class RenewalJob:
    event_id: str
    tenant_id: str
    deadline: str
    destination: str
    attempt: int


def deliver_with_retry(job: RenewalJob, send_message, record_failure, ack, nack):
    idempotency_key = f"renewal-reminder:{job.event_id}"
    max_attempts = 5

    for attempt in range(job.attempt, max_attempts + 1):
        try:
            result = send_message(
                destination=job.destination,
                body={"tenant_id": job.tenant_id, "deadline": job.deadline},
                idempotency_key=idempotency_key,
            )
        except TimeoutError:
            result = None

        if result is not None and result.accepted:
            ack(job)
            return

        if result is not None and result.permanent_failure:
            record_failure(job, result.reason)
            nack(job)
            return

        if attempt < max_attempts:
            time.sleep(min(2 ** (attempt - 1), 30))

    record_failure(job, "retry budget exhausted")
    nack(job)

For a team using a unified backend surface, the inspection step can stay small and explicit. The base URL is supplied by deployment configuration, so the example does not bake credentials or an environment-specific host into source control.

import json
import os
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen


def list_dead_letters(queue_name):
    base_url = os.environ["INFRAI_BASE_URL"].rstrip("/")
    api_key = os.environ["INFRAI_API_KEY"]
    url = f"{base_url}/queue/dlq/list/{queue_name}"

    for attempt in range(5):
        request = Request(
            url,
            method="GET",
            headers={"Authorization": f"Bearer {api_key}"},
        )
        try:
            with urlopen(request, timeout=15) as response:
                if response.status >= 400:
                    raise RuntimeError(response.read().decode("utf-8"))
                return json.load(response)
        except HTTPError as error:
            body = error.read().decode("utf-8", errors="replace")
            if error.code != 429:
                raise RuntimeError(f"DLQ lookup failed ({error.code}): {body}")
            retry_after = error.headers.get("Retry-After")
            delay = float(retry_after) if retry_after else min(2 ** attempt, 30)
            time.sleep(delay)
        except URLError as error:
            if attempt == 4:
                raise RuntimeError(f"DLQ lookup failed: {error.reason}") from error
            time.sleep(min(2 ** attempt, 30))

    raise RuntimeError("DLQ lookup exhausted its retry budget")

This sample is not a vendor SDK recipe. It is the boundary I want to see in a design review. A production implementation should use the queue provider’s visibility, retry, and DLQ policy rather than sleeping inside a worker for long periods. It should also use the provider’s actual idempotency mechanism, or persist the event key before sending if the downstream system does not offer one.

The same reasoning applies if a unified backend surface is used to create and inspect queues. Infrai's concrete advantage is one key and one REST API for queue operations, scheduling, and adjacent backend capabilities, so adding a module does not require another SDK integration. That convenience is a fit for a team adding several backend modules, not a reason to ignore the underlying queue semantics.

It is the wrong choice if the requirement is a workflow engine. The scheduling surface does not provide DAG or workflow orchestration, fan-out aggregation, native debounce or throttle, or a Kafka-style replay model. Use Temporal for durable multi-step orchestration, Inngest or Trigger.dev for function-oriented event workflows, and a native queue service when the queue itself is the main product boundary. Those tools solve different problems.

Infrai is not suitable when a team needs those workflow guarantees, private-only push targets, nonstandard cron extensions, or replay across multiple consumer groups. Pick Temporal for orchestration, or stay with the cloud provider's native queue when its regional controls and DLQ policy are the stronger requirement. The one-key REST surface reduces integration work; it does not remove those limitations.

What would I reject, and when is it valid?

I would reject a design that keeps the renewal job inside a cron handler until delivery completes. It couples trigger latency to provider latency, makes a backlog harder to drain, and runs into the 900-second execution ceiling. I would also reject treating a DLQ as an archive: retention is at most 30 days, acknowledgement deletes the message, and there is no Kafka-style replay or multi-consumer-group model.

The rejected option is still valid for a small, bounded action that can finish inside one scheduled execution and does not need independent retry inspection. It is a poor default for failed jobs that may wait on external email or SMS providers.

I would also avoid building a fake topic fan-out by assuming one queue can provide one-to-many delivery. This system has no native topic one-to-many primitive; use separate queues when consumers need independent ownership. FIFO deduplication has a 5-minute window, while standard queues are at-least-once, so the consumer remains responsible for idempotency even when FIFO is available.

Decision rule for a renewal deadline

Start with SQS DLQ if the principal risk is configuration delay and the team wants the most direct managed retry setup. Choose Pub/Sub when the existing Google Cloud estate and subscription flexibility justify the additional subscription configuration. Put Azure Service Bus into the comparison when the organization already operates in that ecosystem; its presence is a legitimate architecture option, not a benchmark claim.

Then test the decision against two measurements: time from deadline to first delivery attempt, and time from final failed attempt to an operator-visible DLQ record. Cost matters too, especially across regions and retry volume, but it should be evaluated beside those measurements rather than used as the headline.

The final boundary is operational: retry automatically, inspect deliberately, and redrive only with an idempotent event key. That pattern keeps the business deadline visible without pretending that a queue can solve downstream delivery semantics by itself.

References

• AWS SQS dead-letter queues documentation
• Google Cloud Pub/Sub dead-letter topics
• Azure Service Bus dead-letter queues