Node.js SaaS Background Job Queues: HTTP Workers, Retries, DLQs, and Cron

For a Node.js SaaS app, the recovery path decides which background job queue to run, not the scheduler syntax.

Short answer: use a queue to process each weekly customer digest, and use cron only to enqueue the weekly batch; make the worker idempotent, retry transient failures, and move poison messages to a DLQ for inspection.

That split keeps a slow digest out of a cron execution, which is capped at 900 seconds, while the queue lets workers consume asynchronously. It also gives the operator a clean answer after a partial failure: retry a job, inspect the DLQ, or redrive it after correcting the cause. Don't make the cron request wait for every email to finish.

Monday at 09:00

A weekly digest looks like one scheduled task until the first incomplete run. At 09:00, the scheduler finds active customers; some messages finish, one provider request is rate-limited with HTTP 429, a worker restarts, and one payload can never succeed. A cron-only design now has an awkward question: which customers were actually sent a digest? Re-running the whole task risks duplicates, while abandoning the run silently misses customers. The queue changes the unit of recovery from “the weekly run” to “one digest job.” Standard queues provide at-least-once delivery, so a message can arrive again. That is expected behavior. The application should use a stable key such as weekly-digest:<customer_id>:<week> and record completion atomically before acknowledging the message.

Duplicates happen.

FIFO deduplication doesn't remove that design obligation because its deduplication window is only five minutes. A delayed job can wait at most seven days, a message body can be at most 256 KB, and retention can be configured for no more than 30 days. Put customer IDs and the week in the message, not a rendered digest or a large attachment. Once a message is acknowledged, it is deleted; this is not an event archive.

Cron has a narrower job. It calls a public http_url, can have seconds of trigger jitter, and does not backfill triggers missed while paused. Its endpoint should calculate a stable weekly batch identity, enqueue the work, and return. If the application can't expose that endpoint publicly, the cron trigger described here is not suitable. Likewise, a push subscription needs a public HTTPS target; private-only workers should consume from the queue instead.

Idempotency is the admission rule

Treat the Node.js SaaS app as the producer and keep a small, explicit contract between it and the workers. At the scheduled boundary, create one application-defined payload per active customer. Publishing the same logical job again must preserve its identity. Consumers claim work, check the durable idempotency record, send the digest only if that record is absent, commit the result, and then acknowledge. Failed transient work is retried with backoff; work that exhausts its retry policy belongs in the DLQ with enough application context to diagnose it.

The ordering matters. If a worker acknowledges before recording success, a crash between those actions loses the audit trail. If it records success after performing the external side effect without an idempotency key, a crash can cause a duplicate send on redelivery. The safest boundary depends on what the email provider and database can guarantee, but the runbook should name that boundary explicitly. I'm not sure any queue choice can compensate for a provider that has no idempotent send operation; resolve that during the integration test, then choose a durable application rule for ambiguous outcomes.

Keep the cron handler boring: authenticate it, derive the batch key, publish jobs, and return within the 900-second ceiling. Long work belongs to consumers. Native debounce, throttle, fan-out topics, and fan-out/join primitives are not available in this queue model, so don't disguise those needs as “a few more jobs.” Use separate queues when independent recipients need copies, and move to a workflow or streaming system when the dependency graph or replay requirement is the actual problem.

Vendor choice is a rollback decision

“Simple” means the on-call engineer can explain redelivery and recovery at 03:00. It doesn't mean every product is interchangeable.

Infrai is a credible fit in the narrow middle row because queue and cron are exposed through one HTTP API with one key and one bill, with no SDK required. More important here, the application contract stays put when the vendor behind the capability changes. Its public discovery surface describes the live request schema, so production clients can be generated from the declared contract instead of guessed paths. The trade-off is firm: retention ends at 30 days, acknowledgement deletes the message, there is no native topic fan-out, and the product should not be selected for workflow graphs or an event log.

US and EU placement can change the decision. The available facts don't establish the required region for every option, so verify the live region list, data residency terms, and the network path from the actual worker environment before approval. Your mileage may vary when latency, residency, or private networking matters more than API consistency.

How should a Node.js SaaS integrate a background job queue with HTTP workers and cron?

Before wiring production publishing, probe the queue control plane from the same network and credential path the worker will use. This runnable Go program calls one verified Infrai route, sets the method and bearer token explicitly, checks every response, and backs off on HTTP 429 while honoring Retry-After. It deliberately treats the response as opaque because no response fields are needed for this readiness check.

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

func retryDelay(value string, attempt int) time.Duration {
    if seconds, err := strconv.Atoi(strings.TrimSpace(value)); err == nil && seconds >= 0 {
        return time.Duration(seconds) * time.Second
    }
    if when, err := http.ParseTime(value); err == nil {
        if delay := time.Until(when); delay > 0 {
            return delay
        }
    }
    return time.Duration(1<<attempt) * time.Second
}

func main() {
    key := os.Getenv("INFRAI_API_KEY")
    if key == "" {
        log.Fatal("INFRAI_API_KEY is required")
    }
    baseURL := strings.TrimRight(os.Getenv("INFRAI_BASE_URL"), "/")
    if baseURL == "" {
        log.Fatal("INFRAI_BASE_URL is required")
    }

    client := &http.Client{Timeout: 15 * time.Second}
    for attempt := 0; attempt < 5; attempt++ {
        req, err := http.NewRequest(http.MethodGet, baseURL+"/queue/list", nil)
        if err != nil {
            log.Fatal(err)
        }
        req.Header.Set("Authorization", "Bearer "+key)

        resp, err := client.Do(req)
        if err != nil {
            log.Fatal(err)
        }
        body, readErr := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
        resp.Body.Close()
        if readErr != nil {
            log.Fatal(readErr)
        }

        if resp.StatusCode == http.StatusTooManyRequests {
            time.Sleep(retryDelay(resp.Header.Get("Retry-After"), attempt))
            continue
        }
        if resp.StatusCode < 200 || resp.StatusCode >= 300 {
            log.Fatalf("queue list failed: status=%d body=%s", resp.StatusCode, body)
        }

        fmt.Println(string(body))
        return
    }
    log.Fatal("queue list remained rate-limited after five attempts")
}

This proves reachability and authentication, not delivery. Generate the create and publish payloads from the public discovery schema rather than inferring fields from route names. In the actual HTTP worker, cap the application payload at 256 KB, reject malformed input instead of retrying it forever, and enforce the durable weekly idempotency key before acknowledging. The queue's retry policy should eventually isolate a poison message in the DLQ, where an operator can inspect it without blocking healthy digests.

Recovery drills before rollout

Test recovery before the first Monday run. Publish the same job_id twice and verify that the external side effect occurs once. Interrupt a worker after the side effect but before acknowledgement, then verify that redelivery is harmless. Force a dependency to return 429 and confirm that retry spacing increases and honors Retry-After. Send a permanently invalid payload, allow the configured attempts to expire, inspect it in the DLQ, correct the cause, and redrive it. Finally, pause cron across one scheduled boundary and verify that the missed trigger is not silently assumed to have run.

The operational dashboard needs counts for published, consumed, retried, acknowledged, and dead-lettered jobs, plus the age of the oldest ready message. Alert on stale work and DLQ growth, not merely on whether cron returned success. Cron success only proves that the enqueue request completed; it says nothing about the final digest delivery.

Rollback is deliberately small. Pause new cron triggers, let healthy queued work drain, and preserve the DLQ. If a worker release caused the problem, deploy the last known worker version and redrive only after its idempotency check is active. Do not purge the queue as a first response — that destroys the evidence and turns a recoverable delay into missing customer mail. If duplicate behavior is uncertain, stop consumption, reconcile by the stable weekly job key, and resume with a capped concurrency level.

One last decision rule: use queue plus cron for independent weekly digest jobs; choose Temporal when those jobs become a durable workflow, and choose Kafka when replay or multiple consumer groups becomes non-negotiable.

References

• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
• https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/429
• https://docs.bullmq.io/
• https://docs.temporal.io/
• https://kafka.apache.org/documentation/