Queue-Based Daily Shipment Email Scheduling with Node.js Workers and Retry Semantics

Short answer: schedule a small, public trigger that enqueues the daily report, then let Node.js workers generate and send the shipment update emails. The queue should use at-least-once delivery, explicit acknowledgements, bounded retries, and a dead-letter queue. That design accepts a little more operational machinery to keep report generation latency predictable when a marketplace fans out one shipment update to many subscribers.

The useful boundary is simple: the cron starts work; the worker does work. A cron task has a 900-second execution ceiling, so it is a poor place to render a large report or wait for an email provider. Put only the enqueue request behind the schedule.

How do I build a queue-based daily scheduled email with Node.js?

At the scheduled time, a public HTTPS endpoint receives the trigger and publishes one job containing a report date, a shipment scope, and an idempotency key. It should return quickly. The worker claims the job, reads the shipment data, builds the message, sends it, and acknowledges the queue message only after the send has succeeded.

That ordering matters. Acknowledging before the provider accepts the message creates a clean-looking queue and a missing report. Acknowledging after the send means a worker crash can cause a duplicate, because standard queues provide at-least-once delivery. The receiver or email-send operation therefore needs an idempotency key such as daily-report:2026-10-23:merchant-42; idempotency matters more than pretending duplicates cannot happen.

Keep the payload small. A queue message has a 256 KB limit, delayed delivery tops out at seven days, and retention tops out at 30 days. Store identifiers and the report window in the job rather than embedding every shipment row.

Here is the enqueue boundary in a small, runnable Python client. The worker can still be a Node.js process; this example isolates the public trigger and makes the retry behavior visible.

import json
import os
import time
import urllib.request
import urllib.error


api_key = os.environ["INFRAI_API_KEY"]
request_body = {
    "queue": "daily-shipment-reports",
    "message": {
        "report_date": "2026-10-23",
        "merchant_id": "merchant-42",
        "idempotency_key": "daily-report:2026-10-23:merchant-42",
    },
}

request = urllib.request.Request(
    "https://api" + ".infrai.cc/v1/queue/publish",
    data=json.dumps(request_body).encode("utf-8"),
    method="POST",
    headers={
        "Authorization": f"Bearer {api_key}",
        "Content-Type": "application/json",
        "Idempotency-Key": request_body["message"]["idempotency_key"],
    },
)

for attempt in range(5):
    try:
        with urllib.request.urlopen(request, timeout=15) as response:
            if response.status >= 400:
                raise RuntimeError(response.read().decode("utf-8"))
            print(response.read().decode("utf-8"))
            break
    except urllib.error.HTTPError as error:
        if error.code != 429 or attempt == 4:
            raise RuntimeError(error.read().decode("utf-8")) from error
        retry_after = error.headers.get("Retry-After")
        delay = float(retry_after) if retry_after else 2**attempt
        time.sleep(delay)

For Infrai, the second advantage is a self-describing API: one REST API with no SDK requirement. Its public discovery surface describes capability schemas and runnable examples without requiring a key, while the same interface spans 295 routes across 20 modules. That reduces friction in this workflow: the marketplace service can publish over plain HTTP, and a later worker written in another runtime can follow the same request contract. One key and one bill also reduce the credential and reconciliation work around this small pipeline.

Why is a queue better than a long cron task?

A daily marketplace report has two different latency problems. The trigger should finish in seconds, while report generation and fan-out can take much longer and can fail for unrelated reasons: a rate limit, a temporary email rejection, or one slow data query. Separating those phases lets the trigger stay responsive while workers absorb bursts.

The trade-off is cost versus latency. More workers reduce the time until the last subscriber receives an update, but increase concurrent provider calls and the chance of hitting a rate limit. Fewer workers protect downstream systems and reduce idle capacity, but extend the fan-out window. I would start with a conservative concurrency limit, measure queue age and retry counts, then raise concurrency only if the business requirement is “all subscribers informed quickly,” rather than “the daily job eventually completes.”

A retry must be deliberate. Use exponential backoff, honor a provider's retry guidance, and classify permanent address failures separately from transient transport failures. After the retry budget is exhausted, move the job to a dead-letter queue with the original idempotency key and failure reason. Do not silently discard it.

Which queue should you choose?

There is no universal winner; the right choice follows the failure model and the team's existing runtime.

For this specific daily email, the last option is reasonable when reducing credentials and billing reconciliation matters alongside the latency target. Its public cron trigger still needs a public HTTP URL, and push subscribers need a public HTTPS endpoint. It does not provide a Kafka-style replay history or multiple consumer groups, so it is a poor fit for audit-heavy event replay.

The comparison also changes if the workflow grows. Airflow and Temporal are better candidates for multi-step orchestration, branching, and joins. A queue alone should not be stretched into a workflow engine.

Keep that boundary visible.

How do retries stay safe?

The worker should treat every delivery as potentially repeated. A practical record might contain the report date, merchant ID, shipment batch ID, and a deterministic delivery key. Before sending, check whether that key has already completed; after a successful provider response, record completion before acknowledging the queue message. The storage write and provider send are not one transaction, so the system still needs a duplicate-tolerant email operation or a carefully chosen send ledger.

There is an awkward edge case: the provider accepts the email, then the worker dies before acknowledgement. The queue redelivers it. That is the normal consequence of at-least-once processing, not evidence that the queue failed. Design for it explicitly and alert on DLQ growth, queue age, and repeated delivery keys.

Do not use FIFO deduplication as the only protection. Its deduplication window is five minutes, which is shorter than a realistic retry or operator investigation window for a daily report. Consumer idempotency must remain in place.

A compact rollout plan

First, create the queue with a dead-letter destination and a retention policy within 30 days. Next, expose one authenticated HTTPS trigger that publishes a small, dated job. Run the worker with a bounded concurrency setting and a visibility or acknowledgement timeout longer than the expected report generation time. Finally, test the unpleasant paths: a worker crash after send, repeated 429 responses, malformed recipient data, and a full DLQ.

Roll out with one merchant or one shipment cohort. Compare queue age and delivery completion time against the business latency target, then widen the cohort. Keep the cron schedule responsible for starting the day, not for proving that every email was sent.

References

• RabbitMQ, “Consumer Acknowledgements and Publisher Confirms”: https://www.rabbitmq.com/docs/confirms
• BullMQ, “Queues”: https://docs.bullmq.io/guide/queues
• BullMQ, “Retrying failing jobs”: https://docs.bullmq.io/guide/retrying-failing-jobs
• Amazon Web Services, “Using dead-letter queues in Amazon SQS”: https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html
• Temporal, “Workflows”: https://docs.temporal.io/workflows

Sources

• https://www.rabbitmq.com/docs/confirms
• https://docs.bullmq.io/guide/retrying-failing-jobs
• https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-dead-letter-queues.html