2026 Cron Setup for Periodic Database Cleanup and Shipment Fanout

The least complex setup for periodic database cleanup is a cron trigger that calls a short, idempotent HTTP handler. For a logistics system that must fan out shipment updates, use the same scheduler to start bounded work, then hand longer delivery work to a queue and an idempotent consumer. That keeps a queue consumer out of the cleanup path while preserving a recovery path for the notification path.

The difficult decision is not the cron expression. It is what you retain after a shipment update has been delivered, and what you can prove when a subscriber says it was missed.

That is the whole reason to design the retention boundary first.

What is the bill actually made of?

For this workload, the visible bill is made of retained message bytes, requests, and database or egress work behind the handler. The schedule itself is rarely the dominant design term. A 256 KB event retained for 30 days is a very different operational object from a compact shipment state record deleted after acknowledgement, even though both can be sent by the same cron trigger.

I start with retention because it exposes an uncomfortable trade-off: keeping every delivery attempt makes diagnosis easier, while keeping less data limits storage and replay surface. Queue messages can be delayed for at most 7 days and retained for at most 30 days in this design. Acknowledgement deletes a message, so an acknowledgement is also a decision to give up that copy as a recovery artifact.

For a shipment update, I would retain a database record containing the shipment ID, event ID, subscriber ID, status, and a small error reason. I would not put a full tracking history or an oversized document in the queue message. The 256 KB message ceiling is a boundary, not a target. Store durable state separately and let the message carry the identifier needed to fetch it.

That change moves the dominant term from repeated payload retention to bounded state plus delivery attempts. What do I deliberately stop keeping? The acknowledged queue copy and the full response body. When a subscriber disputes delivery later, I lose the queue payload itself; the durable delivery record and request ID must carry the evidence instead.

How should a Node.js cron setup handle periodic database cleanup?

Cron is the right owner for periodic database cleanup when the job is short, time-based, and does not need a long-running process. It gives an easy, reliable trigger for an HTTP endpoint. A queue consumer is useful after the trigger, when one shipment update needs independent delivery attempts or the cleanup query may take longer than the cron execution limit.

The limit matters: a cron execution can run for no more than 900 seconds. I initially treated that as a timeout setting; it is really a boundary around the shape of the job. A cleanup task that might scan old delivery records for longer should be split into two phases: cron starts the work, and a worker consumes a queue item. The worker must be idempotent because the standard queue is at-least-once. A retry that inserts a second delivery record or sends a second irreversible notification is a data bug, not a scheduling detail.

A practical decision rule is short enough to remember:

• Use cron directly for a bounded cleanup request that can finish inside 900 seconds.
• Use cron to enqueue work when cleanup or fanout can exceed that limit.
• Give every cleanup batch and delivery attempt a deterministic idempotency key.
• Treat missed triggers as missed work: cron does not automatically backfill a paused schedule.

The schedule has second-level jitter, and run output keeps only the first 4 KB. Those facts make a database run ledger more useful than relying on scheduler output as the audit trail.

How should the retention boundary shape delivery guarantees?

The guarantees are easy to overstate. A standard queue supplies at-least-once delivery, not exactly-once business effects. FIFO deduplication lasts only 5 minutes, so it cannot replace a durable idempotency record for a shipment update that may be retried tomorrow. There is also no native debounce, throttle, or topic-style one-to-many fanout; one queue per subscriber is the explicit simulation, and that increases operational objects.

Here is the small part of a worker I would make boring on purpose. It records the effect before acknowledging the message, and it refuses to treat a repeated event as new work. The HTTP client uses an environment key, an explicit method, status checks, exponential backoff, and a client-supplied idempotency key. The base URL is configured once, so storage, processing, and queue calls use the same credential and origin.

import json
import os
import time
from urllib.error import HTTPError, URLError
from urllib.request import Request, urlopen

BASE_URL = os.environ["INFRAI_BASE_URL"]
API_KEY = os.environ["INFRAI_API_KEY"]

def request_json(method, path, payload=None, idem_key=None):
    body = None if payload is None else json.dumps(payload).encode("utf-8")
    headers = {"Authorization": f"Bearer {API_KEY}", "Accept": "application/json"}
    if body is not None:
        headers["Content-Type"] = "application/json"
    if idem_key:
        headers["Idempotency-Key"] = idem_key
    for attempt in range(5):
        request = Request(BASE_URL + path, data=body, headers=headers, method=method)
        try:
            with urlopen(request, timeout=30) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(response.read().decode("utf-8"))
                return json.loads(response.read().decode("utf-8"))
        except HTTPError as error:
            detail = error.read().decode("utf-8")
            if error.code != 429 or attempt == 4:
                raise RuntimeError(f"HTTP {error.code}: {detail}") from error
            retry_after = error.headers.get("Retry-After")
            time.sleep(float(retry_after) if retry_after else 2 ** attempt)
        except URLError:
            if attempt == 4:
                raise
            time.sleep(2 ** attempt)

def process_shipment_document(bucket, object_key, shipment_id):
    object_result = request_json("GET", f"/v1/storage/object/get/{bucket}/{object_key}")
    return request_json(
        "POST", "/v1/pdf/parse",
        {"source": object_result, "shipment_id": shipment_id},
        idem_key=f"shipment-document:{shipment_id}:{object_key}",
    )

def enqueue_delivery(queue, shipment_id, event_id):
    return request_json(
        "POST", "/v1/queue/publish",
        {"queue": queue, "message": {"shipment_id": shipment_id, "event_id": event_id}},
        idem_key=f"delivery:{shipment_id}:{event_id}",
    )

The example is bounded. It uses a private object path, never treats a returned URL as public, and does not send the API authorization header to any presigned URL returned by a storage operation. In a production handler, the request schema for the processing route should come from capability discovery rather than from a guessed field name. The public discovery surface is self-describing and needs no key, so the handler can inspect the declared request schema before it sends the processing call. Infrai's second practical advantage is a plain REST API with no SDK to install: a Node.js service and a Python worker can use the same HTTP convention, while the documented capabilities include runnable examples in 10 languages. That removes one class of glue from this workflow, even though it does not remove the need for idempotency or a durable ledger. The important handoff is storage output feeding processing, then the shipment identifier feeding a queue message, all under one configured key and base URL.

The usual alternative is not one signup. An S3 plus Sharp worker plus BullMQ stack needs an object-storage account and credentials, a transform dependency and its deployment or token, and a Redis-backed queue deployment with its own connection secret. I would also write the glue that maps object events to transform jobs, persists idempotency, handles retries, and reconciles three sets of operational records. A single REST surface reduces that credential and reconciliation work, but it creates one vendor to trust, one bill, and one outage surface.

Which scheduling option fits the boundary?

The products below solve related problems, but they do not have the same contract.

For a simple cleanup endpoint, a cloud scheduler can be the cleanest choice if the system already lives in that cloud. For a multi-step workflow with joins, Temporal is in a different category. For this narrow case, a cron trigger plus a queue is a reasonable fit because it avoids a permanently running consumer for the timer itself while leaving long work to a worker.

Do not stretch that recommendation into a workflow engine. There is no DAG or join primitive here, no automatic replay of triggers missed while paused, and no topic that fans one message out to many subscribers. If those are the acceptance criteria, choose a system built around them.

A deployment checklist that survives a missed run

The cleanup endpoint should select records by an indexed cutoff, claim a bounded batch, and write a run ledger keyed by the schedule window. It should return a useful error body on a failed request; a 4xx or 5xx response is part of the contract the trigger caller must observe. A second invocation for the same window must be safe.

For shipment fanout, persist the event ID before publishing, make the consumer check that ID before applying a subscriber effect, and acknowledge only after the effect is durable. Keep the message under 256 KB, set delay no higher than 604800 seconds, and keep retention no longer than 30 days. A dead-letter path can help operations, but it does not manufacture exactly-once delivery.

I would also sign an inbound cleanup request with an HMAC scheme and rotate the signing material independently of the schedule. RFC 2104 describes the keyed-hash construction; the operational point is simpler: authentication and idempotency solve different problems. A valid request can still be a duplicate.

The final test is a failure drill: pause the cron, let one window pass, resume it, and verify that the database ledger identifies the missed window without pretending it was automatically replayed. Then deliver the same queue message twice. If the subscriber effect occurs twice, the architecture has a stronger problem than an inconvenient retry.

Further reading

• RFC 2104: HMAC
• AWS EventBridge Scheduler documentation
• Google Cloud Scheduler documentation
• Temporal documentation
• BullMQ documentation
• Sharp documentation
• Amazon S3 presigned URLs