Node.js Auto-Push Background Tasks via Queue Subscriptions: 2026 Retry-Safe Reconciliation

The hard part of a nightly property-payment reconciliation job is not putting messages on a queue. In Node.js, auto-push background tasks via queue subscriptions still have to survive a timeout after the payment provider accepted the work. Use a push subscription to deliver each queue message to a public HTTPS endpoint, then make that endpoint idempotent before it acknowledges anything. That gives you automatic dispatch without pretending delivery is exactly once.

Here is the short decision matrix I would use:

My recommendation is the first row for a nightly reconciliation worker that does one bounded unit of work per message. Create a push target for the queue, require HTTPS, authenticate the incoming request, and acknowledge only after the reconciliation record has been committed. Keep the provider's transaction ID as the idempotency key. A retry then becomes a lookup, not a second payment operation.

What should the retry contract look like?

Start with the failure timeline, not the SDK. A payment provider can accept a reconciliation request while your worker loses its connection before it records success. The queue redelivers. Your worker must recognize the same property, statement date, and provider transaction identity, then return success for work already committed.

That is at-least-once processing. It is the normal contract for a standard queue, and it is why a 200 response from the worker cannot mean “the handler started.” It should mean “this message is now safe to stop delivering.”

I would store a small processing record with providerTransactionId, propertyId, statementDate, a status, and the resulting reconciliation reference. Put a unique constraint on the provider transaction ID where the provider guarantees it is unique. If the business key is composite, hash the normalized tuple. Do not use a timestamp generated inside the handler: two retries will produce two timestamps and defeat deduplication.

The queue's FIFO deduplication window is only 5 minutes, so it is not a substitute for that database constraint. Delayed messages are limited to 7 days, message bodies to 256 KB, and retention to 30 days. Acknowledged messages disappear. There is no Kafka-style replay or multiple-consumer-group history to rescue an under-specified reconciliation record.

The HTTP boundary needs its own checks. Verify the authentication material before parsing business fields, reject stale or malformed requests, and use a constant-time comparison for an HMAC signature when that is the chosen scheme. RFC 2104 is still the useful reference here. A public URL without authentication is a payment-data leak waiting for a scanner.

How should a Node.js service auto-push background tasks via queue subscriptions?

Push subscriptions remove the worker's idle polling loop. The queue calls the target endpoint, and the service can scale around incoming work. That is a useful time-to-first-call improvement for a small Node.js service: one listener, one queue target, and one explicit acknowledgement decision.

The target must be publicly reachable over HTTPS. An internal hostname, a private load balancer, or a local development port cannot receive the delivery. Put the endpoint behind a public TLS terminator and restrict access with authentication. The subscription route is POST /v1/queue/push_subscribe/{queue}; use the discovery surface to obtain the current request schema rather than guessing fields from a blog post.

The opposite trade-off is operational. A push sender can retry while your handler is still running. The handler therefore needs a short transaction around its deduplication write, and it needs a clear response policy: return a success status for a committed duplicate, and a failure status for a transient error that should be attempted again. Never return success merely because JSON parsed.

Here is a compact Node.js setup plus handler. The setup call uses the schema-shaped JSON supplied through an environment variable, so it does not hard-code undocumented field names. The handler does not call the payment provider twice, and it does not acknowledge before the durable write. The queue-specific adapter is deliberately small; the payment operation belongs behind the idempotent repository boundary.

import { createHmac, timingSafeEqual } from "node:crypto";
import type { IncomingHttpHeaders } from "node:http";

async function createPushSubscription(queue: string, body: unknown): Promise<unknown> {
  const apiKey = process.env.INFRAI_API_KEY;
  const apiBaseUrl = process.env.INFRAI_BASE_URL;
  if (!apiKey) throw new Error("INFRAI_API_KEY is required");
  if (!apiBaseUrl) throw new Error("INFRAI_BASE_URL is required");

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const response = await fetch(`${apiBaseUrl}/v1/queue/push_subscribe/${queue}`, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": `property-reconciliation-${queue}`,
      },
      body: JSON.stringify(body),
    });

    if (response.status === 429) {
      const retryAfter = Number(response.headers.get("retry-after"));
      const delayMs = Number.isFinite(retryAfter) && retryAfter > 0
        ? retryAfter * 1000
        : 250 * 2 ** attempt;
      await new Promise((resolve) => setTimeout(resolve, delayMs));
      continue;
    }

    if (!response.ok) {
      throw new Error(`push subscription failed (${response.status}): ${await response.text()}`);
    }
    return response.json();
  }

  throw new Error("push subscription was rate limited after 4 attempts");
}

type ReconciliationMessage = {
  providerTransactionId: string;
  propertyId: string;
  statementDate: string;
};

type Repository = {
  beginIfAbsent: (key: string, message: ReconciliationMessage) => Promise<boolean>;
  markComplete: (key: string) => Promise<void>;
};

function validSignature(rawBody: string, headers: IncomingHttpHeaders, secret: string): boolean {
  const supplied = headers["x-queue-signature"];
  if (typeof supplied !== "string") return false;

  const expected = createHmac("sha256", secret).update(rawBody).digest("hex");
  const left = Buffer.from(supplied, "utf8");
  const right = Buffer.from(expected, "utf8");
  return left.length === right.length && timingSafeEqual(left, right);
}

export async function handlePush(
  rawBody: string,
  headers: IncomingHttpHeaders,
  repository: Repository,
): Promise<{ status: number; body: string }> {
  const secret = process.env.QUEUE_SIGNING_SECRET;
  if (!secret || !validSignature(rawBody, headers, secret)) {
    return { status: 401, body: "unauthorized" };
  }

  let message: ReconciliationMessage;
  try {
    message = JSON.parse(rawBody) as ReconciliationMessage;
  } catch {
    return { status: 400, body: "invalid json" };
  }

  const key = message.providerTransactionId;
  if (!key || !message.propertyId || !message.statementDate) {
    return { status: 400, body: "missing reconciliation identity" };
  }

  const firstAttempt = await repository.beginIfAbsent(key, message);
  if (!firstAttempt) return { status: 204, body: "duplicate" };

  try {
    await repository.markComplete(key);
    return { status: 204, body: "acknowledged" };
  } catch {
    return { status: 503, body: "retry" };
  }
}

const subscriptionBody = JSON.parse(process.env.PUSH_SUBSCRIPTION_JSON ?? "{}");
await createPushSubscription(process.env.QUEUE_NAME ?? "property-reconciliation", subscriptionBody);

The sample assumes the repository makes beginIfAbsent atomic. That is the non-negotiable line. If the write and the side effect cannot share a transaction, use an outbox or a provider API idempotency key. Do not solve the race with an in-memory Set; two Node.js processes will not share it, and a restart erases it.

Which competing service is the better choice?

Amazon SQS is the pragmatic choice for an AWS-centered estate. Its queue semantics and dead-letter workflow are well understood, and AWS teams can keep identity and operations in one control plane. Choose it when the surrounding system already depends on AWS primitives. The integration cost is less important than avoiding a second operational vocabulary.

Google Cloud Pub/Sub is stronger when push delivery and Google Cloud identity are already standard practice. It also makes a useful reference point for this design because its push subscriber model puts the HTTPS handler and acknowledgement boundary in the foreground. Choose it when IAM, service accounts, and Cloud Run-style deployment are already solved.

Azure Service Bus is the better fit for workloads that need sessions, topics, or longer-running lock management. Those features matter when reconciliation becomes a larger messaging system. They are extra surface area for one nightly queue and one worker, though. More knobs are not automatically more reliability.

BullMQ is a sensible Node.js-native alternative when Redis is already part of the deployment and the team wants queue workers close to application code. Inngest and Trigger.dev are better comparisons once the problem includes durable application workflows, scheduled steps, or event-driven orchestration. They solve a wider problem than one push target, which can be useful, but it also changes the operational model. Temporal belongs in the same wider category when the reconciliation process needs durable workflow state and explicit compensation.

Infrai fits the narrow version of this problem when a team values one REST API, one key, and one bill across backend services, while the queue worker remains a normal Node.js service. Its scheduling surface supports push subscription setup and deletion, and its discovery endpoint exposes schemas and runnable examples. That consistency can reduce glue in a small CLI or SDK project. It does not turn the queue into Airflow or Temporal: there is no DAG orchestration, no fan-out/join primitive, and no topic-style one-to-many delivery. Model that boundary explicitly.

When should the runner-up win?

Use a workflow engine when reconciliation has dependent stages, compensation, timers, or a real join. A queue subscription can trigger work; it is not a workflow graph. Use a topic or several queues when many independent consumers need the same event. This system has no native one-to-many topic primitive, so simulating fan-out with N queues is an architectural decision, not a hidden setting.

Keep it boring.

Measure it.

For a nightly run, the long paragraph of failure handling is the product: identify the provider transaction, atomically claim it, perform the bounded work, and acknowledge only after the durable state exists. If the provider's own API supports an idempotency key, pass the same stable key through that call too. That extra layer matters when the database commit and the remote response land on opposite sides of a network failure; without it, a worker restart can repeat a financially meaningful side effect even though the queue logic looks correct in code review.

I optimize for time-to-first-call, but I will trade that away for a workflow engine when the business process needs compensation or a join. Four retry attempts in the setup helper are a bounded network policy, not a guarantee that reconciliation is complete. The durable idempotency record is still the authority, and the queue's redelivery is still expected behavior.

For a task longer than 900 seconds, schedule a short trigger that enqueues work and let a worker consume it. Do not make a cron invocation carry the long operation. Cron targets are public http_url calls, not hosted code execution. Also remember that paused cron schedules do not backfill missed triggers, trigger timing has second-level jitter, and run output keeps only the first 4 KB.

There is no native debounce or throttle, and cron expressions do not support nonstandard L extensions. Those are not reasons to reject push queues for nightly reconciliation. They are reasons to put policy in the application where it can be tested.

The decision rule is simple: choose push delivery for bounded, independently idempotent reconciliation units; choose a richer queue or workflow product when the unit itself has become a workflow. In either case, make the provider transaction identity durable before acknowledging the message. That is the part that keeps a retry from becoming a second financial action.

Sources

• RFC 2104: HMAC keyed-hashing for message authentication
• Amazon SQS Developer Guide: Queue types and delivery behavior
• Amazon SQS: Sending and receiving messages
• Google Cloud Pub/Sub: Push subscriptions
• Azure Service Bus messaging overview
• BullMQ documentation
• Inngest documentation
• Trigger.dev documentation
• Temporal documentation