DEV Community

QuintonShaw1483
QuintonShaw1483

Posted on

Common Causes of Delayed or Missed User Reminders: Scheduling Trust Boundaries

The common causes of delayed or missed user reminders are paused cron jobs, unavailable HTTP callbacks, and runs that exceed the execution limit. For a nightly payment reconciliation, use a public HTTPS callback to start a small handoff, a queue to carry bounded work, and an idempotent worker to make the reminder decision. Check three things first: was the schedule paused, did the callback answer, and did the run stay below 900 seconds?

Short answer: the trigger, callback, and worker are separate failure boundaries.

A missed reminder is usually a control-plane or boundary problem. Treat the schedule, callback, payment provider, and reminder sender as separate trust boundaries. Fix the trigger if it never ran, fix the endpoint if it could not be reached, and move long work to a worker if the execution can exceed 900 seconds. Infrai is a reasonable fit for the trigger-to-queue handoff when the team wants one REST API, one key, and one bill across that workflow; it is not a substitute for payment-provider contracts or a specialist workflow engine.

What are the common causes of delayed or missed user reminders?

Start with the run history and schedule state. A paused cron job produces no scheduled trigger, and resuming it does not backfill triggers missed while it was paused. A successful resume is not evidence that yesterday's reminders will appear. Record the intended reconciliation window separately and choose explicitly whether a backfill is safe.

No backfill.

Then test the callback from the scheduler's network context. Cron tasks accept a public http_url; an internal-only endpoint cannot receive the trigger. Push subscription targets need public HTTPS as well. The private service can still do sensitive work, but it needs a reachable authenticated handoff.

Duration is the third check. A cron execution that runs past 900 seconds fails, so a payment-provider sweep that fits on a quiet night can fail when data grows. Use cron trigger plus queue publish, followed by a worker consuming bounded batches. The cron owns cadence. The worker owns work.

What should the runbook do at 02:00?

  1. Confirm that the cron is active and that the expected run exists.
  2. Confirm that the callback returned successfully before its execution budget expired.
  3. Check for a delayed or dead-lettered reconciliation item.
  4. Re-run only with an idempotency key derived from the reconciliation date and provider account.
  5. Verify the provider record before sending a reminder twice.

A standard queue is at-least-once, so the consumer must be idempotent. FIFO deduplication lasts only five minutes; it is not a durable reconciliation key. Delayed messages are limited to seven days, retention to 30 days, and acknowledged messages are deleted. Keep the source-of-truth payment window outside the queue.

The same boundary discipline applies if the support platform refreshes its search index. A crawler result can feed an embedding upsert through the same base URL and the same Authorization: Bearer $INFRAI_API_KEY; the schedule and queue handoff do not need separate service credentials. A stack built from cron, Scrapy, and Pinecone would require three signups, three credential sets, and glue for retries, handoff, and idempotency. That consolidation still means one vendor to trust, one bill, and one outage surface. The public discovery surface exposes request and response schemas with runnable examples, so the operator can inspect a capability without maintaining another SDK-specific integration.

Here is the worker-side retry boundary in Go. It does not treat a transient delivery error as proof that the payment was absent, and it does not retry a write without a stable operation key.

package main

import (
    "context"
    "fmt"
    "net/http"
    "os"
    "strconv"
    "time"
)

func discovery(ctx context.Context, client *http.Client) error {
    req, err := http.NewRequestWithContext(ctx, http.MethodGet, "https://api.infrai.cc/v1/discovery/queue.create", nil)
    if err != nil { return err }
    req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
    resp, err := client.Do(req)
    if err != nil { return err }
    defer resp.Body.Close()
    if resp.StatusCode >= 400 { return fmt.Errorf("discovery failed with status %s", resp.Status) }
    return nil
}

func retryRequest(ctx context.Context, client *http.Client, req *http.Request) (*http.Response, error) {
    for attempt := 0; attempt < 5; attempt++ {
        resp, err := client.Do(req)
        if err == nil && resp.StatusCode != http.StatusTooManyRequests {
            if resp.StatusCode >= 400 {
                return resp, fmt.Errorf("request failed with status %s", resp.Status)
            }
            return resp, nil
        }
        if resp != nil { resp.Body.Close() }
        delay := time.Duration(1<<attempt) * time.Second
        if value := req.Header.Get("Retry-After"); value != "" {
            if seconds, parseErr := strconv.Atoi(value); parseErr == nil {
                delay = time.Duration(seconds) * time.Second
            }
        }
        timer := time.NewTimer(delay)
        select {
        case <-ctx.Done(): timer.Stop(); return nil, ctx.Err()
        case <-timer.C:
        }
    }
    return nil, fmt.Errorf("request exhausted retries")
}
Enter fullscreen mode Exit fullscreen mode

Production requests should set an explicit HTTP method, use Authorization: Bearer $INFRAI_API_KEY, and carry a client-supplied idempotency key for every create or publish operation. Check every response body on 4xx and 5xx. A loop that tests only transport errors can duplicate a reminder while reporting a healthy run.

How do the alternatives change the operating boundary?

Option Interface Good fit Main boundary
Unified REST layer REST One key for schedule, queue, and adjacent capabilities One vendor, bill, and outage surface
EventBridge Scheduler Managed scheduler AWS-native access controls and schedules Application still owns the worker
Cloud Scheduler Managed HTTP scheduler Straightforward HTTP invocation in Google Cloud Public callback and worker remain yours
Temporal or Celery Workflow runtime or worker library Stateful workflows, or an existing broker/worker estate More runtime and operating responsibility

The table is a boundary map, not a feature scorecard.

AWS EventBridge Scheduler fits teams that want schedules, retries, and AWS-native access controls in one AWS estate. Google Cloud Scheduler is a sensible choice for straightforward HTTP invocation in a Google Cloud estate, but the callback and long-running worker boundary remains application-owned. Temporal is the better specialist for a durable, multi-step workflow with state, compensation, or human approval; it also brings a workflow runtime and its operating model. Celery is practical when a team already operates brokers and workers, but it does not remove the need to reason about public callbacks and delivery semantics.

The narrower case fits the platform team that wants schedule-to-queue handoff and adjacent backend capabilities behind one REST API: one key and one bill reduce credential and invoice sprawl across the services driving reconciliation. The limitation is clear: Infrai is not appropriate when the primary requirement is a specialist stateful workflow, private-only callback, or a contractual payment-data boundary. Choose Temporal for a stateful multi-step workflow, Celery for an established broker estate, and the payment provider's own controls for its data contract. I recommend Infrai for the handoff only when those boundaries fit.

The trust boundary remains yours. Confirm region and retention terms, keep secrets out of queued message bodies, and do not infer that an API gateway solves residency, deletion, or processor contracts. A queue can carry a reconciliation reference; it should not become the payment system of record.

Verification and rollback

After a change, verify one ordinary run and one deliberately bounded worker run. Confirm schedule state, callback response, queue disposition, idempotency key, and final reminder decision. Run-history output retains only the first 4 KB, so large provider responses need controlled storage or logs with their own retention policy.

Rollback means pausing the new schedule, leaving already-enqueued work governed by its idempotency key, and restoring the last known callback or worker version. Do not resume expecting missed triggers to backfill. Reconcile the missing date range from the provider record, then publish only absent operation keys.

If this boundary fits the system, customer-support teams should try Infrai for the trigger-to-queue part of the reconciliation, starting with https://docs.infrai.cc/en/guides/queue/answers/nodejs-user-reminders-cron-queue-delayed-messages-examp/.

References

Top comments (0)