Our app has 601 serverless invocations a day and exactly one background task, and the task gets 1,800 seconds

Nakodo runs almost everything on a schedule. Four cron entries, 601 invocations a day, all of them ordinary serverless functions with a 300-second ceiling, each one claiming a batch of work from a Postgres queue and doing as much of it as it can before the clock runs out.

There is one job that does not fit in 300 seconds, and it runs somewhere else. This is the whole of the configuration that puts it there:

import { defineConfig } from "@trigger.dev/sdk";

// Trigger.dev runs the work too long or heavy for a Vercel function: importing
// businesses from Overture Maps' places files (src/trigger). Everything else
// stays in the cron pipeline.
export default defineConfig({
  // The project's ref, from the Trigger.dev dashboard. Not a secret.
  project: process.env.TRIGGER_PROJECT_REF ?? "your-project-ref",
  dirs: ["./src/trigger"],
  runtime: "node-24",
  // A whole country's places can take several minutes to read.
  maxDuration: 1800,
  machine: "medium-1x",
  retries: {
    enabledInDev: false,
    default: { maxAttempts: 3, minTimeoutInMs: 30_000, maxTimeoutInMs: 300_000, factor: 2, randomize: true },
  },
  build: {
    // DuckDB is a native addon: installed in the image, not bundled.
    external: ["@duckdb/node-api"],
  },
});

Twenty-seven lines for a second runtime. Every one of them is load bearing, so here is what each one is actually buying.

maxDuration 1800 against 300

Our cron routes declare export const maxDuration = 300, which is the ceiling on the hosting plan the app runs on, and the pipeline's internal budget is set below it so that work already in flight can finish inside the window.

The job that moved is a read over a public dataset of business listings published as partitioned files in a cloud bucket. One area is quick. A whole country means pulling row groups out of 16 files, and that takes minutes, not seconds. You cannot split it into 300-second slices without either keeping cursors for a file format that does not want them or re-reading the same metadata on every slice, which is most of the cost.

So the rule that decided the split is not "background work goes on a queue platform". It is narrower: work whose unit cannot be subdivided below the function ceiling goes somewhere with a bigger ceiling. Six times bigger, in this case, and the comment in the config is the whole justification.

machine medium-1x, which is about memory

medium-1x is 2 GB. The task is not CPU bound, it is memory bound, because the query engine holds the dataset's file metadata in memory for the duration of the run, and that cache is the reason a second area in the same run is nearly free.

The task itself adds a second limit on the same ground:

// Each run holds Overture's file metadata in memory; a few at once is plenty.
concurrency: { total: 2 },

Two limits, one at the platform level and one at the task level, and they are not redundant: the machine size bounds one run, the concurrency bounds how many runs exist. The second one is the cheaper lever and the one I have actually tuned.

build.external, which is the only line here about bundling

build: { external: ["@duckdb/node-api"] }

The query engine is a native addon. Bundlers try to inline it, fail in interesting ways, and the fix is to tell the build to leave the import alone so the module is resolved from the image at runtime instead. One line, and without it nothing in this file matters because the task does not start.

This is the kind of configuration I want written down with its reason attached, because in six months the symptom is a module-not-found error inside a packaged bundle and the cause is a decision made by somebody who no longer remembers making it.

enabledInDev: false, which is about me

retries: { enabledInDev: false, default: { maxAttempts: 3, minTimeoutInMs: 30_000, maxTimeoutInMs: 300_000, factor: 2, randomize: true } }

In production: three attempts, the first retry about 30 seconds later, the second about 60, both jittered. The upper bound of 300 seconds never binds with three attempts and a factor of two; it is there so that raising maxAttempts later cannot accidentally produce an hour-long tail.

In development: no retries at all. A transient failure in production deserves patience. The same failure on my machine deserves a stack trace immediately, because it is not transient, it is a bug I wrote 40 seconds ago, and watching a runtime politely retry it three times is the worst possible feedback loop.

The task is 25 lines, and it validates its own input

export const placeImport = schemaTask({
  id: "place-import",
  schema: z.object({
    bbox: z.tuple([z.number(), z.number(), z.number(), z.number()]),
    areas: z.array(z.string()).min(1),
    categories: z.array(z.string()).min(1),
    countryCode: z.string().length(2).optional(),
  }),
  concurrency: { total: 2 },
  run: async (payload) => {
    const result = await importPlaces(payload);
    logger.info("Imported places", { ...result, areas: payload.areas.length, categories: payload.categories });
    return result;
  },
});

Both ends of this are our own code, which is the usual argument for not validating. The schema is there anyway, because the payload crosses a network, a queue and a process boundary, and it can arrive from a version of the app that is no longer deployed. A payload is an input whenever it has been stored somewhere between being written and being read, regardless of who wrote it.

The task body is one call and one log line. All of the actual work lives in src/lib, imported by both the task and the command line tools, so nothing about the import logic knows it is running on a different platform.

How a run is asked for

The caller is the cron pipeline, and the interesting part is the key:

const key = createHash("sha1").update(JSON.stringify(run)).digest("hex");
await tasks.trigger<typeof placeImport>("place-import", run, {
  idempotencyKey: `place-import:${key}`,
  idempotencyKeyTTL: "2h",
  tags: run.countryCode ? [`country:${run.countryCode}`] : ["cells"],
});

A hash of the payload, with a two-hour life. The comment above it names the exact failure it exists for: a request that reached the platform but whose acknowledgement did not reach us. Asked again, it is the same run rather than a second one, and two hours is long enough to cover a retry of the cron run and short enough that the same area can legitimately be asked for again later in the day.

The catch around it is the other half. If the trigger call fails, the pipeline removes the marks it wrote in the database that say "this area has been requested", so the next pass asks again. A request that was never made must not look like one that was.

The thing nothing checks

pnpm trigger:deploy is a separate deploy, to a separate platform, from a separate command. The app deploys when I push. The task deploys when I run that.

The task imports application code from src/lib. So if I change that code and deploy only the app, the task keeps running the previous version until I remember the second command, and nothing anywhere compares the two. No warning, no version check, no failing build. The only thing standing between that and a subtle bug is a line in the README and my memory.

That is the real cost of a second runtime, and it is not the 27 lines of configuration. It is that the project went from one deploy target to three: the app, the task, and the schema migrations, which are applied by hand. Each one is defensible on its own, and together they are an ordering problem that only exists in my head.

If this grew to two tasks I would add a check that compares a build identifier baked into both sides and refuses to run a task deployed from different source than the app. With one task and one developer, I have instead written it down here, which is a weaker guarantee honestly labelled.

The division, in one table

Almost everything is in the left column, and the point of the right column is that it has exactly one occupant. A background job runner is easy to add and hard to keep honest, and the test I apply before anything else moves over is the one the config comment already states: is this work that genuinely cannot be subdivided below 300 seconds. So far, once.

What that one task produces is the business side of the product. It has its own question in the FAQ on how it works ("Can Nakodo find businesses as well as creators?") and it is one of the lines that differs between plans on the pricing page. The 76 pages in the sitemap, as usual, are not involved at all: they are built at deploy time and have never waited on a queue.