Docker Compose 5.6's jobs type only starts through docker compose run

Docker Compose 5.6.0 shipped on 2 October with a new top-level jobs: section, sitting next to services: in the compose file. I wanted to know what actually happens when you declare one, because the release notes say almost nothing beyond "partial support for jobs: only manually-triggered jobs are supported for now."

The first thing that happened was my compose file got rejected.

$ docker compose config
validating /home/user/compose-jobs-test/compose.yaml: jobs.migrate missing property 'triggers'

A jobs: entry is not just a service with a different label. It needs its own triggers: block, and compose will not even parse the file without one. Once I added triggers: {manual: true}, it validated and I could start testing what a job actually does differently from a service.

What a job refuses to do

I wrote a small compose file: a long-running db service, and a migrate job that depends on it, sleeps a second and exits 0.

services:
  db:
    image: alpine:3.20
    command: ["sh", "-c", "echo db-up; sleep 600"]

jobs:
  migrate:
    image: alpine:3.20
    depends_on:
      - db
    triggers:
      manual: true
    command: ["sh", "-c", "echo migrating; sleep 1; echo done; exit 0"]

docker compose up -d started db and said this about the job, then moved on:

time="...Z" level=warning msg="jobs are not started by up; trigger them with `docker compose run`: migrate"

Exit code 0. docker compose ps -a afterwards shows only db. The job is simply absent, not failed, not pending, not listed anywhere up reports on.

create and start are less forgiving. Both refuse outright:

$ docker compose create migrate
job "migrate" can only be triggered with "docker compose run"
$ echo $?
1

Same message, same exit code, for start. So of the five commands that normally bring a service into existence, four of them (up, create, start, and as far as I could find no fifth alternative) will not touch a job at all. Only run works.

The opposite reading: declaring the documented future breaks the present

The release notes are honest that scheduled jobs aren't supported yet, only manual ones. I wanted to see what "not supported yet" meant in practice, so I added a second job with a schedule trigger instead of manual:

jobs:
  nightly:
    image: alpine:3.20
    triggers:
      schedule: ["0 2 * * *"]
    command: ["sh", "-c", "echo nightly-run"]

This is where the asymmetry shows up. A manual-only job that up can't start gets a warning and is skipped; the rest of the stack comes up fine. A job with a schedule trigger makes up refuse to run at all:

$ docker compose up -d
scheduled jobs are not supported in this version: nightly
$ echo $?
1

Not a warning this time. The whole up fails, and nothing starts, not even the unrelated db service sitting in the same file. If you're migrating an existing stack to use jobs and you reach for the field that matches what you actually want (a job that runs on a schedule), you get a worse outcome than if you'd done nothing: the entire compose file stops coming up until you remove that trigger. Manual-only jobs are the only kind that currently coexist peacefully with the rest of your stack.

docker compose run --rm nightly still runs the scheduled job by hand without complaint, which matches what the Compose team's own notes say about manual execution being "always available... unless a job sets triggers.manual: false." I checked that half of the claim directly too: a job declared with triggers: {manual: false} is refused by run as well:

$ docker compose run --rm migrate_locked
job "migrate_locked" is declared with manual: false, it cannot be run manually
$ echo $?
1

Since nothing can trigger it on a schedule yet either, a job with manual: false in this release is not deferred to a scheduler. It's unreachable. There is currently no path in Compose 5.6.0 that starts it.

What a job costs you to run

docker compose run on a job starts its declared dependencies first, same as it does for a one-off command on a service. I measured how much that costs by timing docker compose run --rm migrate three times with db already running, and three times after docker compose down -v so db had to be created and started from scratch each time.

The job's own work (a one-second sleep plus container overhead) is identical in both cases, so the roughly 230ms gap, about 15%, is what it costs to create and start the dependency container before the job can begin. That's a small number on a single Alpine container; it's the number to budget for if your job's real dependency is a database image with its own startup routine, multiplied by however many depends_on entries the job declares.

Exit codes pass through exactly. A job that ends with exit 17 hands that 17 straight back to the shell that ran docker compose run:

$ docker compose run --rm migrate_fail
failing
$ echo $?
17

That matters if you're wiring a job into a CI step and checking $? afterwards, which is presumably the point of a dedicated one-off-task primitive.

Under concurrency

I ran the same job twice at once, both in the background, to see whether two manual triggers would collide:

$ docker compose run --rm migrate & docker compose run --rm migrate & wait
 Container compose-jobs-test-migrate-run-4fdc2a756110 Creating
 Container compose-jobs-test-migrate-run-f3445db266a6 Creating
migrating 1
migrating 1
done 1
done 1

Two distinct container names, both ran to completion, both cleaned up by their own --rm. No lock, no queue, no contention on the shared db dependency either, since it was already running. Jobs don't serialise against each other; if you need that, you still have to build it yourself.

A claim I checked and found true

The one documented behaviour I could directly verify was profile activation: a job's own profiles: entry is meant to be enabled automatically when you run it, without needing --profile on the command line. I gave a job profiles: ["migrations"], ran it with no --profile flag, and it ran:

$ docker compose run --rm migrate
migrated-with-profile
$ echo $?
0

That one held up exactly as documented.

What I got wrong on the way

My first attempt at the schedule trigger was triggers: schedule: "0 2 * * *", a bare string, because that's what cron usually looks like everywhere else. Compose rejected it: jobs.nightly.triggers.schedule must be a array. It wants a list of objects, each with a cron: key — schedule: [{cron: "0 2 * * *"}] once compose canonicalises it. I'd assumed the YAML shorthand I wrote would work because it reads naturally; the schema disagreed, and the error message was specific enough that I didn't need to go digging through compose-go source to fix it.

I also nearly reported "jobs leak containers that down can't clean up" as a jobs-specific finding, since a migrate container left over from a run without --rm survived both docker compose down and docker compose rm -f. Before writing that down as a cost of the new feature, I ran the identical test against an ordinary service behind a profiles: ["tools"] guard, the pattern people have used for one-off tasks in Compose for years. It leaked exactly the same way. Jobs didn't introduce this; they inherited it. It's worth knowing either way, but it would have been the wrong feature to blame.

Run it yourself

This needs Docker Compose 5.6.0 or later (docker compose version to check) and a working Docker daemon.

mkdir jobs-test && cd jobs-test
cat > compose.yaml <<'EOF'
services:
  db:
    image: alpine:3.20
    command: ["sh", "-c", "echo db-up; sleep 600"]

jobs:
  migrate:
    image: alpine:3.20
    depends_on:
      - db
    triggers:
      manual: true
    command: ["sh", "-c", "echo migrating; sleep 1; echo done; exit 0"]
EOF

docker compose up -d          # starts db, warns and skips migrate
docker compose create migrate # refused: "can only be triggered with docker compose run"
docker compose run --rm migrate   # this is the one that actually works
docker compose config --services  # lists db only, migrate is invisible here

Add a second job with triggers: {schedule: [{cron: "0 2 * * *"}]} and run docker compose up -d again to see the whole file refuse to start, rather than just skip the one job.

What I'd do with this

If you're reaching for jobs: today, use triggers: {manual: true} and only that. A schedule trigger doesn't give you a scheduled job yet, it gives you a compose file that up refuses to run at all, and manual: false with no scheduler gives you a job nothing can start. Keep using run --rm rather than bare run, because the orphaned-container problem is the same one you already had with profile-gated one-off services, and jobs don't fix it. And if any script you have loops over docker compose config --services to do per-service work, remember it won't see your jobs at all; you'll need to list them separately until Compose grows a command that does.