From Runbook to Script: Turning My n8n Pre-Publish Checklist Into a One-Command Gate

My last post ended with a five-step pre-publish checklist for n8n LLM workflows. It took twenty minutes to run by hand. You know what happens to a twenty-minute manual checklist at 11 PM before a launch? It gets skipped. So I turned it into one command: ./prepublish.sh. Any gate fails, exit non-zero, nothing ships.

This is the same harness from the testing post — fixture gates, the provider kill matrix, the crash-and-rerun, the cost rehearsal — just executable. The mechanisms were covered in my earlier posts (the evidence-quote gate, the crash boundary, the mini-first cascade); this is the driver that runs the proof.

The script

A bash driver around the workflow runner. Each step prints PASS/FAIL, and the first failure exits non-zero. There is no --skip flag on purpose — the whole point is that it isn't skippable:

#!/usr/bin/env bash
# prepublish.sh — gate an n8n LLM workflow version before it ships.
set -euo pipefail

pass() { echo "PASS: $*"; }
fail() { echo "FAIL: $*"; exit 1; }

# --- Step 1: fixture gates ---
run_fixture clean || fail "clean fixture did not pass first try"
[ "$(run_fixture adversarial --count-items)" = "0" ] \
  || fail "adversarial fixture emitted items"
pass "fixture gates"

# --- Step 2: provider kill matrix ---
for p in openai anthropic gemini; do
  out=$(FORCE_FAIL_PROVIDER="$p" run_workflow messy 2>&1)
  echo "$out" | grep -q "INJECTED_FAILURE provider=$p" \
    || fail "sabotage not injected for $p (test would pass vacuously)"
  echo "$out" | grep -q "fallback engaged" \
    || fail "no fallback when $p failed"
done
out=$(FORCE_FAIL_PROVIDER=all run_workflow messy 2>&1)
echo "$out" | grep -q "INJECTED_FAILURE provider=all" \
  || fail "all-provider sabotage not injected"
echo "$out" | grep -q "LOUD FAILURE" \
  || fail "no loud failure when all providers down"
pass "provider kill matrix"

# --- Step 3: crash and rerun ---
run_workflow messy --crash-after extract_3 >/dev/null 2>&1 || true  # expected to die
sends_before=$(count_sends); spend_before=$(total_spend)
run_workflow messy --resume 2>&1 | grep -q "resumed" \
  || fail "resume did not pick up"
[ "$(count_sends)" = "$sends_before" ] || fail "duplicates after crash"
[ "$(total_spend)" = "$spend_before" ] || fail "re-spend after crash"
pass "crash-and-rerun"

# --- Step 4: cost rehearsal ---
esc=$(run_fixture messy --stats | jq -r .escalation_rate)
cost=$(run_fixture messy --stats | jq -r .cost_usd)
awk "BEGIN{exit !($esc <= 0.15)}" || fail "escalation rate $esc > 0.15"
awk "BEGIN{exit !($cost <= 0.50)}" || fail "per-run cost \$$cost > \$0.50 ceiling"
pass "cost rehearsal"

pass "ALL GATES GREEN — shippable"

run_workflow, run_fixture, count_sends, total_spend are thin wrappers around my n8n runner and the send/spend log — the crash-boundary post covers the upsert-by-id mechanism that makes step 3 possible. The script is the boring part. The interesting part is what broke when I scripted it.

Three things the script caught that the checklist never did

1. The sabotage has to be asserted, not just the outcome. My first version of step 2 checked only "did the fallback fire?" It passed on the very first run — and stayed green for weeks. Then I renamed a provider env var during a refactor, the force-fail injection silently stopped working, and the test kept passing vacuously: no failure was ever injected, so the "fallback" path was never exercised. The grep for the INJECTED_FAILURE marker was the fix. Now the test asserts the sabotage actually happened before it asserts the system survived it. A kill-matrix test that can't prove it killed anything is decoration.

2. The crash test needs a fixed kill point. By hand, I'd stop the execution "somewhere in the middle" — different somewhere every time. Scripted, I added a --crash-after hook at a named node (extract_3, mid-write, the most dangerous spot). Deterministic kills found a real bug the random ones missed: the resume path re-read the input batch but not the dedupe cache, so items extracted before the crash were re-emitted as duplicates on resume. The upsert logic was correct; the cache just wasn't warm on resume. Random crashes had always happened to land in spots where the cache was already loaded. Fixed by snapshotting the dedupe cache at crash time.

3. The cost rehearsal passed vacuously in CI. Locally it ran against a recorded fixture with real token counts. In CI, the model calls were mocked — and the mock returned empty usage metadata. Cost computed as $0.00. Under the ceiling, every time. Green forever. I changed the mock to return realistic token counts from the recorded fixture, so the rehearsal actually rehearses. Any mock that makes the assertion trivially true is the same bug as the unasserted sabotage: the test is lying to you.

Notice the pattern: every failure was the test failing, not the workflow. Scripting the checklist didn't just save twenty minutes — it found three ways my tests were passing without testing anything. The manual checklist could never have caught those, because a human running it by hand skips exactly the assertions that make it honest.

What stays manual

One thing deliberately didn't make it into the script: the review-queue spot check from the testing post. Output quality — is this action item well-written, is the summary readable — is subjective, and no assertion belongs there. The script verifies the mechanical contract (grounded, deduplicated, bounded, loud on failure). A human still eyeballs a sample of the review queue before shipping. That split is load-bearing: deterministic tests for the machine, human eyes for the prose.

I'm building AI Automation Lab — n8n workflow templates for AI automation — and keeping a free lab on GitHub. I also send occasional notes on n8n reliability patterns: get them here.