Your Merge Button Is the Real Quality Gate

A release gate only works if the merge button has to ask it. Here's how to wire one into CI so it fails closed.

In the last post, Nothing Should Merge on an AI's Word, I argued that AI-written code needs a gate between "the agent finished" and "this is a pull request," and that the gate should check evidence: a review that ran, tests that passed on this exact version, and a person who said yes.

There's a hole in that picture. A gate inside one tool only covers changes made through that tool. Pull requests still get opened by hand, branches still get pushed, and the merge button on GitHub or GitLab doesn't know your AI platform exists. If the gate can be skipped by clicking somewhere else, it's not a gate.

So the real question is: how do you make the merge itself ask for a verdict?

Put the check where the merge happens

The answer is boring, which is a good sign: a CI job that asks the gate for a verdict on this pull request, and fails when the answer is no. Then you make that job a required status check in branch protection.

That second step is the one teams forget. A CI job that isn't required is a dashboard. It only becomes a gate when the merge button is greyed out until it passes.

In AI CodeOps, the question goes to one endpoint, POST /api/release/check, with a workspace API key. You send a pull request URL (or a commit SHA, at least 7 characters) and get back a verdict that covers every AI session that touched that change:

• allowed: true or false
• summary and reasons: plain-English explanations
• blockingFindings, testFailures, coverageShortfall, licenseViolations: the detail behind a deny
• runsConsidered: which sessions the verdict was based on

The trap: "200 OK" doesn't mean "allowed"

Here's the detail that makes or breaks a CI gate. The endpoint returns HTTP 200 for both outcomes. A denied change is a successful API call with "allowed": false in the body.

That means the most common way to write this check is wrong:

# Looks fine. Lets everything through.
curl -sf -X POST "$CONSOLE/api/release/check" ...

curl -f only fails on HTTP errors. A deny comes back as 200, so the step goes green and the change merges. That's a gate that fails open, which is the one thing a gate must never do.

The fix is to read the body and treat anything other than an explicit true as a no:

jq -e '.allowed == true' > /dev/null

jq -e exits non-zero when the result is false or null, so a deny fails the job, and so does a missing or malformed field.

A fail-closed GitHub Actions job

Putting it together:

name: ai-release-gate

on:
  pull_request:

jobs:
  release-gate:
    runs-on: ubuntu-latest
    steps:
      - name: Ask the release gate for a verdict
        env:
          WORKSPACE_API_KEY: ${{ secrets.AICODEOPS_API_KEY }}
          PR_URL: ${{ github.event.pull_request.html_url }}
        run: |
          set -euo pipefail

          body=$(jq -n --arg pr "$PR_URL" '{
            pullRequestURL: $pr,
            blockingSeverity: "medium",
            minCoveragePct: 80,
            blockOnInFlight: true
          }')

          resp=$(curl -sSf --retry 3 --retry-all-errors \
            -X POST "https://console.aicodeops.dev/api/release/check" \
            -H "Authorization: Bearer $WORKSPACE_API_KEY" \
            -H "Content-Type: application/json" \
            -d "$body")

          echo "$resp" | jq -r '.summary'
          echo "$resp" | jq '.reasons'

          # Anything other than an explicit "true" fails the job.
          echo "$resp" | jq -e '.allowed == true' > /dev/null

Every way this can go wrong ends in a red check:

Nothing on that list ends in a green check. That's the property you want.

Two practical notes. The key only needs the workspace:read permission, so don't hand CI a more powerful one. And use the same host you open the console on.

Tune the bar per pipeline

The defaults block on open high and critical findings and on failing tests. CI is where you can be stricter, because different branches deserve different bars:

• blockingSeverity: "medium" blocks on medium findings too. Reasonable for anything that touches auth, payments, or infrastructure.
• minCoveragePct sets a coverage floor for the latest run that reported coverage.
• requiredWorkflows lists workflows that must have completed on this change, such as a security audit.
• disallowedLicenses checks the project's SBOM for licenses you can't ship.
• requireDeployedTo: "staging" denies until the change has been deployed to staging. Useful on the job that promotes to production, not on every pull request.
• blockOnInFlight: true denies while a session for the change is still running, so nobody merges halfway through a fix.

The gate looks back 14 days by default (lookbackMs changes that). An old session isn't silently reused forever.

Then keep the receipt

The gate decides whether a change ships. The evidence pack is how you prove later why it did.

Every change gets a signed pack: the task and the models used, each run with its tokens and cost, review findings and how they were triaged, test results and coverage, policy decisions, who was eligible to approve and how each person voted, and the pull request and deployment it ended up in. The code itself isn't included. Instead the pack carries a SHA-256 hash of the diff, so you can match it against the pull request.

The part I like is that you don't have to trust the platform to check it. Packs are signed with Ed25519 over canonical JSON, and the public keys are published at /api/evidence?action=keys. You can verify a pack in the browser at aicodeops.dev/evidence/verify without an account, or offline with Node's built-in crypto module. Keep a copy of the keys file next to your audit records, and verification doesn't depend on the vendor still being reachable in three years.

Retired keys stop signing new packs but are never deleted, so old packs keep verifying.

The checklist

If you're adding AI agents to a repository this quarter, here's the short version:

1. Gate the merge, not just the agent. Run the verdict check in CI and make it a required status check.
2. Read the verdict, not the status code. allowed == true or the job fails.
3. Make every error a no. Bad key, timeout, rate limit, unreadable response: all red.
4. Raise the bar where it matters. Stricter severity and coverage on sensitive paths, a staging requirement before production.
5. Keep evidence you can verify yourself. A signed record beats a chat log.

AI agents can write a lot of code. The merge button is where you decide how much of it you trust.

AI CodeOps is free during the open beta and free forever for non-commercial open-source projects. The full field list is in the API docs, and the gate itself is described at aicodeops.dev/product/quality-gates.

Drafted with AI assistance and edited by the AI CodeOps team.