- Book: LLM Observability Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
The agent said done. The tests passed on your machine. You pushed, opened the pull request, and five minutes later the check is red.
So you open the Actions page, expand the failed job, and scroll past dependency installs and cache restores until you find the one assertion that broke. You select a few hundred lines, copy them, and paste them into a new chat. That chat has never seen this branch. It doesn't know which files the agent changed or why, so it starts by reading the repository again.
There is a shorter loop. Four GitHub CLI commands find the run and pull only the failing steps. One cut and one tail make the log small enough to hand over. Then the failure goes back to the session that wrote the code, in the folder where it wrote it.
Every gh and git command below works on its own. In the Hermes parts I use Hermes IDE, which I build, so weigh those with that in mind.
Step 1: find the run for the commit you pushed
Run this from the agent's worktree, on its branch. A branch collects many runs over a day, so filter by the exact commit instead of only by branch. gh run list takes --commit, --branch, --status, --workflow and --json:
SHA=$(git rev-parse HEAD)
gh run list --commit "$SHA" \
--json databaseId,workflowName,headBranch,status,conclusion
If you have more than one workflow, add --workflow with its name. Then keep the ID of the newest run and wait for it to finish:
RUN_ID=$(gh run list --commit "$SHA" --limit 1 \
--json databaseId --jq '.[0].databaseId // empty')
gh run watch "$RUN_ID" --compact --exit-status
The wait matters. gh won't fetch logs for a run that is still going. Its source returns "run %d is still in progress; logs will be available when it is complete". gh run watch refreshes until the run ends. --compact shows only the relevant or failed steps, and --exit-status makes it exit non-zero when the run fails. If the run already finished, it prints which conclusion it ended with and returns at once, so the same line works either way.
Step 2: pull only the failing steps
First, make sure the log file can never end up in a commit. The agent you are about to hand it to runs git add too:
echo 'ci-*.log' >> "$(git rev-parse --git-path info/exclude)"
--git-path resolves to the right file from any worktree. In a linked worktree, git reads the shared info directory of the main repository (gitrepository-layout), so this one line covers every task folder, and the file stays out of git status.
Now pull the log. gh run view has --log for everything and --log-failed for only the steps that failed:
gh run view "$RUN_ID" --log-failed > ci-raw.log
wc -l ci-raw.log
Each line comes prefixed with the job name and the step name, separated by tabs. Lines gh can't match to a step are filed under UNKNOWN STEP (same source file as above). That prefix is useful for you and noise for the agent, which is why step 3 cuts it.
If several jobs failed and you only care about one, list them and narrow it with --job. The job fields gh exports include databaseId, name and conclusion (source):
gh run view "$RUN_ID" --json jobs --jq '.jobs[]
| select(.conclusion == "failure")
| "\(.databaseId)\t\(.name)"'
gh run view "$RUN_ID" --job 123456789 --log-failed \
> ci-raw.log
Replace 123456789 with a databaseId from the first command, not the number in the job's browser URL. The rerun docs warn that the URL number gets a 404 from --job there.
Step 3: trim it to what the agent needs
Every line you hand over lands in the agent's context and costs tokens. Setup noise also buries the line that matters. Two trims cover most cases:
# keep the step name, drop the job name, keep the end
cut -f2- ci-raw.log | tail -n 150 > ci-failure.log
# or: the lines around each error, capped
grep -n -i -E -B 3 -A 15 'error|fail' ci-raw.log \
| head -n 200 > ci-failure.log
cut -f2- drops the first tab-separated field, the job name, and keeps the rest. tail keeps the end of the failed step, which is where a test runner tends to print its summary and the exit code. Use the grep version when the real error sits in the middle of a long step. Check wc -l ci-failure.log, then skim the file yourself before you hand it over. You are about to paste it into a model, and you should know what is in it.
Step 4: hand it to the session that wrote the code
The agent that wrote the branch knows which files it touched and what it was trying to do. A new chat has to rebuild all of that from the diff, and it may "fix" the test by undoing the feature.
In Claude Code, you don't paste anything. Shell mode runs a line that starts with ! and "adds the command and its output to the conversation context". Claude then responds to the output without a second prompt. In the agent's session, in the same worktree:
! cat ci-failure.log
Then say what you want done with it: "CI failed on this branch. Fix the cause, run the same test command locally, and don't edit the workflow file or loosen the test."
For Codex and the other agents, put the same request in a normal prompt and name the file: "Read ci-failure.log in the repo root. CI failed on this branch...". The file sits inside the agent's worktree, so it can read it like any other file.
If you already closed that session, reopen it from the worktree folder. claude -c loads "the most recent conversation in the current directory". codex resume --last resumes "the most recent chat from the current working directory". Each task has its own worktree, which makes its own directory, so "most recent here" is usually the session that wrote the branch. Claude Code also has claude --from-pr 123, which opens the session picker filtered to sessions linked to that pull request. A session gets linked when Claude is the one that created the PR.

Codex on orders-types and Claude Code on orders-api, each in its own hermes/ worktree, with status, context use, cost and branch in the sidebar.
This is where Hermes saves the lookup. ⌘N (Ctrl+Shift+N on Windows and Linux) starts each task on its own hermes/<name> branch in its own worktree, and the agent runs its own interface in a real terminal there. The headBranch field from step 1 names the task's hermes/ branch, so you know which session to open. Claude's ! shell mode works in that terminal like anywhere else. Hermes never types into a terminal on its own, so you type the prompt yourself. On macOS and Linux the agents keep running while Hermes quits or updates, and you reattach where they left off.

The Review Desk's Worktrees tab: one worktree per task, each with its hermes/ branch, task, age and disk size.
When the agent reports back, open the Review Desk with ⌘G (Ctrl+Shift+G). Every agent turn is saved, so the fix is its own turn. You can read it by turn or by file with risk flags, and restore any single turn, with undo. If CI runs a test command you can run locally, add it to the task's checks: in Hermes, your own checks can decide when a task is done.
Step 5: push the fix, and know what rerun re-runs
Once the agent commits the fix, push it. If your workflow runs on push or pull_request, the new commit starts a new run, so go back to step 1. git rev-parse HEAD now returns the new SHA.
Don't reach for gh run rerun --failed here. --failed will "Rerun only failed jobs, including dependencies" (manual), and GitHub's docs say a re-run uses the same GITHUB_SHA and GITHUB_REF as the original event. It tests the old commit, not the agent's fix.
Rerun is for failures that were not about the code: a flaky test, a runner that timed out, a network call that dropped. Then it saves you a full pipeline:
gh run rerun "$RUN_ID" --failed
gh run watch "$RUN_ID" --compact --exit-status
Add --debug to rerun with debug logging. GitHub allows re-runs up to 30 days after the first run, and up to 50 times per run (same docs page).
The whole loop in one script
Save this as ci-fail.sh and run it from the agent's worktree after you push:
#!/usr/bin/env bash
set -euo pipefail
sha=$(git rev-parse HEAD)
run=$(gh run list --commit "$sha" --limit 1 \
--json databaseId --jq '.[0].databaseId // empty')
if [ -z "$run" ]; then
echo "no run for $sha yet" >&2
exit 1
fi
gh run watch "$run" --compact --exit-status && exit 0
gh run view "$run" --log-failed \
| cut -f2- | tail -n 150 > ci-failure.log
wc -l ci-failure.log
Green, and it exits 0. Red, and you get a ci-failure.log of at most 150 lines in the folder the agent works in. Claude Code reads it with ! cat ci-failure.log, and any other agent reads it when you name the file. Run the exclude line from step 2 once per repository before the first use.
If this was useful
Trimming a CI log before an agent reads it is the same question you ask of any LLM system: what did it see, and what did that cost. My LLM Observability Pocket Guide is about answering that in production, with the tracing and eval tools that fit your team.
If you run several agents at once, Hermes IDE is free, source-available under BSL 1.1, runs on macOS, Windows and Linux, and isn't affiliated with Nous Research, Anthropic, OpenAI or any agent vendor. The code is on GitHub.

Top comments (0)