You spent a few good cycles, did some troubleshooting, and finally, it works!
With A.I., "working" does not always mean "ready to go" though. Something can work but have:
- Security issues
- Resource and cost issues
- Usability issues
- Incomplete features
We are making way more code than we can reasonably review.
The takeaway is not that you should read harder. You cannot read your way out of this. The takeaway is that you have to move the checking somewhere it fits.
So what should we do to handle this?
Whether you are shipping code, a spreadsheet, a policy draft, or a slide deck the A.I. built for you, here are some things to try.
1. Check the plan first, not the output. The plan might be forty lines or so, compared to output that is three thousand. Before you prompt for what you want to build, ask for the approach first: read that carefully, approve it, and then let it build. Ask it to do some research before starting.
2. Define what "wrong" would look like before you start. Try writing out: "X is wrong if Y does not match Z." Then try reviewing the work afterwards against this list.
3. Ask it what it changed that you did not ask for. "List every change you made that was not required by what I asked. For each one, say why." A.I. tools quietly improve things. Renamed a field. Moved a file. Swapped a library. Individually reasonable, collectively how you end up somewhere you never chose.
4. Focus on consequence, not size. Careful not to spend too much time reading every change and line. For critical functionality, read more closely. For other things, read at a high level.
5. Do a security pass. Once the A.I. claims to be done, tell it to check the work it did for any security issues or gaps. I have seen the A.I. forget to implement authentication and basic security from time to time, especially while working on bigger tasks and prompts.
6. Ask the A.I. to explain what it did. Sometimes I ask my agent to give me a summary of what it did and why. You can also ask it to explain how to manually test components or chat through any questions you have.
A few bonus ones for developers
Commit often, not once per session. Be sure to do commits per milestone or feature. This makes things way easier to review ultimately, since you get smaller chunks of code changes.
Review the test, then lock it in. Have the A.I. write the test from the requirement before it writes the code. Then review the tests to ensure they evaluate what you need. From there, you can use these tests as your bar for the coder. Do not let the coder adjust the tests once locked in, this is key.
Some things are best for humans. If the risks are high, you should double check the work yourself. Do not let the automation handle critical actions like deletes or live data manipulation without supervision and review.
Commit before you start. Otherwise, you risk the coding agent wiping out your uncommitted changes. I had this happen to me once, so it is worth throwing out there.
You own it
Though we can move faster than ever, we need to make sure our workflows and processes for quality keep up.
By creating a process, and having some checks before and after you start, you can keep up with the new pace at which everyone is moving.
Example prompts
These are ones I have used, cleaned up so you can paste them as is.
Before you start
Get a plan you can read instead of output you cannot.
I have outlined the requirements and specs for this project in <FILE>.
Before we write any code, do the relevant research and read the whole
requirements file, then write a detailed design document that will guide
the implementation. Save it as design.plan.md in the docs/ directory.
I will review it before we start building.
Make every requirement fail-able.
Rewrite these requirements so each one has a stable ID (REQ-001, REQ-002)
and an "Acceptance:" line describing exactly what must be true for it to
count as done. If a requirement cannot be given a checkable acceptance
criterion, flag it instead of guessing.
Anything it flags is something you had not actually decided yet.
Confirm you have a clean starting point.
Before we begin, confirm the working tree is clean and everything is
committed. If it is not, stop and tell me what is uncommitted.
Guardrails to paste in every time
Stop it from acting before you have answered.
DO NOT MAKE CHANGES WITHOUT ASKING. First investigate and answer my
question. Do not touch the code until I say go.
Stop it from quietly editing the plan.
Do not change the scope of our work in PROGRESS.md. If work remains, we
handle it later. Never delete tasks.
Make the dry run mandatory for anything that touches real data.
Default to read-only. Before any action that writes, deletes, or changes
live data, show me the full plan of what you are about to do and wait for
me to say go. One plan at a time, no chaining. If any part of it cannot be
undone, say so explicitly at the top.
After it says it is done
Find out what it changed on its own.
List every change you made that was not required by what I asked.
For each one, say why you made it.
Run a security pass and send it somewhere you will actually read it.
Review this application and make sure it is secure, both when run as a
normal user and as an administrator. Include code snippets and links to
the relevant code so I can review each finding myself. Put everything in
an MD document in the docs folder rather than answering here.
Shorter version for a smaller project:
Do a security review of this entire app and repo. Identify anything that
needs to be addressed before it goes live.
Writing findings to a file instead of the chat window is a nice trick. When the chat is gone, the file is still there tomorrow.
Ask how to check it yourself.
How do I test this myself? Write a short guide covering how to manually
verify each piece, including anything I need to set up first.
Asking for testing instructions hands you something you can go check independent of the a.i. coder
For developers
Lock the tests before any code exists.
Write the tests from the spec first, before any implementation exists.
Once I approve them, they are locked: you write code to pass those tests
and you do not modify or weaken them. If you believe a test is wrong,
stop and tell me. That is a conversation, not an edit.
Top comments (0)