AI coding tools can write a function in seconds. The harder question is whether that function actually belongs in your codebase. Does it follow the...
For further actions, you may consider blocking this person and/or reporting abuse
The shift from “generated code” to “work after the AI finishes” is the metric I relate to most. A 200-line solution that creates 30 minutes of cleanup isn't productivity. Correction cost is probably a much better measure of AI-assisted development than output volume.
Exactly! 🙌 I think “correction cost” is such an important way to measure AI-assisted development. Generating hundreds of lines of code means very little if you spend the next 30 minutes fixing, cleaning up, and understanding it. The best tools are the ones that reduce the total work, not just the time spent writing code. Thanks for sharing this perspective!
Really enjoyed this comparison. The point that stood out to me most was that context matters more than raw code generation speed. Your debugging and multi-file testing examples make the differences between Cursor, Copilot, and Claude Code much clearer. I also agree that reviewing diffs and testing AI-generated code are still essential. Great practical breakdown!
Thank you so much! 😊 I completely agree, context ended up being one of the biggest differences between the tools. Raw code generation is impressive, but understanding the project, relationships between files, and the broader intent makes a much bigger difference in real-world development. And yes, reviewing diffs and testing are still non-negotiable!
Just a note, that's with different AI harnesses, not different AI tools per say.
Claude is known for their efficient indexing system for large codebases, but that's a harness thing, not a model thing, if you put Claude models via an API key in a different harness, it wont act the same. That's why with CoPilot, almost all the models 'feel' identical, despite being different vendors, perfect place to see the proof, try Haiku there, vs in Claude Code.
If you're going down that line, lemme widen the horizon a bit for you, download Qoder, from Alibaba (Qwen), give it a go, I've tried Copilot, Claude, Antigravity, Cursor, Codex and I found that Qoder is simply smarter, with smarter tools. It's also the only one that offers an actually free model. The Lite model, on a $20 paid plan, is actually uncapped and it's REALLY good... I pushed around 2m+ LOC output through it and it never hit a 'fair usage' cap, it never slowed down, it never got a 'servers too busy', not a single glitch at all.
That's a really important distinction, thanks for spelling it out. The "harness vs. model" point gets lost a lot, people benchmark a vendor's brand when they're actually benchmarking the scaffolding (context management, tool-calling, retrieval) built around it. The Copilot example is a great illustration since it flattens everything into feeling similar.
Appreciate the Qoder callout too. It's built on Alibaba's Qwen3-Coder model with a free plan that currently includes unlimited completions/edits plus limited chat and agent credits during its public preview, so worth noting some of that generosity may be preview-era pricing that shifts later, but definitely enough to justify trying it myself. Will give it a spin on a real repo and see how the indexing actually holds up.
Really liked the focus on useful code over generated code. The point about context being more important than raw generation speed really stood out. I also agree that reviewing diffs and testing AI-generated changes is essential, especially for multi-file refactors. Great practical comparison of Cursor, Copilot, and Claude Code.
Thank you! 🙌 “Useful code over generated code” really sums up the biggest takeaway for me. Fast generation is great, but the real value comes from how well the tool understands the context and how much follow-up work the code creates. Especially with multi-file changes, reviewing and testing everything is still essential. Glad you enjoyed the comparison!
도구가 만든 코드 줄 수보다 “AI가 끝낸 뒤 내가 얼마나 고쳐야 했는가”를 생산성 지표로 본 점이 설득력 있습니다. 특히 구현을 보기 전에 기존 코드를 설명하게 하고, 변경 금지 조건을 적은 뒤 작은 단위로 diff와 테스트를 확인하는 7단계는 어떤 도구를 쓰더라도 재현 가능한 기준이 되겠네요.
정확히 그게 핵심입니다. 생성된 코드 줄 수는 허상 지표입니다. AI가 200줄을 작성했는데 그 중 180줄을 다시 고쳐야 한다면 그건 생산성이 아니라 뒷수습입니다. "수정 전 먼저 설명하게 하기" 단계는 아마 대부분의 사람들이 가장 많이 건너뛰는 부분이고 동시에 가장 많은 문제를 일으키는 부분입니다. 이 단계는 도구가 무언가를 변경하기 전에 컨텍스트를 제대로 이해하고 있다는 것을 증명하게 만듭니다. 작은 단위로 diff를 확인하는 방식은 뼈아픈 경험에서 나온 것입니다. AI가 한 번에 대규모로 생성한 변경 사항은 제대로 검토하기가 거의 불가능합니다. 작은 단위로 나누면 실제로 코드베이스에 반영되는 내용을 직접 통제할 수 있습니다.
What stuck from your month is that the tool barely mattered once the handoff was wrong. On client work I started forcing a four-line sticky at the top of the file before any pair session: goal in one sentence, constraints the model must not invent, the one acceptance check, and what is explicitly out of scope.
If I skip that, every tool happily builds a different wrong thing at high confidence. If I keep it, even a weaker model stays useful because the conversation has a fence. Curious whether your worst sessions were “bad model” or “no shared brief,” and whether you ever pasted that brief back mid-session when the pair drifted.
Both, honestly, but more often no shared brief than bad model. The tool would take an underspecified prompt and commit to an interpretation early, then defend it for the rest of the session. That's not a model failing, that's a model doing exactly what an unconstrained prompt asked for. Your four-line sticky is basically the fix I converged on too, just less formalized. I'd usually restate constraints and acceptance criteria in the message itself. And yes, mid-session I'd paste it back verbatim when things drifted, especially in longer agent sessions where the model had clearly stopped weighting the original goal against however many tool calls of exploration it had done since. Worth turning that sticky into an actual file you attach or pin rather than retyping it. Cheaper than re-litigating scope three turns in.
Pinning the sticky as a real file (not retyped) is the upgrade I needed to hear — re-pasting from memory is how the NEVER line quietly mutates after turn six.
I’ve been putting WHO/DONE/NEVER/NEXT in one tiny file and attaching it at session start and after any tool-call streak longer than ~5. Mid-session reattach beats mid-session re-litigation every time.
Curious whether you version that file per ticket or keep one living sticky and overwrite DONE/NEVER as the work shifts.
The useful difference between the three tools was almost never the model — it was how loud each one was about uncertainty. The quiet ones filled gaps with confident wrong code; the noisy ones forced me to answer the gap before they wrote.
When you compare tools now, do you score them on first-pass speed, or on how often they ask before inventing?
That's a sharper way to put it than I managed in the post. Once I started paying attention to it, "how loud about uncertainty" mattered more than raw first-pass speed. A fast wrong answer just moves the cost to the review step. So no, I don't score on speed anymore. I watch for whether the tool stops and asks when the ticket is genuinely ambiguous, versus picking a plausible interpretation and running with it silently. The silent ones felt faster in the moment and slower by end of day.
Dropping speed as a score and watching for the silent-vs-ask split is exactly the right move — silent-fast is just deferred review debt.
I’ve started treating “asks when ambiguous” as a hard acceptance check on the tool itself: if it never asks on a deliberately underspecified ticket in a dry run, I don’t trust it on a real one. Curious whether you’ve got a canned ambiguous ticket you throw at new tools, or if you learn the loudness only mid-sprint.
Your debugging section is the part that matched my week. Generation quality across Cursor, Copilot, and Claude Code is closer than people admit. The gap shows up when the crash line is only a symptom three files upstream, and only the tool that will chase the data flow finds the real cause. The test prompt change is the other keeper: once I stopped asking for tests of the current implementation, the agent stopped encoding my bugs as expected behavior.
Glad that part landed. It was the one section I almost cut because it felt less quantifiable than the rest, but it's the real finding. Generation is commodity at this point. Tracing is where the tools separate, because it requires the model to distrust the stack trace instead of patching the line it points to. And yeah, the test-prompt change surprised me too. "Write tests for what this should do" versus "write tests for what this does" sounds like a small wording difference and produces completely different test suites, one of which quietly certifies your bugs as spec.
The one-month, three-tool rotation is exactly the right way to test these — most comparisons are written after a weekend. What I'd love to know: did you notice the tools' strengths changing with task type? I run a local model (Ollama) alongside a hosted assistant, and the split I landed on is local for boilerplate/test scaffolding, hosted for gnarly cross-file refactors — the local one wins on latency and not leaking code, the hosted one wins when context spans 10+ files. Also curious how you measured "better" — acceptance rate of suggestions, or time-to-working-code? Those two diverge a lot in my logs.
It did, and probably more than the headline comparison suggested. Boilerplate and scaffolding were close to a wash across all three. The separation showed up on cross-file work and on anything where the "bug" was a symptom of a decision made upstream. Your local/hosted split makes sense for the same reason: latency and privacy don't care about task type, but context span does, so routing on that axis is smart. On measurement, I leaned closer to time-to-working-code than acceptance rate, mainly because acceptance rate rewards suggestions that look right in isolation, and the failures I cared about were the ones that looked right and weren't. They diverge exactly where you'd expect, anywhere the model is confidently wrong rather than visibly unsure.
I really like the focus on context rather than raw code generation speed. In real projects, generating a function is usually the easy part—the difficult part is making sure it fits the existing architecture, handles edge cases, and doesn’t create problems elsewhere.
I’ve found the same thing when working on WordPress projects and plugins: an AI can produce a working solution quickly, but understanding the existing codebase and making the right change without breaking something else is where the real value is.
The comparison between Cursor, Copilot, and Claude Code is interesting, especially the point that each tool performs better depending on the type of task. Great experiment!
Appreciate that, and the WordPress parallel tracks closely with what I saw. Plugin ecosystems are a good stress test for exactly this because so much of the "correct" answer depends on conventions and hooks that aren't visible in the function you're asking for. The model can write valid PHP that ignores how the rest of the site expects to be extended. Which task types did you find each tool handled that context better or worse on?
Useful distinction between generated code and useful code. One measurement I’d add is a small task matrix with pinned tool versions and the same repository snapshot: time to first workable patch, review/correction time, test pass rate, and reverted changes. That makes “least correction” less subjective without pretending a month-long self-test is a controlled benchmark.
For debugging across files, I’d also record how much context was supplied versus discovered by the tool. Otherwise a tool that looks better may simply have received a more complete slice of the codebase. The practical gate for me is still the diff plus behavior tests, especially for refactors where a passing test suite may encode the old bug.
Fair, and I'll say it plainly: this was a self-test, not a benchmark, and I don't want to oversell "least correction" as more rigorous than it was. Your matrix is the right shape for anyone who wants to make this replicable: pinned versions, same snapshot, time-to-first-workable-patch, correction time, pass rate, reverted changes. The context-supplied-vs-discovered point is the one I'd flag as most important and least reported anywhere. A tool that autonomously greps the right three files before answering is doing something categorically different from one that only reasons over what's in the prompt, and comparisons that don't separate those two are comparing apples to a tool with a rake. Agreed on the gate too. For refactors specifically, a green test suite is only as good as whether the tests encode intended behavior or just current behavior, which is its own trap.
Sorry, but you lost me at "that distinction matters". This article is generated. Not that I have anything against it, but I'm not reading it further.
Anyway, wanted to share a bit about my experience with AI for coding so far: I'm using Codex inside IntelliJ Idea, and the project is fairly big, but it doesn't get much off track, largely because:
No hard feelings, you're welcome to sit this one out. For what it's worth, the process you're describing is worth its own post: a living design-doc corpus the agent consults and updates, tests run after every task rather than at the end, and periodic fresh-context code reviews so the reviewer isn't anchored to the same assumptions as the implementer. That last one especially, deliberately starting a clean chat for review, is a detail I didn't do enough of and probably should have. How often do you run that review pass, and has it ever caught something a same-context review missed?
Maybe the real benchmark for AI coding tools isn't tokens/sec, and more about how small the diff is between generated code and production-ready code at the end of days
That's a good compression of the whole post, honestly better than my own framing. Tokens/sec measures the model. Diff-to-production measures the tool plus your workflow plus how well it understood the codebase it's touching, which is the thing that actually costs time. If I rerun this, that's probably the one number I'd try to track directly instead of inferring it from correction time.