A few technical points to add on the “Data and auth” column, since that’s where most vibe-coded apps fail:
Row-level security is the first thing to audit. Many generated apps rely on client-side checks only. If you’re on Supabase or Postgres, confirm RLS policies exist on every table and test them by calling the API directly with another user’s JWT, not through the UI.
Check for leaked secrets. Generated code often puts API keys in frontend env vars (VITE_*, NEXT_PUBLIC_*), which ship to the browser. Anything sensitive should live in a server-side function.
Validate on the server, not just in the form. Client validation is cosmetic. Re-validate every payload at the API layer.
For the “Yours” tools (Cursor, Claude Code, Copilot), add guardrails. Run tests and a linter in CI, and require a reviewed diff before merging. Agents tend to “fix” a failing test by editing the test instead of the code.
Pin and audit dependencies. Agents sometimes add packages you don’t need, or hallucinate package names, which is a supply-chain risk. Check package.json diffs carefully.
On lock-in: the practical test is whether you can clone the repo, run it locally with your own DB, and deploy it elsewhere without the platform. If not, it’s a prototype, not a product.
“Data and auth” is where most vibe-coded apps break, so it’s worth spelling out how to audit it. A few thoughts building on your points:
RLS testing: Calling the API directly with another user’s JWT is the right way to verify it. The UI will hide problems that the database layer exposes. I’d add that a table with RLS enabled but no policies blocks everything, while a table with RLS disabled is wide open, and generated schemas often end up in one of those two states without anyone noticing.
Leaked secrets: The VITE_* and NEXT_PUBLIC_* prefixes are a good rule of thumb. If a variable carries one of those prefixes, assume it’s public. Sensitive keys belong in server-side functions, and any key that has already shipped to a browser should be rotated, not just moved.
Server-side validation: Agreed. Client validation is a UX feature, not a security control. Schema validation at the API layer (Zod, Pydantic and similar) keeps this manageable.
Guardrails for the “Yours” tools: The point about agents editing the failing test instead of the code is something I’ve seen too. CI with tests and a linter, plus a reviewed diff before merge, catches it. Treating agent output like a junior contributor’s PR is a good mental model.
Dependencies: Hallucinated package names are a real supply-chain risk, since attackers can register those names. Reviewing package.json diffs and using a lockfile with audit tooling is cheap insurance.
Lock-in: Your clone, run locally with your own DB, deploy elsewhere test is a great litmus check. I’ll fold it into the comparison as a “prototype vs. product” criterion.
Most roundups in this space stop at “which one makes the prettiest first preview,” so judging on the second, third, and tenth change is the right approach.
Three things I liked:
The “Yours” rating for editors and terminal agents: It’s honest. Cursor and Claude Code don’t give you a backend, so scoring them on data/auth or deployment would have been misleading.
The Windsurf → Devin Desktop note: Lots of older comparisons still point people to a product name that has changed.
The lock-in and exit-strategy section: People usually find out about export limits after they’ve built something real.
One question: where did the app builders actually break for you? You mention regressions when prompts get broad, but I’d love a concrete example, like a permissions rule or a database change that went wrong after a few iterations. That’s the kind of thing that would help a first-time builder decide between Lovable, Bolt, and Base44.
Judging on the second, third, and tenth change was the whole point, since the first preview is almost always impressive.
Glad the “Yours” rating, the Windsurf → Devin Desktop note, and the exit-strategy section were useful. Export limits are the thing people only find out about after they’ve built something real.
On where the app builders broke: the pattern I saw most is that the tool handles a narrow, specific prompt well but starts undoing earlier work when a prompt is broad. The risky areas are permissions and data. A prompt like “add team accounts” or “let users share projects” can quietly change access rules or the schema in ways you only notice later, for example a role check that stops applying to a page you didn’t touch, or a table change that breaks a form that worked two iterations ago.
I can’t give you a clean, verified head-to-head on Lovable vs Bolt vs Base44 for a specific permissions bug, and I’d rather not invent one. For a first-time builder, though, the practical advice is the same across all three:
Make permissions and database changes one small prompt at a time.
Test with a second, non-admin account after every change.
Check the version history or rollback option before you start, because it’s the safety net you’ll lean on most.
If you want, I can write up a small test script for this (add a role, change a table, re-test old flows) and post it as a follow-up.
That’s a useful rule of thumb, especially testing with a second non-admin account after every change. Most first-time builders only check with the account they built the app with, which is exactly where a broken role check hides. And no worries about the head-to-head, I’d take an honest “I haven’t verified that” over a made-up comparison.
Yes, please post the test script as a follow-up. Add-a-role, change-a-table, re-test-old-flows is a good structure. If you can include a short checklist of old flows to re-run after each prompt (login, forms, shared pages), that would make it easy to reuse across Lovable, Bolt and Base44.
Thanks, and agreed: the builder account is the one place a broken role check can’t show up, because it passes every check by design.
Here’s the test script. It works the same on Lovable, Bolt, and Base44, since it only tests what you can see in the running app.
Before you start
Create two test accounts: one admin (your builder account) and one plain user with no special role.
Note the current version or checkpoint so you can roll back.
Write down 5 to 8 “old flows” (checklist below) and what a pass looks like for each.
The loop (repeat for every permissions or data prompt)
Add the role. Prompt for one change only, e.g. “add an Editor role that can edit but not delete projects.”
Re-test old flows. Run the full checklist below as the admin, then as the plain user.
Change the table. Prompt for one schema change only, e.g. “add a due date field to tasks.”
Re-test old flows again. Same checklist, both accounts.
Test the new feature last, from the non-admin account first.
If anything fails, roll back and re-prompt with a narrower request instead of asking the tool to “fix it.”
Old-flow checklist
Sign up and log in with a fresh email, then log out and back in
Password reset or magic link still works
Every existing form submits and the data shows up where it should
Editing and deleting an existing record works (and is blocked where it should be)
A shared or public page loads for a logged-out visitor
A private page does not load for a logged-out visitor or the wrong role
The plain user can’t see or change another user’s data
Old records created before the change still display correctly
The two items that catch the most problems are the private-page check and the “can’t see another user’s data” check, since a broken role check usually shows up there first.
I haven’t run this script head-to-head across all three tools, so treat it as a method rather than a benchmark. If you run it on Lovable, Bolt, or Base44 and find something that breaks, I’d like to hear about it.
This is exactly the kind of reusable method I was hoping for. Alternating “change one thing, re-test everything, both accounts” is what makes it work, because it turns regressions into something you catch at the step that caused them instead of three prompts later.
Two small additions I’d make when I run it:
Log the result per step: A simple pass/fail table (checklist item × admin/plain user) after each prompt. When something breaks, you can see exactly which prompt introduced it, and that makes rollback decisions much easier.
Add one “direct URL” check: Paste the URL of a private page into a logged-out window, not just click through the UI. Role checks that only hide a button but don’t block the route are a common miss.
I’ll try it on one of the three builders with a small project and report back if anything breaks. Thanks for sharing the checklist and for being upfront that it’s a method rather than a benchmark.
Your pass/fail table is the piece most people skip, and I'd tie each row to something you can get back to, a GitHub sync commit or the builder's checkpoint, because "prompt 7 broke the plain user" only helps if you can still land on prompt 6… I'd also run your direct URL check once against the raw response, logged in as the plain user with the Network tab open in dev tools, since Rafid's salary-field example is a page hiding a field the server still sends. Report back when you run it, I'd like to see which step catches the first break.
Full-stack developer and tech writer who enjoys making complex ideas simple. I write about web development, system design, and practical coding techniques to help developers improve more quickly.
The “second, third, and tenth change” line is the best way I’ve seen to judge these tools. Every app builder looks great on the first prompt. The real test is whether the project is still easy to work on after a dozen revisions.
I’ve seen the pattern you describe in the bolt-and-regression point. A broad instruction fixes one screen and quietly breaks another, so a short manual test pass after each big change saves a lot of pain. Credit-based pricing makes it worse, because every “fix the thing you just broke” prompt uses up allowance too.
I also appreciated the note on Windsurf becoming Devin Desktop. Comparison posts go stale fast in this space, and a lot of older roundups still list features for products that have changed or been renamed. The “last reviewed” date at the top helps a lot.
The “what these tools can’t replace” section makes a point that gets less attention than it should. A public URL shows an app is reachable, not that it’s safe. Row-level permissions are where I’d tell non-technical founders to slow down and test with two different accounts before real users arrive.
One idea for a follow-up: take the same small app (auth, a CRUD table, one role-based rule) and build it in Lovable, Cursor and Claude Code. Then compare how many prompts it took, what broke along the way, and how easy it was to export and run the result elsewhere. That would put the exit-strategy advice to a real test.
Which of the three categories do you think people outgrow first, app builders or editors?
Thanks for the thoughtful read. Judging a tool by its tenth change instead of its first screenshot was the main idea behind the comparison.
You’re right about credits. With a credit-based builder, a regression costs you twice: the broken feature, then the prompt to repair it. A short manual pass after each big change, ideally with a saved checklist of core flows, is cheap insurance.
The two-account test is the one I’d push hardest for non-technical founders. Log in as user A, create a record, then try to reach it as user B through the UI and by changing the URL or ID. It takes minutes and catches the most damaging class of mistakes before real users arrive.
I like your follow-up idea. The same small app (auth, a CRUD table, one role-based rule) in Lovable, Cursor and Claude Code would be a better test than my scorecard, which is editorial judgment, not measured results. I’d track prompts to a working version, regressions, and whether the exported project runs outside the original tool. Adding a “change the role rule after launch” step would test the tenth-change claim directly.
On your question, my guess is app builders get outgrown first, usually around the point where the app needs custom logic, tighter permissions, or integrations the builder doesn’t handle cleanly. That’s also when export and code control start to matter. But it depends on the person. A non-developer may never move to an editor, and some developers skip builders entirely.
Have you seen people hit that wall, and what triggered it?
Full-stack developer and tech writer who enjoys making complex ideas simple. I write about web development, system design, and practical coding techniques to help developers improve more quickly.
Yes, and the trigger is usually permissions or integrations. A rule like “managers see their team’s records but not salary fields” often looks right in the UI and still leaks through the API. A half-supported webhook or payment flow forces the same choice: fight the tool or export the code.
Another sign someone has outgrown a builder is when their prompts start with “don’t touch anything else, only change X.” They’re trying to get an editor’s precision through a chat box.
I’d be curious whether your follow-up test shows the same pattern, especially at the “change the role rule after launch” step.
The “don’t touch anything else, only change X” prompt is a good signal. It usually means the person wants a reviewable diff and is trying to get one out of a tool that regenerates things.
Your salary-field example is also why I’d test permissions at the API level, not just in the UI. Hiding a field on screen doesn’t mean the server won’t return it. I’d check the raw response as the manager role instead of trusting what the page shows.
If I run the follow-up, I’ll make the role-rule change the final step and log it separately: prompts needed, whether anything else regressed, and whether the restriction held at the API. That should show whether the pattern you describe holds up in practice. Thanks for pushing the idea further, it’s a stronger test than my scorecard alone.
Mayur Upadhyay delivers tech-driven solutions in Web Development, e-commerce development, and mobile app development, helping businesses to grow worldwide.
The “second, third, and tenth change” line is the best test in the whole comparison. Most roundups judge tools on the first prompt, but the real difference shows up once you’re iterating on a project that’s no longer a blank slate.
I also appreciated that every “Where it falls short” section says the same thing in different words: a polished preview doesn’t prove the permissions and data rules are correct. That’s the gap I see people miss most with app builders, and it’s easy to forget when the demo looks this good.
The Windsurf → Devin Desktop note was a useful catch too. Older comparisons are still ranking for the old name.
Curious about one thing: for the “Yours” ratings on data/auth and deployment (Cursor, Claude Code, etc.), which backend and hosting stack do you usually pair them with? That choice seems to decide whether the editor and terminal-agent route ends up as smooth as the builders.
Thanks, that “second, third, and tenth change” test is the part I’d defend hardest. A first prompt tells you almost nothing about a tool. The real signal comes once the project has structure, and the tool has to change one thing without quietly breaking three others.
Your point about permissions and data rules matches what I saw most. The preview looks finished long before the access logic is actually correct, so I’d always check row-level rules and auth flows by hand before trusting a demo.
On the pairing question: I don’t have one fixed stack. The editor and terminal-agent tools work best with a backend that has explicit, readable rules, such as Postgres with row-level security or Supabase policies in version control. The agent can then edit them as code, and you can review the diff. For hosting, I lean toward platforms with preview deployments per branch (Vercel or Netlify for the front end, a managed container host for APIs), since that gives you a safe place to test each agent change before it reaches production. The builders feel smoother mainly because they bundle all of this for you. With Cursor or Claude Code, you have to choose it yourself, which is more work but also gives you more control.
Good catch on Windsurf → Devin Desktop too. Old comparisons keep ranking under the previous name, so I wanted that note to be easy to find.
Mayur Upadhyay delivers tech-driven solutions in Web Development, e-commerce development, and mobile app development, helping businesses to grow worldwide.
That makes sense. Supabase policies and Postgres RLS in version control are the strongest part of that setup, because the agent edits the same files a reviewer reads, and a bad access rule shows up as a diff instead of a surprise in production.
Per-branch preview deployments are a good safety net for the same reason. I’d add one thing: a preview only helps if it runs against seeded test data with realistic roles, so you can log in as a regular user and confirm they can’t see another user’s rows. Otherwise, the preview can look right while the rules are still wrong.
It also explains why the builders feel smoother. They aren’t better at the hard part; they just make those choices for you. Thanks for spelling out the stack; it’s a useful reference for anyone choosing between the two routes.
A preview that only has one admin-ish test account can look perfectly fine while the policies are wrong. Two roles are the minimum: a regular user, and a second user whose rows the first one should never see.
If you want to go a step further, turn that check into a small automated test that runs against each preview. Sign in as user A, try to read or update user B’s rows, and assert it fails. Add an anonymous request too, since a missing policy on a new table is the easiest thing to ship by accident. That way the agent can’t quietly break an access rule and still get a green deploy.
And yes, that’s the trade-off with the builders. They’re great when you want something working fast, but the access rules are exactly where you want to see and review the choices yourself once real user data is involved.
The distinction between "AI building for you" (Lovable, Bolt) and "AI helping you build" (Cursor, Claude Code) is the most crucial takeaway here. The "exit strategy" warning is also spot-on — I've seen too many founders get trapped in app builders because they didn't check if they could export the backend and migrate the data. For actual developers, Cursor + Claude Code is an absolute cheat code, but only if you still know how to read a stack trace.
That "AI building for you" vs. "AI helping you build" split was the main thing I wanted readers to take away, so I'm glad it landed. Your point about the exit strategy is a good one. Before committing to any app builder, it's worth checking that you can export the code, move the backend, and migrate your data. Otherwise, a quick prototype can turn into lock-in.
I fully agree with your last line too. Cursor + Claude Code is powerful, but it works best when you can read the stack trace and understand what the AI just changed. The tools speed you up, and they still need someone who can debug.
Have you had to migrate off one of these builders yourself? I'd be curious which one it was and how it went.
Thanks for reading! A quick note on how I approached this: I scored the tools on what happens after the first prompt (data, auth, iteration, exit options), not on how good the first preview looks.
I’m curious about two things from people who’ve shipped with these:
Which tool held up best by the 10th change, not just the first?
Have you hit lock-in or export problems with any app builder?
Also, Windsurf has moved to Devin Desktop, so if you’ve tried the new version, I’d like to hear how it compares. I’ll update the post with anything useful from the comments.
Judging the tools on what happens after the first prompt, instead of how good the first preview looks, is the right lens. Most comparisons skip that.
The “Exit strategy” point is the one I’d push hardest. Credit-based pricing and lock-in only really hurt around the tenth revision, when you’re deep in a project and migrating is no longer cheap. A quick export test early on (can I get the code and the data out cleanly?) would save a lot of people pain.
Also useful that you flagged the Windsurf → Devin Desktop change. Outdated tutorials are a real problem in this space.
One question: have you tried the same small task across a builder like Lovable and a terminal agent like Claude Code? I’m curious where each one starts to struggle, especially on auth and permissions, since you note that generated UIs can hide broken access rules.
Thanks, that’s the lens I was going for. Most comparisons stop at the first preview, and the tenth revision is where the real differences show up.
Agreed on the export test. A good early check is to pull the code and data out in the first hour, while leaving is still cheap.
On your question: no, I haven’t run the same task across Lovable and Claude Code. The scorecard in the post is my editorial judgment from documented capabilities, not measured test results, and I flagged that in the post on purpose. I’d rather say that than imply a head-to-head test I didn’t do.
Your auth and permissions question is the right one to test, though. If I ran it, I’d pick a small multi-user app, like notes or tasks where each user should only see their own records. Then I’d check three things:
Can user A read or edit user B’s data by changing an ID in the URL or API call?
Are the access rules enforced on the backend or database side, or only hidden in the UI?
What happens to those rules after a few follow-up prompts that touch the same feature?
My expectation, not a finding, is that the builders struggle where the UI looks right but the rules aren’t enforced. With a terminal agent, the weak point is that you have to specify and verify the rules yourself, since it won’t supply a backend. If you’ve seen it go differently in your own tests, I’d like to hear where each one broke.
Levine Mundro has over 30 years of experience in sales and marketing. He focuses on driving growth, building client relationships, and shaping leadership strategy.
The line “the best option depends on whether you want AI to build the app for you or help you build and maintain the code yourself” is the most useful sentence in the post. Most people pick a tool by how good the demo looks, then get stuck when they outgrow it.
I’d add that this doesn’t have to be an either/or choice. A workflow I’ve seen work is to prototype in a builder like Lovable or Bolt, sync the project to GitHub, and then continue in Cursor or a terminal agent once the requirements get real. The builder gets you to something testable in a day, and the editor takes over when you need proper control over the code. The “code and project control” column in your scorecard is basically the checklist for whether that handoff will be smooth.
I also liked that the Claude Code vs Gemini CLI section says to try both on the same small task in your own repo instead of declaring a winner. That’s more honest than most comparisons.
A question: for the builders you rated “Medium” on code control, how painful was the export in practice? Did the exported project run locally without much rework, or did it stay tied to the platform’s services? That detail would help people decide before they commit.
Thanks, glad that line landed. And your workflow is a good one: prototype in a builder, sync to GitHub, then continue in an editor or terminal agent once the requirements get real. That’s the handoff the “Code control” column is meant to help people judge.
On your question, I’ll be upfront that the ratings in the scorecard come from the documented capabilities of each tool (GitHub sync, code export, hosting and backend ties), not from a hands-on export test of every project. So I can’t tell you how a given export runs locally. “Medium” was my way of saying the code is yours to take, but the app may lean on platform services such as hosting, auth, and the database. That’s where the rework usually shows up, so I’d treat it as a checklist rather than a guarantee.
If you’ve done the export yourself, I’d like to hear how it went. Did the app run locally with only the env vars and backend connection reconfigured, or did you have to rewire auth and data? Those details would be more useful to readers than my ratings.
Sidra Jefferi, Marketing Director, uses her knowledge of corporate actions and global markets to help improve reliable internet access in rural and underserved areas.
One thing that decides whether a tool survives past the demo is how it handles regressions across iterations, and I’d love to see that benchmarked.
With app builders, a broad prompt like “add filtering” can quietly rewrite shared components and break a feature that worked two prompts ago. Editors and terminal agents handle this better because the change shows up as a git diff, so you can review it, run tests and revert. If an app builder doesn’t give you real version control with GitHub sync, treat every prompt as a risky deploy.
The credit-based pricing point is also underrated. Debugging loops burn credits fastest, because the model re-reads context and retries. Writing a precise spec up front (schema, roles, validation rules) usually costs less than iterating your way to it.
For terminal agents, scoping matters more than model choice. A narrow task with explicit acceptance criteria (“make these failing tests pass, don’t touch the public API”) gives far more reliable output than “refactor this module”.
Thanks for flagging the Windsurf to Devin Desktop change. Stale comparisons are a real problem in this space.
The "don't touch the public API" half of your example is the part I'd push on, because "make these failing tests pass" is a check and "don't touch the public API" is a wish, and an agent can't fail a wish. I'd turn that half into a command too, an API snapshot test or a diff over the exported types, so both halves come back with a result… It's a big piece of why cairn works the way it does (the harness in my comment on the post), a task's verify command gets quoted back before the work starts so the bar can't get negotiated after, and TDD tasks commit red and green separately, so when an agent "fixes" a failing test by editing it, like Devers said, you can see it by diffing the test between the two commits.
A few technical points to add on the “Data and auth” column, since that’s where most vibe-coded apps fail:
VITE_*, NEXT_PUBLIC_*), which ship to the browser. Anything sensitive should live in a server-side function.package.jsondiffs carefully.On lock-in: the practical test is whether you can clone the repo, run it locally with your own DB, and deploy it elsewhere without the platform. If not, it’s a prototype, not a product.
“Data and auth” is where most vibe-coded apps break, so it’s worth spelling out how to audit it. A few thoughts building on your points:
RLS testing: Calling the API directly with another user’s JWT is the right way to verify it. The UI will hide problems that the database layer exposes. I’d add that a table with RLS enabled but no policies blocks everything, while a table with RLS disabled is wide open, and generated schemas often end up in one of those two states without anyone noticing.
Leaked secrets: The
VITE_*andNEXT_PUBLIC_*prefixes are a good rule of thumb. If a variable carries one of those prefixes, assume it’s public. Sensitive keys belong in server-side functions, and any key that has already shipped to a browser should be rotated, not just moved.Server-side validation: Agreed. Client validation is a UX feature, not a security control. Schema validation at the API layer (Zod, Pydantic and similar) keeps this manageable.
Guardrails for the “Yours” tools: The point about agents editing the failing test instead of the code is something I’ve seen too. CI with tests and a linter, plus a reviewed diff before merge, catches it. Treating agent output like a junior contributor’s PR is a good mental model.
Dependencies: Hallucinated package names are a real supply-chain risk, since attackers can register those names. Reviewing
package.jsondiffs and using a lockfile with audit tooling is cheap insurance.Lock-in: Your clone, run locally with your own DB, deploy elsewhere test is a great litmus check. I’ll fold it into the comparison as a “prototype vs. product” criterion.
Most roundups in this space stop at “which one makes the prettiest first preview,” so judging on the second, third, and tenth change is the right approach.
Three things I liked:
One question: where did the app builders actually break for you? You mention regressions when prompts get broad, but I’d love a concrete example, like a permissions rule or a database change that went wrong after a few iterations. That’s the kind of thing that would help a first-time builder decide between Lovable, Bolt, and Base44.
Judging on the second, third, and tenth change was the whole point, since the first preview is almost always impressive.
Glad the “Yours” rating, the Windsurf → Devin Desktop note, and the exit-strategy section were useful. Export limits are the thing people only find out about after they’ve built something real.
On where the app builders broke: the pattern I saw most is that the tool handles a narrow, specific prompt well but starts undoing earlier work when a prompt is broad. The risky areas are permissions and data. A prompt like “add team accounts” or “let users share projects” can quietly change access rules or the schema in ways you only notice later, for example a role check that stops applying to a page you didn’t touch, or a table change that breaks a form that worked two iterations ago.
I can’t give you a clean, verified head-to-head on Lovable vs Bolt vs Base44 for a specific permissions bug, and I’d rather not invent one. For a first-time builder, though, the practical advice is the same across all three:
If you want, I can write up a small test script for this (add a role, change a table, re-test old flows) and post it as a follow-up.
That’s a useful rule of thumb, especially testing with a second non-admin account after every change. Most first-time builders only check with the account they built the app with, which is exactly where a broken role check hides. And no worries about the head-to-head, I’d take an honest “I haven’t verified that” over a made-up comparison.
Yes, please post the test script as a follow-up. Add-a-role, change-a-table, re-test-old-flows is a good structure. If you can include a short checklist of old flows to re-run after each prompt (login, forms, shared pages), that would make it easy to reuse across Lovable, Bolt and Base44.
Thanks, and agreed: the builder account is the one place a broken role check can’t show up, because it passes every check by design.
Here’s the test script. It works the same on Lovable, Bolt, and Base44, since it only tests what you can see in the running app.
Before you start
The loop (repeat for every permissions or data prompt)
Old-flow checklist
The two items that catch the most problems are the private-page check and the “can’t see another user’s data” check, since a broken role check usually shows up there first.
I haven’t run this script head-to-head across all three tools, so treat it as a method rather than a benchmark. If you run it on Lovable, Bolt, or Base44 and find something that breaks, I’d like to hear about it.
This is exactly the kind of reusable method I was hoping for. Alternating “change one thing, re-test everything, both accounts” is what makes it work, because it turns regressions into something you catch at the step that caused them instead of three prompts later.
Two small additions I’d make when I run it:
I’ll try it on one of the three builders with a small project and report back if anything breaks. Thanks for sharing the checklist and for being upfront that it’s a method rather than a benchmark.
Your pass/fail table is the piece most people skip, and I'd tie each row to something you can get back to, a GitHub sync commit or the builder's checkpoint, because "prompt 7 broke the plain user" only helps if you can still land on prompt 6… I'd also run your direct URL check once against the raw response, logged in as the plain user with the Network tab open in dev tools, since Rafid's salary-field example is a page hiding a field the server still sends. Report back when you run it, I'd like to see which step catches the first break.
The “second, third, and tenth change” line is the best way I’ve seen to judge these tools. Every app builder looks great on the first prompt. The real test is whether the project is still easy to work on after a dozen revisions.
I’ve seen the pattern you describe in the bolt-and-regression point. A broad instruction fixes one screen and quietly breaks another, so a short manual test pass after each big change saves a lot of pain. Credit-based pricing makes it worse, because every “fix the thing you just broke” prompt uses up allowance too.
I also appreciated the note on Windsurf becoming Devin Desktop. Comparison posts go stale fast in this space, and a lot of older roundups still list features for products that have changed or been renamed. The “last reviewed” date at the top helps a lot.
The “what these tools can’t replace” section makes a point that gets less attention than it should. A public URL shows an app is reachable, not that it’s safe. Row-level permissions are where I’d tell non-technical founders to slow down and test with two different accounts before real users arrive.
One idea for a follow-up: take the same small app (auth, a CRUD table, one role-based rule) and build it in Lovable, Cursor and Claude Code. Then compare how many prompts it took, what broke along the way, and how easy it was to export and run the result elsewhere. That would put the exit-strategy advice to a real test.
Which of the three categories do you think people outgrow first, app builders or editors?
Thanks for the thoughtful read. Judging a tool by its tenth change instead of its first screenshot was the main idea behind the comparison.
You’re right about credits. With a credit-based builder, a regression costs you twice: the broken feature, then the prompt to repair it. A short manual pass after each big change, ideally with a saved checklist of core flows, is cheap insurance.
The two-account test is the one I’d push hardest for non-technical founders. Log in as user A, create a record, then try to reach it as user B through the UI and by changing the URL or ID. It takes minutes and catches the most damaging class of mistakes before real users arrive.
I like your follow-up idea. The same small app (auth, a CRUD table, one role-based rule) in Lovable, Cursor and Claude Code would be a better test than my scorecard, which is editorial judgment, not measured results. I’d track prompts to a working version, regressions, and whether the exported project runs outside the original tool. Adding a “change the role rule after launch” step would test the tenth-change claim directly.
On your question, my guess is app builders get outgrown first, usually around the point where the app needs custom logic, tighter permissions, or integrations the builder doesn’t handle cleanly. That’s also when export and code control start to matter. But it depends on the person. A non-developer may never move to an editor, and some developers skip builders entirely.
Have you seen people hit that wall, and what triggered it?
Yes, and the trigger is usually permissions or integrations. A rule like “managers see their team’s records but not salary fields” often looks right in the UI and still leaks through the API. A half-supported webhook or payment flow forces the same choice: fight the tool or export the code.
Another sign someone has outgrown a builder is when their prompts start with “don’t touch anything else, only change X.” They’re trying to get an editor’s precision through a chat box.
I’d be curious whether your follow-up test shows the same pattern, especially at the “change the role rule after launch” step.
The “don’t touch anything else, only change X” prompt is a good signal. It usually means the person wants a reviewable diff and is trying to get one out of a tool that regenerates things.
Your salary-field example is also why I’d test permissions at the API level, not just in the UI. Hiding a field on screen doesn’t mean the server won’t return it. I’d check the raw response as the manager role instead of trusting what the page shows.
If I run the follow-up, I’ll make the role-rule change the final step and log it separately: prompts needed, whether anything else regressed, and whether the restriction held at the API. That should show whether the pattern you describe holds up in practice. Thanks for pushing the idea further, it’s a stronger test than my scorecard alone.
The “second, third, and tenth change” line is the best test in the whole comparison. Most roundups judge tools on the first prompt, but the real difference shows up once you’re iterating on a project that’s no longer a blank slate.
I also appreciated that every “Where it falls short” section says the same thing in different words: a polished preview doesn’t prove the permissions and data rules are correct. That’s the gap I see people miss most with app builders, and it’s easy to forget when the demo looks this good.
The Windsurf → Devin Desktop note was a useful catch too. Older comparisons are still ranking for the old name.
Curious about one thing: for the “Yours” ratings on data/auth and deployment (Cursor, Claude Code, etc.), which backend and hosting stack do you usually pair them with? That choice seems to decide whether the editor and terminal-agent route ends up as smooth as the builders.
Thanks, that “second, third, and tenth change” test is the part I’d defend hardest. A first prompt tells you almost nothing about a tool. The real signal comes once the project has structure, and the tool has to change one thing without quietly breaking three others.
Your point about permissions and data rules matches what I saw most. The preview looks finished long before the access logic is actually correct, so I’d always check row-level rules and auth flows by hand before trusting a demo.
On the pairing question: I don’t have one fixed stack. The editor and terminal-agent tools work best with a backend that has explicit, readable rules, such as Postgres with row-level security or Supabase policies in version control. The agent can then edit them as code, and you can review the diff. For hosting, I lean toward platforms with preview deployments per branch (Vercel or Netlify for the front end, a managed container host for APIs), since that gives you a safe place to test each agent change before it reaches production. The builders feel smoother mainly because they bundle all of this for you. With Cursor or Claude Code, you have to choose it yourself, which is more work but also gives you more control.
Good catch on Windsurf → Devin Desktop too. Old comparisons keep ranking under the previous name, so I wanted that note to be easy to find.
That makes sense. Supabase policies and Postgres RLS in version control are the strongest part of that setup, because the agent edits the same files a reviewer reads, and a bad access rule shows up as a diff instead of a surprise in production.
Per-branch preview deployments are a good safety net for the same reason. I’d add one thing: a preview only helps if it runs against seeded test data with realistic roles, so you can log in as a regular user and confirm they can’t see another user’s rows. Otherwise, the preview can look right while the rules are still wrong.
It also explains why the builders feel smoother. They aren’t better at the hard part; they just make those choices for you. Thanks for spelling out the stack; it’s a useful reference for anyone choosing between the two routes.
A preview that only has one admin-ish test account can look perfectly fine while the policies are wrong. Two roles are the minimum: a regular user, and a second user whose rows the first one should never see.
If you want to go a step further, turn that check into a small automated test that runs against each preview. Sign in as user A, try to read or update user B’s rows, and assert it fails. Add an anonymous request too, since a missing policy on a new table is the easiest thing to ship by accident. That way the agent can’t quietly break an access rule and still get a green deploy.
And yes, that’s the trade-off with the builders. They’re great when you want something working fast, but the access rules are exactly where you want to see and review the choices yourself once real user data is involved.
The distinction between "AI building for you" (Lovable, Bolt) and "AI helping you build" (Cursor, Claude Code) is the most crucial takeaway here. The "exit strategy" warning is also spot-on — I've seen too many founders get trapped in app builders because they didn't check if they could export the backend and migrate the data. For actual developers, Cursor + Claude Code is an absolute cheat code, but only if you still know how to read a stack trace.
That "AI building for you" vs. "AI helping you build" split was the main thing I wanted readers to take away, so I'm glad it landed. Your point about the exit strategy is a good one. Before committing to any app builder, it's worth checking that you can export the code, move the backend, and migrate your data. Otherwise, a quick prototype can turn into lock-in.
I fully agree with your last line too. Cursor + Claude Code is powerful, but it works best when you can read the stack trace and understand what the AI just changed. The tools speed you up, and they still need someone who can debug.
Have you had to migrate off one of these builders yourself? I'd be curious which one it was and how it went.
Thanks for reading! A quick note on how I approached this: I scored the tools on what happens after the first prompt (data, auth, iteration, exit options), not on how good the first preview looks.
I’m curious about two things from people who’ve shipped with these:
Also, Windsurf has moved to Devin Desktop, so if you’ve tried the new version, I’d like to hear how it compares. I’ll update the post with anything useful from the comments.
Judging the tools on what happens after the first prompt, instead of how good the first preview looks, is the right lens. Most comparisons skip that.
The “Exit strategy” point is the one I’d push hardest. Credit-based pricing and lock-in only really hurt around the tenth revision, when you’re deep in a project and migrating is no longer cheap. A quick export test early on (can I get the code and the data out cleanly?) would save a lot of people pain.
Also useful that you flagged the Windsurf → Devin Desktop change. Outdated tutorials are a real problem in this space.
One question: have you tried the same small task across a builder like Lovable and a terminal agent like Claude Code? I’m curious where each one starts to struggle, especially on auth and permissions, since you note that generated UIs can hide broken access rules.
Thanks, that’s the lens I was going for. Most comparisons stop at the first preview, and the tenth revision is where the real differences show up.
Agreed on the export test. A good early check is to pull the code and data out in the first hour, while leaving is still cheap.
On your question: no, I haven’t run the same task across Lovable and Claude Code. The scorecard in the post is my editorial judgment from documented capabilities, not measured test results, and I flagged that in the post on purpose. I’d rather say that than imply a head-to-head test I didn’t do.
Your auth and permissions question is the right one to test, though. If I ran it, I’d pick a small multi-user app, like notes or tasks where each user should only see their own records. Then I’d check three things:
My expectation, not a finding, is that the builders struggle where the UI looks right but the rules aren’t enforced. With a terminal agent, the weak point is that you have to specify and verify the rules yourself, since it won’t supply a backend. If you’ve seen it go differently in your own tests, I’d like to hear where each one broke.
The line “the best option depends on whether you want AI to build the app for you or help you build and maintain the code yourself” is the most useful sentence in the post. Most people pick a tool by how good the demo looks, then get stuck when they outgrow it.
I’d add that this doesn’t have to be an either/or choice. A workflow I’ve seen work is to prototype in a builder like Lovable or Bolt, sync the project to GitHub, and then continue in Cursor or a terminal agent once the requirements get real. The builder gets you to something testable in a day, and the editor takes over when you need proper control over the code. The “code and project control” column in your scorecard is basically the checklist for whether that handoff will be smooth.
I also liked that the Claude Code vs Gemini CLI section says to try both on the same small task in your own repo instead of declaring a winner. That’s more honest than most comparisons.
A question: for the builders you rated “Medium” on code control, how painful was the export in practice? Did the exported project run locally without much rework, or did it stay tied to the platform’s services? That detail would help people decide before they commit.
Thanks, glad that line landed. And your workflow is a good one: prototype in a builder, sync to GitHub, then continue in an editor or terminal agent once the requirements get real. That’s the handoff the “Code control” column is meant to help people judge.
On your question, I’ll be upfront that the ratings in the scorecard come from the documented capabilities of each tool (GitHub sync, code export, hosting and backend ties), not from a hands-on export test of every project. So I can’t tell you how a given export runs locally. “Medium” was my way of saying the code is yours to take, but the app may lean on platform services such as hosting, auth, and the database. That’s where the rework usually shows up, so I’d treat it as a checklist rather than a guarantee.
If you’ve done the export yourself, I’d like to hear how it went. Did the app run locally with only the env vars and backend connection reconfigured, or did you have to rewire auth and data? Those details would be more useful to readers than my ratings.
One thing that decides whether a tool survives past the demo is how it handles regressions across iterations, and I’d love to see that benchmarked.
With app builders, a broad prompt like “add filtering” can quietly rewrite shared components and break a feature that worked two prompts ago. Editors and terminal agents handle this better because the change shows up as a git diff, so you can review it, run tests and revert. If an app builder doesn’t give you real version control with GitHub sync, treat every prompt as a risky deploy.
The credit-based pricing point is also underrated. Debugging loops burn credits fastest, because the model re-reads context and retries. Writing a precise spec up front (schema, roles, validation rules) usually costs less than iterating your way to it.
For terminal agents, scoping matters more than model choice. A narrow task with explicit acceptance criteria (“make these failing tests pass, don’t touch the public API”) gives far more reliable output than “refactor this module”.
Thanks for flagging the Windsurf to Devin Desktop change. Stale comparisons are a real problem in this space.
The "don't touch the public API" half of your example is the part I'd push on, because "make these failing tests pass" is a check and "don't touch the public API" is a wish, and an agent can't fail a wish. I'd turn that half into a command too, an API snapshot test or a diff over the exported types, so both halves come back with a result… It's a big piece of why cairn works the way it does (the harness in my comment on the post), a task's verify command gets quoted back before the work starts so the bar can't get negotiated after, and TDD tasks commit red and green separately, so when an agent "fixes" a failing test by editing it, like Devers said, you can see it by diffing the test between the two commits.
A useful comparison of coding tools should surface the exit costs as clearly as the first-run speed.