If you give an AI a vague prompt, it will give you a confident answer. And that confidence will cost you a weekend of debugging.
Last month I decided to test how far I could push an AI agent. I didn't want to use it to write a simple function. I wanted it to build an entire backend feature from scratch, end-to-end.
The feature was a standard requirement: A CSV export endpoint for user data.
I opened my AI tool, cracked my knuckles, and typed a prompt that took me exactly 5 seconds to write:
"Create an Express API endpoint that exports all user records from the database into a CSV file and returns it for download."
Thirty seconds later, the AI spat out a perfectly formatted controller, a well-written database query, and even a utility function to convert JSON to CSV. It looked elegant. The variable names were clean. It compiled immediately.
I felt like a 10x developer. I pushed the code, merged the PR, and deployed to production.
And then, within 10 minutes, the support tickets started rolling in.
The Illusion of Perfect AI Code
Here is the thing about AI: It doesn't know what it doesn't know.
Because I didn't give the AI any context about our system, it made three massive assumptions that completely broke production:
1. The All Users Security Nightmare
I wrote "exports all user records." The AI did exactly that. But wait I work on a multi-tenant SaaS. "All users" means every single user across every single company in our database.
An admin from Company A downloaded the CSV and suddenly had the names and emails of Company B. It was a massive data leak because I forgot to tell the AI, "Only export users belonging to the authenticated user's organization."
2. The Memory Explosion (OOM)
The AI wrote a beautiful query: const users = await db.user.findMany(); followed by mapping over the array. It worked perfectly in local testing where I had 50 users.
In production, we had 2.5 million users. The server didn't just crash; it had an Out-Of-Memory (OOM) error because the AI tried to load 2.5 million records into memory simultaneously. I forgot to tell the AI, "Use a streaming approach or pagination."
3. The CSV Injection Vulnerability
The AI did a simple join(',') to create the CSV. But users can enter their own names. If a user named themselves =SUM(1+1) or =cmd|' /C calc'!A0, Excel would actually execute that formula when the admin opened the CSV file. I forgot to tell the AI, "Sanitize the data to prevent CSV injection."
None of these bugs were the AI's fault. The AI did exactly what I asked.
The failure was 100% mine, because I gave the AI the "What" but not the "Where" and the "How."
The Context Debt is Real
This experiment taught me one massive lesson: AI is not a mind reader. It is an incredibly fast, highly obedient intern with amnesia.
When you don't give the AI the full context of your system, its architectural rules, and your security boundaries, it writes code in a vacuum. And code written in a vacuum is just technical debt waiting to explode in production.
We spend so much time arguing about which AI model is smarter GPT-4o, Claude 3.5, Llama 3. But a smarter model wouldn't have saved me here. Even a genius model can't guess that I'm using a multi-tenant database with millions of rows.
How to Fix It: Context-Driven Development
If you want AI to actually save you time (and not get you fired), you have to stop writing Twitter-style prompts and start writing Context-Rich prompts.
Here are the 3 rules I now follow:
1. Share the Database Schema
Don't just say "user data." Paste your Prisma schema, your TypeORM entities, or your SQL table definitions into the prompt. AI is amazing at spotting relations when it can actually see the tenant_id foreign key.
2. State the Constraints Explicitly
If your system has rules, write them down.
"Assume the table has 2 million rows. Use a streaming approach."
"This is a multi-tenant app. Always filter by organization ID."
3. Define the Edge Cases Before You Prompt
Tell the AI to be paranoid.
"Assume users can input malicious strings. Sanitize all CSV outputs."
"Add rate limiting to this endpoint."
The Ultimate Question: Prompts vs. Models?
We are living in a golden era of AI models. They are getting exponentially smarter every six months. But my production incident proved that a brilliant model with terrible context is still a disaster.
So, I want to ask you:
Is better prompting more important than better AI models?
Will we eventually reach a point where AI knows our codebases so well (via RAG and local indexing) that we don't need to prompt perfectly? Or is "Context Engineering" the real skill of the future?
Drop your thoughts in the comments below. I'd love to know how you handle giving AI the right context!
Top comments (40)
I think context is just as important as the model itself. The multi-tenant security issue is a particularly serious example because clean, working code can still violate fundamental business rules. Sharing the schema, defining constraints, and reviewing edge cases before deployment should be part of every AI-assisted development workflow. Better models help, but they can't replace system knowledge.
Bhavin clean, working code can still violate fundamental business rules is the perfect way to describe the trap of AI-generated code. It looks so aesthetically pleasing in a PR that our brains just automatically approve it.
You are 100% right: context is just as important as the model itself. A smarter model will still confidently walk off a cliff if it doesn't know where the edge is System knowledge is something no LLM can magically guess it has to be provided.
Sharing the schema and defining constraints upfront isn't just a nice to have anymore it's becoming a mandatory part of the engineering workflow. Thanks for the great summary!
Better context helps, but it still depends on you remembering every rule on the day you write the prompt, and you said yourself the gap is the thing you forget to mention. The fix that doesn't depend on memory is a test that encodes the rule once: put two tenants in the fixtures and assert an export never contains the other tenant's rows; add a user named =SUM(1+1) and assert the cell comes out escaped. Then any code, AI or human, that breaks the rule fails before merge. The step that actually let all three through was push, merge, deploy with nothing in between.
Gautam this is the most practical take in this entire thread. You are 100% right: Context Engineering is just a band-aid if it relies on human memory. Tests are the only real cure. the two-tenant fixture and the =SUM(1+1) CSV injection test are exactly what should have been in the codebase. If I had those tests, the AI's code would have failed in CI and my weekend wouldn't have been ruined. It completely shifts the responsibility from writing a perfect prompt to building a safety net.
And that last line hits hard: push, merge deploy with nothing in between. I completely skipped the automated safety gates. That's the real lesson here. Thank you for this I'm updating our test fixtures today!
Glad it helped, and sorry about the weekend. If you add one gate first, make it the two-tenant test in CI before merge. It's the one that catches the bug no prompt will ever remember.
Haha the weekend was a necessary tuition fee! I'm putting the two-tenant test as our first mandatory CI gate today. You're right it's the only safety net that doesn't rely on human memory. Thanks again for the great insight, Gautam!
Your first bug worries me most, because a better prompt only fixes it if someone knew to write the rule down. At work we moved that step out of the prompt: every feature gets a 5-10 minute discussion that updates the product document before anyone builds, and QA writes its cases from that document, not from the code. Tenant scoping would land in the doc. Streaming and CSV injection might not. Where do rules like those live for you: per endpoint, or in one file the agent always reads?
Panth this is a fantastic workflow. A 5-10 minute alignment before building is exactly what prevents the AI assumption problem. And having QA write tests from the doc (not the code) is the best way to kill the echo chamber. You hit a critical point about business logic making it to the doc while technical vulnerabilities (like CSV injection) get missed. To answer your question: for us it's a mix of both.
High-level business rules (tenant scoping, feature limits) live in the ticket/acceptance criteria. But technical guardrails live in a centralized AI_INSTRUCTIONS.md (or .cursorrules) file at the root of the repo. We put rules like Always sanitize CSV exports or Never use findMany() without pagination in there so the agent auto-reads them on every prompt. How does your team handle the technical edge cases that don't naturally fit into the product doc? Do the senior devs add those manually to the tickets, or do you rely on QA to catch them?
Our QA reads only the product doc and tests as a client would, so technical edge cases sit with the dev, who does their own QA first, and the dev lead, who checks a short video on the feature's interfaces and contract. For agents, curated repo skills do your AI_INSTRUCTIONS.md job. Does anything check the agent followed that file?
Panth this is a brilliantly layered defense. Dev QA + Dev lead video check + QA acting as a client is exactly how you catch both UX and technical edge cases. To answer your question honestly no. The uncomfortable truth about AI is that it's probabilistic not deterministic. We can put a rule in AI_INSTRUCTIONS.md and the agent will follow it 95% of the time. But that 5% hallucination rate is exactly what causes production incidents.
So the only real check is our CI/CD pipeline. We write custom ESLint rules to block bad patterns (like findMany() without a limit), and our automated test suite (like the two-tenant fixture) acts as the final gate. If the agent breaks a rule the CI fails.
We also do a quick manual diff review specifically looking for rule violations before merging. It's not automated, but it works. Have you found any specific agent framework that actually enforces these rules natively, or do you rely on CI just like we do?
the multi-tenant one is scary because the code looks completely fine. it compiles, tests pass, and you only find out a customer from another company got someone else's data. do you think agents should ask clarifying questions before touching anything with auth or data scoping, or is that on us to spell out every time?
Ivan exactly. That's the scariest part it passes all the happy path tests because the test data only has one tenant. It looks completely innocent until an admin downloads someone else's data.
To answer your question: I think ideally AI agents should absolutely ask clarifying questions before touching anything related to auth or data scoping. If an agent sees a User model, it should ask, 'Is this multi-tenant?' before writing the query.
But in reality, right now, the burden is completely on us. Because if I forget that my system is multi-tenant, I'm not going to know to tell the AI. And if the AI doesn't know my architecture, it won't know what to ask.
Until we have agents that can ingest our entire architectural context and proactively flag security boundaries, we have to spell it out every single time. It's exhausting, but it's the only way to survive right now.
Do you think agents asking 10 clarifying questions upfront would actually be helpful, or would it just slow us down?
the multi tenant leak is the scariest of your three because it passes every test that runs against one org. we hit the same shape with an AI generated report endpoint: the query was right, the tenant filter just never existed because nothing in the prompt mentioned it.
what helped was putting tenant scope inside a repository wrapper the model has to call, so forgetting it becomes a type error and not a prompt problem.
did you add any guardrail after this, or is it still on the reviewer to catch missing org filters?
Mudassir turning a prompt problem into a compile-time type error is the most brilliant architectural fix in this entire thread. You basically forced the compiler to be the code reviewer. I'm definitely adopting that repository wrapper pattern.
To answer your question: right after the incident, we did two things. First, we added a strict two-tenant test fixture (shoutout to Gautam in this thread) that fails if cross-tenant data leaks. Second we added some custom ESLint rules and CI checks to flag raw findMany() calls without a limit.
But honestly relying on the reviewer to catch missing org filters is a ticking time bomb. It works for now but your wrapper idea is the real permanent guardrail.
Quick question: how do you handle edge cases with the wrapper, like when an admin needs to query across all tenants? Do you have a separate AdminRepository, or do you require a specific flag to bypass the wrapper?
separate admin path, yes. the normal repository takes the tenant id in its constructor and has no way to drop it, and the cross tenant reader is its own class with an explicit audit log line per call. that way a bypass is a visible import in review, not a boolean someone forgets to flip. the eslint rule on raw findMany is a good catch too. do your admin queries need to be fast, or is a slower reporting replica fine?
Mudassir this is elite-level multi-tenant architecture. The tenantId in constructor pattern combined with an explicit audit log for the cross-tenant reader is bulletproof. It forces the AI into the pit of success it literally cannot write a bad query for the normal path because the compiler won't let it.
To answer your question: a slower reporting replica is totally fine for our admin queries. Admin dashboards usually involve wide scans or heavy analytics, so we route those to a read replica. We don't need millisecond latency for an internal admin panel; we just need the primary transactional DB to survive.
Plus, if the admin query is hitting a replica, the replication lag naturally acts as a slight buffer against accidental data corruption.
Thanks for the architecture masterclass. I'm definitely refactoring our admin path to match this pattern!
The multi-tenant leak is the scary one because the code was correct for the sentence you wrote. "All users" meant every user in the database, and nothing in the prompt said otherwise.
Gautam's reply is the fix I would push hardest. Better prompts still depend on remembering every rule on the day you write them. A fixture with two tenants and an assertion that one never sees the other encodes the rule once, and it keeps working when the next person, or the next model, touches the endpoint.
The truncated marker row story is a nice detail too. People open the file and start working, so a warning on row 1,000 is not a warning. It probably belongs in the filename or the UI before the download starts.
At Ascendra Ventures we write tenant scope, row limits and export format into the acceptance criteria before any code is generated, because those are the decisions an assistant will otherwise make for you. Did the team end up moving to streamed exports, or keep the hard limit?
Ascendra writing the constraints into the acceptance criteria before generating code is a masterclass in engineering. That is exactly where we went wrong we treated the AI like an autocomplete tool instead of a junior dev who needs a proper spec.
You're also right about the multi-tenant leak. The AI isn't malicious; it just reads instructions literally. All users meant all users, and it did exactly what I asked. Gautam's point about the two-tenant fixture is definitely the ultimate safety net for that.
To answer your question: we actually stuck with the hard limit. We realized that building a robust streaming pipeline (that can handle mid-stream failures gracefully) was overkill for a simple CSV export. We kept the hard LIMIT, moved the warning to the UI, and changed the filename. Pragmatic over perfect!
Since you write these constraints into acceptance criteria upfront, do you have a specific checklist or template you use with the AI? I'd love to know how you structure that.
One surface is missing from this list, and it's the one that leaves your visibility entirely: the exported artifact itself.
Your prompt rules and the dual-tenant fixtures both guard inside the boundary. But a CSV cell that starts with = becomes a formula the moment a customer opens the file in Excel - on their machine, in their mail client, weeks after your logs rotated. Once the file is forwarded or renamed you have zero telemetry on it.
Worth one outbound-encoding test next to the fixture work: neutralize the leading = on the way out and assert on the generated bytes, not the prompt - the rule belongs in the serializer, where CI can actually check it. Generation time or download time? Generation feels safer, since the raw value can land in the LLM's own logs either way.
Nan this is a brilliant and slightly terrifying addition. You're absolutely right about the outbound boundary. We spend so much time securing the database but once that CSV hits a user's laptop and opens in Excel it's a completely unmonitored threat surface.
The outbound-encoding test in the serializer is a masterstroke. Testing the actual generated bytes is deterministic; testing the prompt is just wishful thinking.
But the point you made about the raw values landing in the LLM logs is the one that's going to keep me up at night now. Since the data leaves our DB and goes to a third-party model provider, how are you handling that? Are you masking PII before sending it to the LLM or do you have a specific redaction layer in your serializer for that too?
The OOM one is the most underrated of the three — a findMany() that passes locally is exactly the failure mode that survives code review too, because a reviewer sees a clean query, not the row count. We hit the same with a report export: the fix that stuck was a hard LIMIT in the query itself (say 100k rows + a "truncated" marker row at the end) rather than just streaming, since a streaming pipe can still die halfway and leave the client with a broken file.
Curious about the multi-tenant leak part: did you catch it in your own QA or purely from the Company A admin ticket? After a similar incident I started pasting the auth middleware alongside the schema into the prompt, since the tenant filter lives there, not in the Prisma model — the AI never saw the org scoping otherwise.
Max you are spot on about the OOM bug. findMany() looks incredibly clean in a code review. The reviewer sees a beautifully formatted query and completely ignores the fact that it's missing a .take(1000). It's a silent killer.
And I absolutely love your fix: a hard LIMIT plus a 'truncated' marker in the CSV. You're right, streaming can sometimes leave the client with a half-broken file, which is a terrible user experience. A hard limit is much safer.
To answer your question: honestly, it was purely from the Company A ticket. My QA passed with flying colors because I only had one tenant in my local database. I completely missed it.
Your tip about pasting the auth middleware into the prompt alongside the schema is a genius move. I was only pasting the Prisma schema, and like you said, the tenantId was there but the actual filtering logic lived in the middleware. The AI never saw it. I'm definitely adding that to my workflow. Thanks for sharing!
The multi-tenant one is scary because the code was "correct", it just answered the wrong question. I've started writing the constraints before the feature: who's allowed to see this, how big the data gets, what a hostile input looks like. Two minutes of typing that saves a weekend. Did you add tests for the tenant scoping after this?
Nilamadhab correct code answering the wrong question is the perfect way to describe this entire disaster. AI is incredibly obedient to the literal prompt, even if the prompt is missing the actual business logic.
And two minutes of typing that saves a weekend is a rule I wish I had framed on my desk. Writing the constraints (who can see it, how big the data is) before writing the feature is exactly the shift we need.
To answer your question: Yes absolutely. We added a strict two-tenant test fixture that fails the build if cross-tenant data leaks. It's now impossible to merge a query without the proper org_id filter. It was a hard lesson, but it's permanently fixed in CI now.
Since you've started writing constraints beforehand, do you have a specific template or checklist you use for those two minutes? I'd love to steal a good process.
The hard LIMIT plus a truncated marker row is the fix I'd actually steal — streaming sounds cleaner until the pipe dies halfway and hands someone a broken file. Did the truncated row ever cause a support ticket of its own, people assuming the export was complete?
Paul you read my mind. Yes, the truncated row absolutely caused its own support tickets! 😂
People download the CSV open it in Excel, and immediately start working. They never scroll to row 1,000 to read the tiny warning at the bottom. We quickly realized that putting a warning inside the file is useless if users don't read it. We ended up doing two things: moving the warning to the UI before they even click download, and changing the actual filename to export_truncated_1000.csv. That was the only way to make it impossible to ignore.
And you're right about streaming it sounds elegant until the pipe dies and the user gets a half-corrupted file. A hard limit plus a loud UI warning is the safest path. Shoutout to Max in this thread for the original idea!