I think context is just as important as the model itself. The multi-tenant security issue is a particularly serious example because clean, working code can still violate fundamental business rules. Sharing the schema, defining constraints, and reviewing edge cases before deployment should be part of every AI-assisted development workflow. Better models help, but they can't replace system knowledge.
Bhavin clean, working code can still violate fundamental business rules is the perfect way to describe the trap of AI-generated code. It looks so aesthetically pleasing in a PR that our brains just automatically approve it.
You are 100% right: context is just as important as the model itself. A smarter model will still confidently walk off a cliff if it doesn't know where the edge is System knowledge is something no LLM can magically guess it has to be provided.
Sharing the schema and defining constraints upfront isn't just a nice to have anymore it's becoming a mandatory part of the engineering workflow. Thanks for the great summary!
Better context helps, but it still depends on you remembering every rule on the day you write the prompt, and you said yourself the gap is the thing you forget to mention. The fix that doesn't depend on memory is a test that encodes the rule once: put two tenants in the fixtures and assert an export never contains the other tenant's rows; add a user named =SUM(1+1) and assert the cell comes out escaped. Then any code, AI or human, that breaks the rule fails before merge. The step that actually let all three through was push, merge, deploy with nothing in between.
Gautam this is the most practical take in this entire thread. You are 100% right: Context Engineering is just a band-aid if it relies on human memory. Tests are the only real cure. the two-tenant fixture and the =SUM(1+1) CSV injection test are exactly what should have been in the codebase. If I had those tests, the AI's code would have failed in CI and my weekend wouldn't have been ruined. It completely shifts the responsibility from writing a perfect prompt to building a safety net.
And that last line hits hard: push, merge deploy with nothing in between. I completely skipped the automated safety gates. That's the real lesson here. Thank you for this I'm updating our test fixtures today!
Glad it helped, and sorry about the weekend. If you add one gate first, make it the two-tenant test in CI before merge. It's the one that catches the bug no prompt will ever remember.
Haha the weekend was a necessary tuition fee! I'm putting the two-tenant test as our first mandatory CI gate today. You're right it's the only safety net that doesn't rely on human memory. Thanks again for the great insight, Gautam!
Your first bug worries me most, because a better prompt only fixes it if someone knew to write the rule down. At work we moved that step out of the prompt: every feature gets a 5-10 minute discussion that updates the product document before anyone builds, and QA writes its cases from that document, not from the code. Tenant scoping would land in the doc. Streaming and CSV injection might not. Where do rules like those live for you: per endpoint, or in one file the agent always reads?
Panth this is a fantastic workflow. A 5-10 minute alignment before building is exactly what prevents the AI assumption problem. And having QA write tests from the doc (not the code) is the best way to kill the echo chamber. You hit a critical point about business logic making it to the doc while technical vulnerabilities (like CSV injection) get missed. To answer your question: for us it's a mix of both.
High-level business rules (tenant scoping, feature limits) live in the ticket/acceptance criteria. But technical guardrails live in a centralized AI_INSTRUCTIONS.md (or .cursorrules) file at the root of the repo. We put rules like Always sanitize CSV exports or Never use findMany() without pagination in there so the agent auto-reads them on every prompt. How does your team handle the technical edge cases that don't naturally fit into the product doc? Do the senior devs add those manually to the tickets, or do you rely on QA to catch them?
Our QA reads only the product doc and tests as a client would, so technical edge cases sit with the dev, who does their own QA first, and the dev lead, who checks a short video on the feature's interfaces and contract. For agents, curated repo skills do your AI_INSTRUCTIONS.md job. Does anything check the agent followed that file?
Panth this is a brilliantly layered defense. Dev QA + Dev lead video check + QA acting as a client is exactly how you catch both UX and technical edge cases. To answer your question honestly no. The uncomfortable truth about AI is that it's probabilistic not deterministic. We can put a rule in AI_INSTRUCTIONS.md and the agent will follow it 95% of the time. But that 5% hallucination rate is exactly what causes production incidents.
So the only real check is our CI/CD pipeline. We write custom ESLint rules to block bad patterns (like findMany() without a limit), and our automated test suite (like the two-tenant fixture) acts as the final gate. If the agent breaks a rule the CI fails.
We also do a quick manual diff review specifically looking for rule violations before merging. It's not automated, but it works. Have you found any specific agent framework that actually enforces these rules natively, or do you rely on CI just like we do?
the multi-tenant one is scary because the code looks completely fine. it compiles, tests pass, and you only find out a customer from another company got someone else's data. do you think agents should ask clarifying questions before touching anything with auth or data scoping, or is that on us to spell out every time?
Ivan exactly. That's the scariest part it passes all the happy path tests because the test data only has one tenant. It looks completely innocent until an admin downloads someone else's data.
To answer your question: I think ideally AI agents should absolutely ask clarifying questions before touching anything related to auth or data scoping. If an agent sees a User model, it should ask, 'Is this multi-tenant?' before writing the query.
But in reality, right now, the burden is completely on us. Because if I forget that my system is multi-tenant, I'm not going to know to tell the AI. And if the AI doesn't know my architecture, it won't know what to ask.
Until we have agents that can ingest our entire architectural context and proactively flag security boundaries, we have to spell it out every single time. It's exhausting, but it's the only way to survive right now.
Do you think agents asking 10 clarifying questions upfront would actually be helpful, or would it just slow us down?
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
the multi tenant leak is the scariest of your three because it passes every test that runs against one org. we hit the same shape with an AI generated report endpoint: the query was right, the tenant filter just never existed because nothing in the prompt mentioned it.
what helped was putting tenant scope inside a repository wrapper the model has to call, so forgetting it becomes a type error and not a prompt problem.
did you add any guardrail after this, or is it still on the reviewer to catch missing org filters?
Mudassir turning a prompt problem into a compile-time type error is the most brilliant architectural fix in this entire thread. You basically forced the compiler to be the code reviewer. I'm definitely adopting that repository wrapper pattern.
To answer your question: right after the incident, we did two things. First, we added a strict two-tenant test fixture (shoutout to Gautam in this thread) that fails if cross-tenant data leaks. Second we added some custom ESLint rules and CI checks to flag raw findMany() calls without a limit.
But honestly relying on the reviewer to catch missing org filters is a ticking time bomb. It works for now but your wrapper idea is the real permanent guardrail.
Quick question: how do you handle edge cases with the wrapper, like when an admin needs to query across all tenants? Do you have a separate AdminRepository, or do you require a specific flag to bypass the wrapper?
I write about Next.js + TypeScript + AI engineering, with a focus on making sites discoverable to LLMs and AI search. Long-form notes at mudassirkhan.me
separate admin path, yes. the normal repository takes the tenant id in its constructor and has no way to drop it, and the cross tenant reader is its own class with an explicit audit log line per call. that way a bypass is a visible import in review, not a boolean someone forgets to flip. the eslint rule on raw findMany is a good catch too. do your admin queries need to be fast, or is a slower reporting replica fine?
Mudassir this is elite-level multi-tenant architecture. The tenantId in constructor pattern combined with an explicit audit log for the cross-tenant reader is bulletproof. It forces the AI into the pit of success it literally cannot write a bad query for the normal path because the compiler won't let it.
To answer your question: a slower reporting replica is totally fine for our admin queries. Admin dashboards usually involve wide scans or heavy analytics, so we route those to a read replica. We don't need millisecond latency for an internal admin panel; we just need the primary transactional DB to survive.
Plus, if the admin query is hitting a replica, the replication lag naturally acts as a slight buffer against accidental data corruption.
Thanks for the architecture masterclass. I'm definitely refactoring our admin path to match this pattern!
The multi-tenant leak is the scary one because the code was correct for the sentence you wrote. "All users" meant every user in the database, and nothing in the prompt said otherwise.
Gautam's reply is the fix I would push hardest. Better prompts still depend on remembering every rule on the day you write them. A fixture with two tenants and an assertion that one never sees the other encodes the rule once, and it keeps working when the next person, or the next model, touches the endpoint.
The truncated marker row story is a nice detail too. People open the file and start working, so a warning on row 1,000 is not a warning. It probably belongs in the filename or the UI before the download starts.
At Ascendra Ventures we write tenant scope, row limits and export format into the acceptance criteria before any code is generated, because those are the decisions an assistant will otherwise make for you. Did the team end up moving to streamed exports, or keep the hard limit?
Ascendra writing the constraints into the acceptance criteria before generating code is a masterclass in engineering. That is exactly where we went wrong we treated the AI like an autocomplete tool instead of a junior dev who needs a proper spec.
You're also right about the multi-tenant leak. The AI isn't malicious; it just reads instructions literally. All users meant all users, and it did exactly what I asked. Gautam's point about the two-tenant fixture is definitely the ultimate safety net for that.
To answer your question: we actually stuck with the hard limit. We realized that building a robust streaming pipeline (that can handle mid-stream failures gracefully) was overkill for a simple CSV export. We kept the hard LIMIT, moved the warning to the UI, and changed the filename. Pragmatic over perfect!
Since you write these constraints into acceptance criteria upfront, do you have a specific checklist or template you use with the AI? I'd love to know how you structure that.
One surface is missing from this list, and it's the one that leaves your visibility entirely: the exported artifact itself.
Your prompt rules and the dual-tenant fixtures both guard inside the boundary. But a CSV cell that starts with = becomes a formula the moment a customer opens the file in Excel - on their machine, in their mail client, weeks after your logs rotated. Once the file is forwarded or renamed you have zero telemetry on it.
Worth one outbound-encoding test next to the fixture work: neutralize the leading = on the way out and assert on the generated bytes, not the prompt - the rule belongs in the serializer, where CI can actually check it. Generation time or download time? Generation feels safer, since the raw value can land in the LLM's own logs either way.
Nan this is a brilliant and slightly terrifying addition. You're absolutely right about the outbound boundary. We spend so much time securing the database but once that CSV hits a user's laptop and opens in Excel it's a completely unmonitored threat surface.
The outbound-encoding test in the serializer is a masterstroke. Testing the actual generated bytes is deterministic; testing the prompt is just wishful thinking.
But the point you made about the raw values landing in the LLM logs is the one that's going to keep me up at night now. Since the data leaves our DB and goes to a third-party model provider, how are you handling that? Are you masking PII before sending it to the LLM or do you have a specific redaction layer in your serializer for that too?
The OOM one is the most underrated of the three — a findMany() that passes locally is exactly the failure mode that survives code review too, because a reviewer sees a clean query, not the row count. We hit the same with a report export: the fix that stuck was a hard LIMIT in the query itself (say 100k rows + a "truncated" marker row at the end) rather than just streaming, since a streaming pipe can still die halfway and leave the client with a broken file.
Curious about the multi-tenant leak part: did you catch it in your own QA or purely from the Company A admin ticket? After a similar incident I started pasting the auth middleware alongside the schema into the prompt, since the tenant filter lives there, not in the Prisma model — the AI never saw the org scoping otherwise.
Max you are spot on about the OOM bug. findMany() looks incredibly clean in a code review. The reviewer sees a beautifully formatted query and completely ignores the fact that it's missing a .take(1000). It's a silent killer.
And I absolutely love your fix: a hard LIMIT plus a 'truncated' marker in the CSV. You're right, streaming can sometimes leave the client with a half-broken file, which is a terrible user experience. A hard limit is much safer.
To answer your question: honestly, it was purely from the Company A ticket. My QA passed with flying colors because I only had one tenant in my local database. I completely missed it.
Your tip about pasting the auth middleware into the prompt alongside the schema is a genius move. I was only pasting the Prisma schema, and like you said, the tenantId was there but the actual filtering logic lived in the middleware. The AI never saw it. I'm definitely adding that to my workflow. Thanks for sharing!
The multi-tenant one is scary because the code was "correct", it just answered the wrong question. I've started writing the constraints before the feature: who's allowed to see this, how big the data gets, what a hostile input looks like. Two minutes of typing that saves a weekend. Did you add tests for the tenant scoping after this?
Nilamadhab correct code answering the wrong question is the perfect way to describe this entire disaster. AI is incredibly obedient to the literal prompt, even if the prompt is missing the actual business logic.
And two minutes of typing that saves a weekend is a rule I wish I had framed on my desk. Writing the constraints (who can see it, how big the data is) before writing the feature is exactly the shift we need.
To answer your question: Yes absolutely. We added a strict two-tenant test fixture that fails the build if cross-tenant data leaks. It's now impossible to merge a query without the proper org_id filter. It was a hard lesson, but it's permanently fixed in CI now.
Since you've started writing constraints beforehand, do you have a specific template or checklist you use for those two minutes? I'd love to steal a good process.
I research AI tools, APIs & agents — comparing specs, pricing and real user reports — then publish honest rankings so you don't waste money. Research-first, never sponsored fluff.
The hard LIMIT plus a truncated marker row is the fix I'd actually steal — streaming sounds cleaner until the pipe dies halfway and hands someone a broken file. Did the truncated row ever cause a support ticket of its own, people assuming the export was complete?
Paul you read my mind. Yes, the truncated row absolutely caused its own support tickets! 😂
People download the CSV open it in Excel, and immediately start working. They never scroll to row 1,000 to read the tiny warning at the bottom. We quickly realized that putting a warning inside the file is useless if users don't read it. We ended up doing two things: moving the warning to the UI before they even click download, and changing the actual filename to export_truncated_1000.csv. That was the only way to make it impossible to ignore.
And you're right about streaming it sounds elegant until the pipe dies and the user gets a half-corrupted file. A hard limit plus a loud UI warning is the safest path. Shoutout to Max in this thread for the original idea!
happened to me last week. ai generated a whole settings page, looked perfect, and quietly dropped the rate limit field because i never mentioned it in the prompt. the feature 'worked' in the demo and ate our api quota in prod lol. now i list the edge cases first, not the feature
Haha eating up the API quota in production is a rite of passage for every developer using AI 😂 It's wild how the demo looks perfect because the happy path works but the AI completely ignores rate limits unless you explicitly spell them out.
Now I list the edge cases first not the feature is a brilliant rule. I'm definitely stealing that. If we tell the AI how the system should fail it actually writes much safer code. Thanks for sharing the war story!
This is easily one of the most honest and practical articles on AI-assisted development I’ve read recently. The multi-tenant data leak scenario you mentioned is genuinely terrifying because on paper, the code looks absolutely flawless and passes all standard unit tests.
To answer your question: Right now, 'Context Engineering' is absolutely the real skill. Even a GPT-5 or Claude 4 cannot guess business logic or hidden multi-tenant architectures unless we feed it that context. However, I believe the future isn't just about us writing better prompts, but rather tools evolving to map our repositories automatically (like advanced codebase indexing/RAG). Until then, your 3 rules should honestly be pinned on every developer's desk who uses AI. Great write-up Harsh.
yash thank you so much! That means a lot. The multi-tenant leak is exactly the kind of bug that keeps me up at night because it's completely invisible to unit tests. You are 100% right about the future being RAG and automatic codebase indexing. We are currently in this awkward middle phase where the models are incredibly smart, but they are completely blind to our architectural reality. We have to manually feed them the context that a proper indexing tool should already know.
Once tools can automatically map the entire repo and understand the relationships between tables, Context Engineering will become much easier. Until then, we are stuck playing the role of the AI's memory!
Glad you liked the 3 rules. Hopefully, we won't need them forever!
Better context helps, but tenant isolation, streaming limits, and CSV sanitization should also be enforced by tests and architecture. Prompts should inform the code, not become the security boundary.
Paul Prompts should inform the code, not become the security boundary' is the most important sentence in this entire thread. You've perfectly summarized the exact lesson I learned the hard way. Relying on a prompt for tenant isolation is like relying on a polite request to stop a hacker. It's just not a real defense. The AI can forget the prompt, ignore the prompt or hallucinate a completely different approach.
As you said, architecture and tests are the only true boundaries. The prompt just helps the AI get it right on the first try but the CI pipeline and the two-tenant test fixture are what actually keep the data safe.
This is definitely going straight into my team's engineering guidelines. Thanks for the brilliant insight!
Exactly. The two-tenant test fixture is what turns that principle into an enforceable guarantee. Prompts improve the first attempt, while CI prevents the dangerous one from shipping.
Paul Prompts improve the first attempt while CI prevents the dangerous one from shipping is the perfect TL;DR for this entire article. I'm definitely quoting that line in my next team meeting.
You've completely shifted my perspective from prompt engineering to systems engineering. The prompt is just a helpful guide; the CI is the actual safety net.
The 'compiles immediately, breaks in production' pattern is exactly what we see on the Electron side: an AI-generated IPC handler looks clean, but it silently assumes the sender window exists, that the payload is already validated, and that the dialog result fits a single await. In desktop the cost is worse than a support ticket — a main-process dialog blocking the event loop freezes every window. The fix is the same as you landed on: force the AI to state its assumptions back before writing code, and add the failure path as a first-class requirement rather than an afterthought.
mp3sss, this is a brilliant addition. The Electron angle takes this bug from annoying to catastrophic. A blocked main process freezing the entire app is a nightmare, and you're right the cost is way worse than just a support ticket. AI is so used to writing happy path code that it completely forgets about process boundaries. It just assumes the sender window exists and the payload is perfectly sanitized.
Your phrase add the failure path as a first-class requirement is the exact mindset shift we need. If we don't explicitly prompt for what happens when things go wrong, the AI will just pretend they never will. For your IPC handlers, do you use a specific TypeScript validation library (like Zod) on the receiving end to enforce those assumptions at runtime, or do you just rely on the prompt to generate the checks?
Pang too obvious to ever write down is the perfect description of the curse of knowledge. We assume everyone including the AI knows that a user must belong to an organization, so we don't bother stating it.
But the AI has zero context. It doesn't know what's obvious in our system. Those unspoken, invisible constraints are exactly where the scariest production bugs hide.
Such a simple but profound observation. Thanks for sharing it!
the annoying flip side is once you do paste the full context, every turn resends it and the bill climbs. I wrote up how that works here if it helps anyone: warpnav.com/ai-token-explained
Haha this is the exact trade-off I'm wrestling with right now. You're 100% right dumping the entire schema and middleware into every prompt is a fast track to a massive API bill. Context isn't free, and token limits are real.
What has helped me recently is being selective. Instead of pasting the whole repo I just grab the specific database model the relevant middleware and the acceptance criteria. It's a bit of manual work, but it saves a ton on tokens.
I'll definitely give your article a read token economics is something every dev using AI needs to understand right now. Thanks for sharing!
We're a place where coders share, stay up-to-date and grow their careers.
I think context is just as important as the model itself. The multi-tenant security issue is a particularly serious example because clean, working code can still violate fundamental business rules. Sharing the schema, defining constraints, and reviewing edge cases before deployment should be part of every AI-assisted development workflow. Better models help, but they can't replace system knowledge.
Bhavin clean, working code can still violate fundamental business rules is the perfect way to describe the trap of AI-generated code. It looks so aesthetically pleasing in a PR that our brains just automatically approve it.
You are 100% right: context is just as important as the model itself. A smarter model will still confidently walk off a cliff if it doesn't know where the edge is System knowledge is something no LLM can magically guess it has to be provided.
Sharing the schema and defining constraints upfront isn't just a nice to have anymore it's becoming a mandatory part of the engineering workflow. Thanks for the great summary!
Better context helps, but it still depends on you remembering every rule on the day you write the prompt, and you said yourself the gap is the thing you forget to mention. The fix that doesn't depend on memory is a test that encodes the rule once: put two tenants in the fixtures and assert an export never contains the other tenant's rows; add a user named =SUM(1+1) and assert the cell comes out escaped. Then any code, AI or human, that breaks the rule fails before merge. The step that actually let all three through was push, merge, deploy with nothing in between.
Gautam this is the most practical take in this entire thread. You are 100% right: Context Engineering is just a band-aid if it relies on human memory. Tests are the only real cure. the two-tenant fixture and the =SUM(1+1) CSV injection test are exactly what should have been in the codebase. If I had those tests, the AI's code would have failed in CI and my weekend wouldn't have been ruined. It completely shifts the responsibility from writing a perfect prompt to building a safety net.
And that last line hits hard: push, merge deploy with nothing in between. I completely skipped the automated safety gates. That's the real lesson here. Thank you for this I'm updating our test fixtures today!
Glad it helped, and sorry about the weekend. If you add one gate first, make it the two-tenant test in CI before merge. It's the one that catches the bug no prompt will ever remember.
Haha the weekend was a necessary tuition fee! I'm putting the two-tenant test as our first mandatory CI gate today. You're right it's the only safety net that doesn't rely on human memory. Thanks again for the great insight, Gautam!
Your first bug worries me most, because a better prompt only fixes it if someone knew to write the rule down. At work we moved that step out of the prompt: every feature gets a 5-10 minute discussion that updates the product document before anyone builds, and QA writes its cases from that document, not from the code. Tenant scoping would land in the doc. Streaming and CSV injection might not. Where do rules like those live for you: per endpoint, or in one file the agent always reads?
Panth this is a fantastic workflow. A 5-10 minute alignment before building is exactly what prevents the AI assumption problem. And having QA write tests from the doc (not the code) is the best way to kill the echo chamber. You hit a critical point about business logic making it to the doc while technical vulnerabilities (like CSV injection) get missed. To answer your question: for us it's a mix of both.
High-level business rules (tenant scoping, feature limits) live in the ticket/acceptance criteria. But technical guardrails live in a centralized AI_INSTRUCTIONS.md (or .cursorrules) file at the root of the repo. We put rules like Always sanitize CSV exports or Never use findMany() without pagination in there so the agent auto-reads them on every prompt. How does your team handle the technical edge cases that don't naturally fit into the product doc? Do the senior devs add those manually to the tickets, or do you rely on QA to catch them?
Our QA reads only the product doc and tests as a client would, so technical edge cases sit with the dev, who does their own QA first, and the dev lead, who checks a short video on the feature's interfaces and contract. For agents, curated repo skills do your AI_INSTRUCTIONS.md job. Does anything check the agent followed that file?
Panth this is a brilliantly layered defense. Dev QA + Dev lead video check + QA acting as a client is exactly how you catch both UX and technical edge cases. To answer your question honestly no. The uncomfortable truth about AI is that it's probabilistic not deterministic. We can put a rule in AI_INSTRUCTIONS.md and the agent will follow it 95% of the time. But that 5% hallucination rate is exactly what causes production incidents.
So the only real check is our CI/CD pipeline. We write custom ESLint rules to block bad patterns (like findMany() without a limit), and our automated test suite (like the two-tenant fixture) acts as the final gate. If the agent breaks a rule the CI fails.
We also do a quick manual diff review specifically looking for rule violations before merging. It's not automated, but it works. Have you found any specific agent framework that actually enforces these rules natively, or do you rely on CI just like we do?
the multi-tenant one is scary because the code looks completely fine. it compiles, tests pass, and you only find out a customer from another company got someone else's data. do you think agents should ask clarifying questions before touching anything with auth or data scoping, or is that on us to spell out every time?
Ivan exactly. That's the scariest part it passes all the happy path tests because the test data only has one tenant. It looks completely innocent until an admin downloads someone else's data.
To answer your question: I think ideally AI agents should absolutely ask clarifying questions before touching anything related to auth or data scoping. If an agent sees a User model, it should ask, 'Is this multi-tenant?' before writing the query.
But in reality, right now, the burden is completely on us. Because if I forget that my system is multi-tenant, I'm not going to know to tell the AI. And if the AI doesn't know my architecture, it won't know what to ask.
Until we have agents that can ingest our entire architectural context and proactively flag security boundaries, we have to spell it out every single time. It's exhausting, but it's the only way to survive right now.
Do you think agents asking 10 clarifying questions upfront would actually be helpful, or would it just slow us down?
the multi tenant leak is the scariest of your three because it passes every test that runs against one org. we hit the same shape with an AI generated report endpoint: the query was right, the tenant filter just never existed because nothing in the prompt mentioned it.
what helped was putting tenant scope inside a repository wrapper the model has to call, so forgetting it becomes a type error and not a prompt problem.
did you add any guardrail after this, or is it still on the reviewer to catch missing org filters?
Mudassir turning a prompt problem into a compile-time type error is the most brilliant architectural fix in this entire thread. You basically forced the compiler to be the code reviewer. I'm definitely adopting that repository wrapper pattern.
To answer your question: right after the incident, we did two things. First, we added a strict two-tenant test fixture (shoutout to Gautam in this thread) that fails if cross-tenant data leaks. Second we added some custom ESLint rules and CI checks to flag raw findMany() calls without a limit.
But honestly relying on the reviewer to catch missing org filters is a ticking time bomb. It works for now but your wrapper idea is the real permanent guardrail.
Quick question: how do you handle edge cases with the wrapper, like when an admin needs to query across all tenants? Do you have a separate AdminRepository, or do you require a specific flag to bypass the wrapper?
separate admin path, yes. the normal repository takes the tenant id in its constructor and has no way to drop it, and the cross tenant reader is its own class with an explicit audit log line per call. that way a bypass is a visible import in review, not a boolean someone forgets to flip. the eslint rule on raw findMany is a good catch too. do your admin queries need to be fast, or is a slower reporting replica fine?
Mudassir this is elite-level multi-tenant architecture. The tenantId in constructor pattern combined with an explicit audit log for the cross-tenant reader is bulletproof. It forces the AI into the pit of success it literally cannot write a bad query for the normal path because the compiler won't let it.
To answer your question: a slower reporting replica is totally fine for our admin queries. Admin dashboards usually involve wide scans or heavy analytics, so we route those to a read replica. We don't need millisecond latency for an internal admin panel; we just need the primary transactional DB to survive.
Plus, if the admin query is hitting a replica, the replication lag naturally acts as a slight buffer against accidental data corruption.
Thanks for the architecture masterclass. I'm definitely refactoring our admin path to match this pattern!
The multi-tenant leak is the scary one because the code was correct for the sentence you wrote. "All users" meant every user in the database, and nothing in the prompt said otherwise.
Gautam's reply is the fix I would push hardest. Better prompts still depend on remembering every rule on the day you write them. A fixture with two tenants and an assertion that one never sees the other encodes the rule once, and it keeps working when the next person, or the next model, touches the endpoint.
The truncated marker row story is a nice detail too. People open the file and start working, so a warning on row 1,000 is not a warning. It probably belongs in the filename or the UI before the download starts.
At Ascendra Ventures we write tenant scope, row limits and export format into the acceptance criteria before any code is generated, because those are the decisions an assistant will otherwise make for you. Did the team end up moving to streamed exports, or keep the hard limit?
Ascendra writing the constraints into the acceptance criteria before generating code is a masterclass in engineering. That is exactly where we went wrong we treated the AI like an autocomplete tool instead of a junior dev who needs a proper spec.
You're also right about the multi-tenant leak. The AI isn't malicious; it just reads instructions literally. All users meant all users, and it did exactly what I asked. Gautam's point about the two-tenant fixture is definitely the ultimate safety net for that.
To answer your question: we actually stuck with the hard limit. We realized that building a robust streaming pipeline (that can handle mid-stream failures gracefully) was overkill for a simple CSV export. We kept the hard LIMIT, moved the warning to the UI, and changed the filename. Pragmatic over perfect!
Since you write these constraints into acceptance criteria upfront, do you have a specific checklist or template you use with the AI? I'd love to know how you structure that.
One surface is missing from this list, and it's the one that leaves your visibility entirely: the exported artifact itself.
Your prompt rules and the dual-tenant fixtures both guard inside the boundary. But a CSV cell that starts with = becomes a formula the moment a customer opens the file in Excel - on their machine, in their mail client, weeks after your logs rotated. Once the file is forwarded or renamed you have zero telemetry on it.
Worth one outbound-encoding test next to the fixture work: neutralize the leading = on the way out and assert on the generated bytes, not the prompt - the rule belongs in the serializer, where CI can actually check it. Generation time or download time? Generation feels safer, since the raw value can land in the LLM's own logs either way.
Nan this is a brilliant and slightly terrifying addition. You're absolutely right about the outbound boundary. We spend so much time securing the database but once that CSV hits a user's laptop and opens in Excel it's a completely unmonitored threat surface.
The outbound-encoding test in the serializer is a masterstroke. Testing the actual generated bytes is deterministic; testing the prompt is just wishful thinking.
But the point you made about the raw values landing in the LLM logs is the one that's going to keep me up at night now. Since the data leaves our DB and goes to a third-party model provider, how are you handling that? Are you masking PII before sending it to the LLM or do you have a specific redaction layer in your serializer for that too?
The OOM one is the most underrated of the three — a findMany() that passes locally is exactly the failure mode that survives code review too, because a reviewer sees a clean query, not the row count. We hit the same with a report export: the fix that stuck was a hard LIMIT in the query itself (say 100k rows + a "truncated" marker row at the end) rather than just streaming, since a streaming pipe can still die halfway and leave the client with a broken file.
Curious about the multi-tenant leak part: did you catch it in your own QA or purely from the Company A admin ticket? After a similar incident I started pasting the auth middleware alongside the schema into the prompt, since the tenant filter lives there, not in the Prisma model — the AI never saw the org scoping otherwise.
Max you are spot on about the OOM bug. findMany() looks incredibly clean in a code review. The reviewer sees a beautifully formatted query and completely ignores the fact that it's missing a .take(1000). It's a silent killer.
And I absolutely love your fix: a hard LIMIT plus a 'truncated' marker in the CSV. You're right, streaming can sometimes leave the client with a half-broken file, which is a terrible user experience. A hard limit is much safer.
To answer your question: honestly, it was purely from the Company A ticket. My QA passed with flying colors because I only had one tenant in my local database. I completely missed it.
Your tip about pasting the auth middleware into the prompt alongside the schema is a genius move. I was only pasting the Prisma schema, and like you said, the tenantId was there but the actual filtering logic lived in the middleware. The AI never saw it. I'm definitely adding that to my workflow. Thanks for sharing!
The multi-tenant one is scary because the code was "correct", it just answered the wrong question. I've started writing the constraints before the feature: who's allowed to see this, how big the data gets, what a hostile input looks like. Two minutes of typing that saves a weekend. Did you add tests for the tenant scoping after this?
Nilamadhab correct code answering the wrong question is the perfect way to describe this entire disaster. AI is incredibly obedient to the literal prompt, even if the prompt is missing the actual business logic.
And two minutes of typing that saves a weekend is a rule I wish I had framed on my desk. Writing the constraints (who can see it, how big the data is) before writing the feature is exactly the shift we need.
To answer your question: Yes absolutely. We added a strict two-tenant test fixture that fails the build if cross-tenant data leaks. It's now impossible to merge a query without the proper org_id filter. It was a hard lesson, but it's permanently fixed in CI now.
Since you've started writing constraints beforehand, do you have a specific template or checklist you use for those two minutes? I'd love to steal a good process.
The hard LIMIT plus a truncated marker row is the fix I'd actually steal — streaming sounds cleaner until the pipe dies halfway and hands someone a broken file. Did the truncated row ever cause a support ticket of its own, people assuming the export was complete?
Paul you read my mind. Yes, the truncated row absolutely caused its own support tickets! 😂
People download the CSV open it in Excel, and immediately start working. They never scroll to row 1,000 to read the tiny warning at the bottom. We quickly realized that putting a warning inside the file is useless if users don't read it. We ended up doing two things: moving the warning to the UI before they even click download, and changing the actual filename to export_truncated_1000.csv. That was the only way to make it impossible to ignore.
And you're right about streaming it sounds elegant until the pipe dies and the user gets a half-corrupted file. A hard limit plus a loud UI warning is the safest path. Shoutout to Max in this thread for the original idea!
happened to me last week. ai generated a whole settings page, looked perfect, and quietly dropped the rate limit field because i never mentioned it in the prompt. the feature 'worked' in the demo and ate our api quota in prod lol. now i list the edge cases first, not the feature
Haha eating up the API quota in production is a rite of passage for every developer using AI 😂 It's wild how the demo looks perfect because the happy path works but the AI completely ignores rate limits unless you explicitly spell them out.
Now I list the edge cases first not the feature is a brilliant rule. I'm definitely stealing that. If we tell the AI how the system should fail it actually writes much safer code. Thanks for sharing the war story!
This is easily one of the most honest and practical articles on AI-assisted development I’ve read recently. The multi-tenant data leak scenario you mentioned is genuinely terrifying because on paper, the code looks absolutely flawless and passes all standard unit tests.
To answer your question: Right now, 'Context Engineering' is absolutely the real skill. Even a GPT-5 or Claude 4 cannot guess business logic or hidden multi-tenant architectures unless we feed it that context. However, I believe the future isn't just about us writing better prompts, but rather tools evolving to map our repositories automatically (like advanced codebase indexing/RAG). Until then, your 3 rules should honestly be pinned on every developer's desk who uses AI. Great write-up Harsh.
yash thank you so much! That means a lot. The multi-tenant leak is exactly the kind of bug that keeps me up at night because it's completely invisible to unit tests. You are 100% right about the future being RAG and automatic codebase indexing. We are currently in this awkward middle phase where the models are incredibly smart, but they are completely blind to our architectural reality. We have to manually feed them the context that a proper indexing tool should already know.
Once tools can automatically map the entire repo and understand the relationships between tables, Context Engineering will become much easier. Until then, we are stuck playing the role of the AI's memory!
Glad you liked the 3 rules. Hopefully, we won't need them forever!
Better context helps, but tenant isolation, streaming limits, and CSV sanitization should also be enforced by tests and architecture. Prompts should inform the code, not become the security boundary.
Paul Prompts should inform the code, not become the security boundary' is the most important sentence in this entire thread. You've perfectly summarized the exact lesson I learned the hard way. Relying on a prompt for tenant isolation is like relying on a polite request to stop a hacker. It's just not a real defense. The AI can forget the prompt, ignore the prompt or hallucinate a completely different approach.
As you said, architecture and tests are the only true boundaries. The prompt just helps the AI get it right on the first try but the CI pipeline and the two-tenant test fixture are what actually keep the data safe.
This is definitely going straight into my team's engineering guidelines. Thanks for the brilliant insight!
Exactly. The two-tenant test fixture is what turns that principle into an enforceable guarantee. Prompts improve the first attempt, while CI prevents the dangerous one from shipping.
Paul Prompts improve the first attempt while CI prevents the dangerous one from shipping is the perfect TL;DR for this entire article. I'm definitely quoting that line in my next team meeting.
You've completely shifted my perspective from prompt engineering to systems engineering. The prompt is just a helpful guide; the CI is the actual safety net.
Thanks for the brilliant discussion Paul.
The 'compiles immediately, breaks in production' pattern is exactly what we see on the Electron side: an AI-generated IPC handler looks clean, but it silently assumes the sender window exists, that the payload is already validated, and that the dialog result fits a single await. In desktop the cost is worse than a support ticket — a main-process dialog blocking the event loop freezes every window. The fix is the same as you landed on: force the AI to state its assumptions back before writing code, and add the failure path as a first-class requirement rather than an afterthought.
mp3sss, this is a brilliant addition. The Electron angle takes this bug from annoying to catastrophic. A blocked main process freezing the entire app is a nightmare, and you're right the cost is way worse than just a support ticket. AI is so used to writing happy path code that it completely forgets about process boundaries. It just assumes the sender window exists and the payload is perfectly sanitized.
Your phrase add the failure path as a first-class requirement is the exact mindset shift we need. If we don't explicitly prompt for what happens when things go wrong, the AI will just pretend they never will. For your IPC handlers, do you use a specific TypeScript validation library (like Zod) on the receiving end to enforce those assumptions at runtime, or do you just rely on the prompt to generate the checks?
The constraints that bite hardest are the ones that feel too obvious to ever write down.
Pang too obvious to ever write down is the perfect description of the curse of knowledge. We assume everyone including the AI knows that a user must belong to an organization, so we don't bother stating it.
But the AI has zero context. It doesn't know what's obvious in our system. Those unspoken, invisible constraints are exactly where the scariest production bugs hide.
Such a simple but profound observation. Thanks for sharing it!
the annoying flip side is once you do paste the full context, every turn resends it and the bill climbs. I wrote up how that works here if it helps anyone: warpnav.com/ai-token-explained
Haha this is the exact trade-off I'm wrestling with right now. You're 100% right dumping the entire schema and middleware into every prompt is a fast track to a massive API bill. Context isn't free, and token limits are real.
What has helped me recently is being selective. Instead of pasting the whole repo I just grab the specific database model the relevant middleware and the acceptance criteria. It's a bit of manual work, but it saves a ton on tokens.
I'll definitely give your article a read token economics is something every dev using AI needs to understand right now. Thanks for sharing!