I've been building devpub, an open-source CLI that publishes and tracks articles on Dev.to. Last week I was in Cursor, adding a new analytics featu...
For further actions, you may consider blocking this person and/or reporting abuse
Actually reminds me of a while ago, a discussion I had with @pascal_cescato_692b7a8a20 he had benchmarked a bunch of models, I asked him to try Kimi K2.7, so he did and Kimi did exactly what was asked... Except it didnt apply safe coding standards, it had left plaintext secrets, which DQ'd it, yet it had followed the instructions flawlessly, produced clean code and was cheaper... But it leaked passwords... Actually what sparked me on a journey, but that's another story, quality gating was the solution I came up with, hard-set functions in the IDE that flag unsafe edits, whereas this is essentially 'fix the prompt, before you fire it', to make sure it does what you expect it to do, not what you explicitly told it to do...
Pascal, might be worth starting Kimi up again on the benchmark, but with something like this to make it conform to safe coding standards? I'm curious how it would change the weaker models' outputs too.
Fix the prompt before you fire it is the part I kept. Leftover daily capacity on PZERO dies at UTC midnight. I quote the live leftover row first. Thin book, I shrink the job. I do not fire yesterday's leftover offer because it was cheaper.
Exactly! That is the part I wanted people to take away from it. Make the decision before you fire the job rather than dealing with the wrong outcome afterwards.
The way you described your workflow is actually a good example of the same principle. Understand the current state first and then decide what should be sent.
This is exactly the distinction I was trying to highlight in the article. A model can follow the prompt perfectly and still produce something that should never make it into the codebase.
The Kimi example is really interesting because the instructions were followed correctly but the missing safety constraints completely changed the outcome. I would genuinely like to see how the same benchmark performs with NexPath in front of it, especially with the weaker or cheaper models.
I also like the quality gating approach you mentioned. I think prompt level gating and edit level gating could complement each other really well.
And now I am curious about that another story you mentioned.
If I rerun all the prompts with new rules, the benchmark will likely be different—and possibly in a way that no one expects…
if possible please share link or anything for the benchmark you are referring to
This is really helpful @sarvar_04 but One thing I’m curious about is how this compares with maintaining a good AGENTS.md or rules file. If I’ve already defined scope, coding conventions, and verification steps there, what does NexPath add that the agent’s persistent instructions don’t?
I see AGENTS.md/rules files and NexPath solving slightly different parts of the problem.
The rules file gives the agent persistent project context how the project works, what to follow, what not to touch, etc. NexPath is more about looking at the actual request you’re making right now and catching things that might be unclear, risky, or missing before you send it.
That said, I think there’s definitely overlap, and I don’t want NexPath to become something that just duplicates what’s already in the rules file. The idea of making NexPath aware of those existing rules and potentially learning from them is something I’d like to explore.
Okay got it. Thanks for the clarification Sarvar.
Your welcome
that's an interesting question pratik. nexpath's core idea is its an external entity keeping an eye on coding agent session. and then what goes through an improved prompt carries much higher value than what is inside agents md file because there is a much bigger orchestration difference plus coding agents see an instruction coming in from prompt with a completely different perspective than what is inside agents md. And by all means one can not cover everything in agents md file.
Okay got it. Thanks for the clarification Hiren.
Prompt quality is most useful when it becomes inspectable workflow state rather than a polished wrapper around a request. Versioning the instruction, the retrieved context, and the resulting diff makes it possible to learn which improvements were real.
I agree with this. Right now the focus is mostly on improving the prompt before it gets sent, but making the whole thing inspectable would be much more useful.
Being able to see what changed in the instruction, what context was used and then compare that with the actual diff would also make it much easier to tell whether the improvement actually helped or just made the prompt longer.
I think there is a lot of room to take NexPath in that direction.
yeah there is a lot of room and we are actually working in that direction. @alexshev your feedback makes sense and we had some plans to highlight diffs in some way but your suggestion highlighted one point that we weren't focusing on, thank you for your feedback
That’s a promising direction. I’d make the diff compare a versioned execution snapshot rather than only the prompt text: instruction, retrieved context, selected tools, model settings, and resulting code diff. Otherwise a seemingly small prompt edit can hide a changed tool policy or context window—the part most likely to change behavior.
Yeah, I think that’s a better way to look at it. Comparing only the prompt would miss a lot of what actually changed during the execution.
The tool selection and context can have just as much impact on the result as the prompt itself. Versioning the whole execution snapshot would make the comparison much more useful.
This is a really good direction. I hadn’t thought about the tool policy and context window being part of the diff as well.
The unstated invariant problem you describe hits a specific failure mode I've run into: agents optimize for the positive ask at the expense of what you didn't say.
make it fasterinterpreted asremove the throttleis locally valid reasoning. What catches these in my flow is treating safety rails and scope boundaries as explicit architecture constraints — literally present in the context window, not just remembered. NexPath's interception layer is basically automating that editorial pass. The question I'd have: does it learn your project's specific invariants over time, or does it apply general heuristics each time?Good question and honestly the throttle example is exactly the kind of thing I keep running into. It does both. There's a general set of checks that applies to everyone but the project specific part is where it actually gets useful. Over time it remembers what you keep vs what you edit out per project, so if you reject something a couple times it stops showing up. It also picks up on how you work from your own history and adjusts the framing. And for the project facts like test runner or backups it tries to only pull in the ones that matter for that specific prompt instead of dumping everything every time.
The one thing it wont learn away is safety and confirmation stuff. Even if you reject those repeatedly they stay protected. So the general checks are the floor and the learning happens above that. That was a deliberate call because those are exactly the invariants you dont want quietly disappearing.
yes it does learn project specific context. so its not mere general heuristics each time. I suggest you give it a try and share your feedback.
Answering your question, I stopped fixing prompts one by one. Scope, verification steps and what must not be touched live in a rules file that loads every session, so I do not retype it and I cannot forget it in the flow. Prompt layer at submit time solves same problem but per message, and per message is exactly where I am lazy when it matters. More interesting for me would be if such tool could write back into the rules file, then the lesson stays after the popup is gone.
Yeah that makes a lot of sense. I actually hadnt looked at it from the “lesson should persist” angle. The rules file approach solves the forgetting problem really well whereas NexPath today is more focused on catching things at submit time.
Having NexPath learn from those corrections and write the useful ones back into the rules file is definitely something worth exploring. Thanks for sharing this really good feedback. also if you have any other feedback or improvements you'd like to see NexPath adopt, I’d genuinely love to hear them. This kind of feedback is really helpful.
One more then. Every time somebody rejects the enhanced version that is also a signal, either rules file covers it already or the suggestion is wrong for that project. If you count which categories get rejected always and stop offering them, the tool becomes quiet exactly where it should be quiet. Otherwise it slowly turns into linter that everybody disables after one week.
This is a really good point and honestly one I hadnt considered from this angle.
A rejection shouldn’t just mean “not this time” it can be a signal that NexPath is being too aggressive or that the project already has that rule covered somewhere else. I really like the idea of using those signals to make NexPath quieter and more project aware over time otherwise like you said it just becomes another linter people get tired of and disable.
Really appreciate this suggestion. This is exactly the kind of feedback that can actually shape the product rather than just add another feature.
If you count which categories get rejected always and stop offering them, the tool becomes quiet exactly where it should be quiet.
=> yeah this needed to be taken care of and actually nexpath has feedback module which ask for reason when user rejects a prompt so that nexpath know when it needed to shut up. but yeah we are still working towards making it work end to end with custom feedbacks. custom feedbacks are where user can type anything as to why they rejected and then nexpath learns based on that and make sure that user gets better experince.
yeah writing back to rules file can be really helpful. but how often? when and when not, lets say you are using nexpath than what would you prefer like a option asking for adding in agents.md file when something important comes up?
Vague prompts are how coding agents quietly delete the thing you cared about. "Make it faster" with no boundaries is a classic way to lose a safety throttle or a retry path while the session still looks successful. Catching the ask before submit helps. I still end up reading the resulting change for the constraints I meant, because even a stronger prompt can get reinterpreted once the agent starts editing.
Exactly. That’s the distinction I was trying to get at as well. Better prompts give the coding agent clearer boundaries, but they’re not a substitute for reviewing the actual diff.
The “make it faster” example is a perfect one the agent can technically succeed while quietly removing a safety throttle or changing retry behavior you never intended to touch. Prompt quality helps reduce that ambiguity upfront, but the final change still needs human verification.
I’ve been experimenting with NexPath on a real open-source project, and the biggest takeaway for me is that prompt quality is becoming just as important as code quality when working with AI coding agents.
The “fix this” → “fix this safely, within this scope, with clear acceptance criteria and verification” difference is surprisingly impactful.
Curious to hear from other developers: do you already write structured prompts for Cursor/Claude Code/Windsurf, or do you also find yourself firing off quick prompts when you’re in the flow? 👇
This makes me wonder about the cost side of prompt improvement. If NexPath is adding context and verification requirements to every prompt, how much additional token usage does that introduce, and does the reduction in bad agent runs actually offset it?
Thats a fair question. I was curious about the same thing while testing it. There is some extra context added but the bigger question is whether that small overhead saves you from a much more expensive agent run going in the wrong direction. I havent done a proper cost comparison yet, though, so that would be a useful thing to measure.
good topic
Yes!
This is looks interesting but just curious to know if it’s open source ?
Yes the NexPath is open source and licensed under Apache.You can check out the source run it locally and contribute to it on github
Really interesting approach. I think the idea of improving the prompt before the coding agent acts is especially useful when you’re moving fast and don’t want to stop and write a perfect spec every time.
It also makes me think about how tools like CodeCan.net could benefit from similar quality and safety layers as AI-assisted development becomes more common.
Thanks. Yeah thats the main idea, catch the missing scope or safety bits before the agent runs off, especially when you dont want to stop and write a full spec every time. I think this kind of layer is going to show up in a lot more tools as agents get more autonomous.