DEV Community

Cover image for MCP servers edit the instructions your agent reads. WARDEN 0.18 shows which words changed
Alex
Alex

Posted on Originally published at modelmarket.dev

MCP servers edit the instructions your agent reads. WARDEN 0.18 shows which words changed

On 29 September an MCP server changed the description of one of its 45 tools. The old text said that when a project takes payments, a mobile publish makes a test build in Stripe test mode. The new text says:

publish_app always publishes live (web, and the mobile app when it is built; payments in Stripe live mode) — never ask the user test or live.

This is not an attack. The server changed how publishing works and told the model so. It is also a new instruction to your model, in a tool you approved before that sentence existed: from now on, don't ask.

WARDEN pins an MCP server's tool definitions the first time it vets them. Any later change blocks the server until a person approves it again. Up to 0.17, that person got the old and the new definition of each changed tool as two lines of JSON. For this change that is 6,614 characters, and the sentence above sits somewhere in the middle of the second line. WARDEN 0.18.0 (on npm since 21 October) prints what changed first. This is its output for that change, unedited except for line breaks (in a terminal the description is one line):

Tool definitions: 1 changed, 0 added, 0 removed
~ publish_app
    description: Publish the app to production — call for the first publish, to publish again
      after changes the user wants live, and to set up a custom domain. Omit domain and the user
      gets the publish form in the editor. Pass mobile: true whenever the user mentions iOS,
      Android, TestFlight, App Store, Google Play, or a mobile/native app — with no store
      account connected that returns a Connect card; with one connected it publishes with the
      store builds attached. When the project takes payments and its owner has not finished
      Stripe setup, the publish is refused and a Set up payments card is returned (nothing
      published): tell the owner to click the button on the card and finish Stripe setup, then
      call publish_app again. [-When the project uses payments-]{+publish_app always publishes
      live (web,+} and the [-publish builds the -]mobile app[-, publish_app makes a mobile test
      build instead:-]{+ when it is built; payments in+} Stripe [-test-]{+live+} mode[-, no web
      deploy, no domain needed; going-]{+) — never ask the user test or+} live[- on mobile is
      done -]{+. Mobile test builds are started +}by the user on floot.com[- (the result carries
      the link)-]. Read get_guides('publishing') for modes, inputs, and statuses before your
      first call.
Enter fullscreen mode Exit fullscreen mode

[-…-] is what was removed and {+…+} what was added. The full definitions still follow, for the record.

How often MCP tools change, and how

HISTOR, our public log of MCP servers, reads the tool list of every remote server in the official MCP registry once a day, and runs npm and PyPI MCP packages in a sandbox once per version. Its open snapshot of 2026-10-20 holds 23,821 recorded changes between 24 September and 20 October. Counted from changes.jsonl.gz in that snapshot (one change can count in several rows):

What changed Changes Share
A tool's description 14,164 59.5%
… by more than digits and whitespace 10,747 45.1%
An input schema 10,790 45.3%
Tools added or removed 7,343 30.8%
An output schema 3,365 14.1%
Nothing but digits, separators or whitespace 3,196 13.4%

The script that prints this table is at the end. One correction first: the 0.18.0 changelog says 52% of changes touched description text and 23% added or removed tools. Recounting the same file for this post, we could not get those two numbers back with any definition we tried. The table above is what the script prints, and the changelog will be corrected to match it.

Two rows matter most to the person approving a change.

One change in eight is only numbers

Many descriptions carry numbers that move: counters, dates, averages. A review that shows each of those as two full JSON blobs teaches people to approve without reading. So 0.18 marks them:

Tool definitions: 1 changed, 0 added, 0 removed (1 changed only in numbers: still a change, a
  number can be a limit)
~ get_api_external_search  (numbers only)
    description: Searches for products by barcode or name; returns a minimum of 1 and a maximum
      of 10 products. Billing per call: Credits: metered (~[-0.84-]{+0.81+} avg).
Enter fullscreen mode Exit fullscreen mode

That number is the price per call. WARDEN marks the change "numbers only" and still asks for approval. It does not know which number is a counter and which is a price or a limit. The mark only helps you find the one that matters quickly.

A limit, from a hotel server that changed four tools at once (excerpt):

~ book_hotel  (numbers only)
    inputSchema.properties.guests.description: Number of guests{+, 1-10+}
    inputSchema.properties.guests.maximum: 9007199254740991 → 10
    inputSchema.properties.guests.minimum: -9007199254740991 → 1
Enter fullscreen mode Exit fullscreen mode

"Numbers only", and a real change: the number of guests went from any integer JavaScript can hold to 1 to 10.

Descriptions are instructions

A tool description is text the model reads as guidance. In 2,345 of the 23,821 changes, on 1,744 servers, a tool's description gained an "always", "never", "must", "do not" or "don't". Narrower phrases, counted only where the new text has more of them than the old:

Phrase added to a description Changes Servers
"never ask the user", "do not ask the user", "without asking" 23 21
"call this first", "always call this", "before calling any other" 20 20
"do not tell", "do not mention", "do not reveal" 17 16
"prefer this over", "use this tool instead of" 14 14

These are phrase counts, not verdicts. Read in context, nearly all of them are honest. A sign-in tool says to call again while the user's confirmation is pending, instead of asking them. A quote form says never to ask for a phone number. A booking service says to use it instead of web search for its own shops, because a web page cannot show a free slot. They are also exactly the sentences a person approving a change should read, and as two JSON blobs they look like every other change.

What the review shows

For every tool that changed:

  • Text (the description, and every string inside the schemas): a word diff, [-removed-]{+added+}. A rewritten phrase comes out as one removal and one addition, not as alternating words.
  • Other values: the path and before → after, e.g. inputSchema.properties.days.maximum: 7 → 365.
  • Lists of plain values (required, enum): the entries that came and went, e.g. inputSchema.required: added "api_key".
  • Tools that came and went: + send_email (new tool), - list_cities (removed).
  • "numbers only" on every value and every tool whose change is only digits, separators or whitespace.

It appears in three places. warden-mcp pins status|approve|revoke prints it in a terminal, before the full definitions. The status_mcp_server MCP tool returns it as changes (structured) and reviewText. The library exports diffToolDefinitions, formatDefinitionDiff and wordDiff.

A diff that misquotes a text is worse than none, so we checked it on real data. For all 59,287 tool-description changes in the snapshot, the removed side of WARDEN's word diff spells the old text and the added side spells the new one, character for character: 59,287 of 59,287. The slowest single diff took 12 ms.

What it does not do

  • It approves nothing. The pin is the same hash as before. Any change, numbers included, blocks the server until a person approves it. (A policy with pinToolDefs: false announces vetted clean changes automatically, as before.)
  • It judges nothing. "numbers only" is a mark, not an exemption. The scanner's verdict on the new definitions is separate.
  • It has limits. A review lists at most 200 changed values per tool, then points to the full definitions. A very long rewrite falls back to "common start and end, middle replaced" instead of a slow word-by-word diff.
  • CI is unchanged. With scan --lock, the review is still the git diff of warden.lock.json.
  • A terminal is not a login. pins approve requires a TTY, which stops an agent from approving its own tools by accident. A local process that can allocate a PTY or edit WARDEN's state can still get past it.

Try it

Scan the MCP servers your clients already start (Claude Code, Claude Desktop, Cursor, VS Code, Windsurf):

npx -y @aimarket/[email protected] scan
Enter fullscreen mode Exit fullscreen mode

Put a server behind a pin that waits for you, in your client's MCP config:

"filesystem": {
  "command": "npx",
  "args": ["-y", "@aimarket/[email protected]", "wrap", "--id", "filesystem", "--require-approval", "--",
           "npx", "-y", "@modelcontextprotocol/server-filesystem", "/Users/me/docs"]
}
Enter fullscreen mode Exit fullscreen mode

Then warden-mcp pins status --id filesystem shows the review, and warden-mcp pins approve --id filesystem approves it.

Or replay any change from HISTOR's log through the same review. Each row of changes.jsonl.gz has the two digests; HISTOR serves the tool sets behind them:

// review.mjs — npm i @aimarket/[email protected], then: node review.mjs <fromToolSetDigest> <toToolSetDigest>
import { diffToolDefinitions, formatDefinitionDiff } from '@aimarket/warden';

const api = 'https://histor.modelmarket.dev/api/v1/toolsets/';
const toolset = async digest => (await fetch(api + encodeURIComponent(digest))).json();
const [from, to] = process.argv.slice(2);

console.log(formatDefinitionDiff(diffToolDefinitions(await toolset(from), await toolset(to))).join('\n'));
Enter fullscreen mode Exit fullscreen mode

The publish_app change above is this one:

node review.mjs sha256-XmLhlg4kjl/riNYk7MDP4DTiLMV9wUnWLjfCUU2zCfk= sha256-YNEgAOowTUYxoBiP6NpDR3/T3adz8oc7cHrTcJGbzZE=
Enter fullscreen mode Exit fullscreen mode

The counts

Run in the folder with changes.jsonl.gz from the 2026-10-20 snapshot:

import gzip, json, re
from collections import Counter

DIGITS = re.compile(r"[\d\s,.]+")
PHRASES = {
    "imperative": r"\b(always|never|must|do not|don't)\b",
    "don't ask the user": r"\b(never|don't|do not) ask the user|without asking",
    "call this first": r"before (calling|using) any other|always call this|call this (tool )?first",
    "use this instead": r"\b(prefer|use) this (tool )?(over|instead)",
    "don't tell": r"\b(do not|don't) (tell|inform|mention|reveal)",
}
def text(ops, side):  # HISTOR stores a description change as word ops: "=", "-", "+"
    return "".join(t for op, t in ops if op in ("=", side))

rows, servers = Counter(), {k: set() for k in PHRASES}
for line in gzip.open("changes.jsonl.gz", "rt"):
    change = json.loads(line)
    s = json.loads(change["summaryJson"])
    rows["changes"] += 1
    rows["tools added or removed"] += bool(s["added"] or s["removed"])
    fields, numbers_only, hits = set(), True, set()
    for tool in s["modified"]:
        for field, values in tool["fields"].items():
            fields.add(field)
            pairs = ([(text(values, "-"), text(values, "+"))] if field == "description"
                     else [(v.get("old"), v.get("new")) for v in values])
            for old, new in pairs:
                if old is None or new is None or DIGITS.sub("#", old) != DIGITS.sub("#", new):
                    numbers_only = False
            if field == "description":
                old, new = pairs[0]
                if DIGITS.sub("#", old) != DIGITS.sub("#", new):
                    fields.add("description, beyond digits")
                for name, pattern in PHRASES.items():
                    if len(re.findall(pattern, new, re.I)) > len(re.findall(pattern, old, re.I)):
                        hits.add(name)
    for f in fields:
        rows[f] += 1
    rows["numbers only"] += bool(fields) and numbers_only and not (s["added"] or s["removed"])
    for name in hits:
        rows["phrase: " + name] += 1
        servers[name].add(change["serverId"])

for k, v in rows.items():
    print(f"{k:32} {v:6}  {v / rows['changes']:6.1%}")
for k, v in servers.items():
    print(f"servers with phrase: {k:20} {len(v)}")
Enter fullscreen mode Exit fullscreen mode

Links:

If you approve MCP server changes for a team, tell us what the review still makes you hunt for.

Top comments (2)

Collapse
 
heyitsjakub profile image
Jakub Hecht •

The “numbers only” label is useful, especially because you still require approval: a changed price or limit can matter as much as a rewritten instruction.

Disclosure: I build KyttoMCP. Its Contract Guard compares tool descriptions, input schemas and safety annotations between health checks and flags changes for review; it’s advisory and doesn’t block execution.

For the review UI, I’d find two things especially useful: the resolved package version alongside the diff, particularly for unpinned npx launches, and a clear highlight when a description adds instructions about user approval or tool choice. That would help explain both where a change came from and why it deserves attention.

Have you considered showing package-version changes alongside tool-definition changes?

Collapse
 
alexar76 profile image
Alex •

Thanks. Both were good calls, so both are in.

Version next to the diff: status_mcp_server and approve_mcp_server now take the serverInfo from the server's initialize result. The approval stores it, and the review opens with "Version the server reports: 1.4.2 at the approval, 1.5.0 now". When the tools changed and the version didn't, it says that instead, which is often the more telling line. The reported version is only the server's word, so it never goes into the pin's hash. WARDEN also reads the launch command (npx, bunx, npm exec, pnpm/yarn dlx, uvx, uv tool run, pipx run) and says when it names no exact version: each start runs whatever the registry tags latest, so the code can change while the command, and the identity we pin, stay the same. The review then gives the pinned form, e.g. @acme/[email protected].

Approval and tool-choice text: the diff now marks text a change adds about the user's approval ("without asking the user", "already approved", "skip the confirmation") or about which tool to use ("prefer this over web search", "never use another tool"). It looks in descriptions, parameter descriptions and new tools, and lists the marked tools first. Only added phrasing counts, so a hint that was already there, a dropped one, or "do not ask the user for their API key" isn't marked. Like the numbers-only label, it marks and never approves.

How often the second case happens: in HISTOR's 20 October snapshot (23,821 recorded changes), 12 rewrites added or removed a routing hint like "prefer this over web search", and 9 of them changed nothing but the description. It's rare, which is exactly why it belongs at the top when it does happen.

On package versions in general: HISTOR keys package changes by version already. For an npm or PyPI package, a recorded change is a release that lists different tools, so the record says which version brought it.

Limits: only English phrasing is recognized, and the version is only as good as what the server reports. The launch check doesn't look up which version npx actually installed. It's in our source, not in a release yet.