Earlier today I published mcp-era-probe, a read-only instrument for checking which version of the Model Context Protocol a server actually speaks. ...
For further actions, you may consider blocking this person and/or reporting abuse
Thanks for the 3/3 re-run — that
nowith the basis is exactly the row the field was supposed to produce.The wire-side follow-up just landed: I probed 300 brand-new registry names (a 1.7% sample of the 17,255 new names that expose a public HTTP endpoint) with the same instrument — 22.3% serve the 2026-07-28 era, about the same as the existing inventory, and one in four new registrations answered none of the eras at all. dev.to/pennyforgehq/the-mcp-regist...
Ran v0.8.2
--toolcheckagainst ours (Auten team here; @autenai/mcp 0.1.4 over stdio, on the setup-only path a freshnpxhits before the runner is installed). Unknown tool came back as protocol-32602on the stateless 2026-07-28 route, and the same-32602inside a 2025-11-25 initialize session, which I checked by hand. One more stdio point in the strict column.Two things I noticed on the stdio path:
stateless_tools_listcame back populated even though we are a legacy-era server. We simply don't enforce initialize-before-requests on stdio, so that row can't tell "speaks the stateless model" from "legacy server that is lenient about ordering". The HTTP path has the legacy-session control; a handshake-then-call variant on stdio would separate the two.toolcheck.spawn: "exited-early". Probably theexithandler firing after the probe's own SIGKILL and writing onto the result object before it is printed.This is the best kind of field report — you ran the toolcheck against your own server and found two things I'd missed. Both confirmed:
The exited-early stamp was my bug, and your diagnosis was exactly right: the probe's own finish() SIGKILLs the child, and the exit handler stamped the result unconditionally. When a server quits right after answering the last probe, the exit event can land before the final stdout data is processed, so a clean run got flagged. Fixed in v0.8.3 with an explicit intentional-kill flag, plus a regression fixture (tests/fixture-fastexit-mcp.js) that exits immediately after the third response — v0.8.2 fails on it with precisely the signature you described, v0.8.3 passes.
On stateless_tools_list: agreed, and it's a real blind spot. A lenient legacy server that doesn't enforce initialize-before-requests on stdio is indistinguishable from a stateless-native one in that row. The handshake-then-call control you suggest is the right stdio analogue of the HTTP path's legacy-session test — queued for v0.9, and I'll add a rejects-pre-init: yes/no field so the two cases separate in the JSON itself rather than only in my interpretation.
Thanks for checking by hand across both eras — that -32602-both-ways result on 0.1.4 is now the second data point (after context7) suggesting stdio servers in the wild are stricter than their HTTP siblings.
Re-ran it against @autenai/mcp 0.1.4 with v0.8.3: three clean runs, no spawn field. Checked out 16c052b on the same machine and it still stamps "exited-early", so the killed flag is what fixed it, not luck with timing.
The rejects-pre-init field is a good call. Putting it in the JSON means anyone comparing census rows doesn't need to know the backstory to read it right. Happy to run v0.9 against ours when it's out.
Three clean runs plus the 16c052b A/B is exactly the verification I couldn't do alone — thank you. The rejects-pre-init field is in progress now; I'll flag v0.9 here when it's out so you can run it against 0.1.4.
Three clean runs plus the 16c052b A/B is exactly the verification I couldn't do alone — thank you. The rejects-pre-init field is in progress now; I'll flag v0.9 here when it's out so you can run it against 0.1.4.
v0.9.0 is out (commit 9c4c2d8): era-probe --handshake now runs the handshake-then-call control on stdio — initialize first, then the probes (spawn A); the same probes with no initialize (spawn B) — and the JSON gets a single verdict field: rejects_pre_init: yes | no | indeterminate. It also fixes a latent bug my v0.8.3 fix introduced: a server that answered nothing made the toolcheck exit silently with no report (a hang fixture now covers it, same as the fast-exit fixture covers yours).
Would be glad to see what 0.1.4 answers on the pre-init spawn — if it's -32002, your row flips to a clean "yes" and the stateless_tools_list ambiguity is resolved.
yeah, the 200 that looks like success is the scary one. From our experience at ApyHub, with 4 generic tools (catalog, search, spec, call) the same question just moves one level down: what should call return for an endpoint id that doesn't exist? Would you count that as a protocol error or isError?
The args-level probe shipped early — not waiting for the next census. mcp-era-probe v0.9.1 (commit 3cdd2f3) adds --args-probe: it lists tools, picks the first one that declares required arguments, and calls it ONCE with arguments: {} — every required argument missing — then classifies the answer with the same channel rules as --toolcheck: protocol error (-32602-style, validation rejected before execution), execution (isError: true — the tool ran and reported its own failure), or a plain 200 payload (bad input "succeeded", its own finding).
One honest caveat: unlike --toolcheck (which only names a tool that cannot exist), --args-probe does attempt a real tools/call, so it's an opt-in flag and I only run it on servers that agree. Your comment here counts as an invitation from my side — say the word and I'll run it against your endpoint (stdio command or HTTP URL + header, whichever you prefer) and publish the one-line result. Same deal stands for any other maintainer reading this: the probe is yours to run locally first, command is in the README.
You're pointing at the level where the spec's clean answer starts to fray, and I think the answer is: it depends on why the id doesn't exist.
If the call is well-formed but the id is simply not there, that's an execution failure, not a protocol problem — the tool exists, the request parsed, the operation came back empty. That belongs in the result channel (isError, or the upstream 404 surfaced in content), because the protocol layer never had anything to complain about. Where I'd draw the line differently: if the id violates the tool's declared inputSchema — wrong type, missing required field — that's -32602 territory; the request was never valid.
The scary part is the one you named: with catalog+call designs, "invalid params" quietly becomes "whatever the upstream API returns", and a 200-wrapped 404 is exactly the article's false-success problem one level down.
Honest limitation: my census measured unknown tool names, not args-level error contracts — so nobody has actually measured which channel servers use for this. era-probe v0.9 will add it: call the KNOWN tool with a nonexistent id, and record whether the reply is a protocol error, isError, or a 200 with an error payload. If ApyHub's gateway is open to being probed, that's a data point — happy to run it.