Most advice about MCP server design says keep the tool count small. I read 91 sources: 38 say design fewer tools, 20 give a number (mostly 10–20), ...
For further actions, you may consider blocking this person and/or reporting abuse
The vague-names result (exact schema loaded in 69 of 72 runs with honest names, 7 of 72 with vague ones) matches something we hit one layer down: when the name is the index, the matching code can fail before the model gets a choice. Our tool search over 306 finance tools stemmed "rates" to "rat" and ranked the wrong tools first until we fixed it, and it crashed on the literal word "constructor". After the fixes, 46 of 48 everyday requests it was never tuned on get the right tool first.
On the context side we ended up with three front tools: 1,109 tokens of definitions per request, against 34,508 for the seven servers installed separately (our own bench, our own tools, so one data point, not a general claim). I built it, it's free and runs locally:
npx -y canli-mcp, code at github.com/arhancanli/canlicapital.A question on your runs: with honest names at 100 tools, was the first ToolSearch hit ever a look-alike (pdf_optimize vs pdf_compress), or always the exact tool?
Good question, and the short answer is: there was no first hit to be a look-alike, because with honest names Claude Code never searched. In all 36 runs at 100 tools (18 tasks × 2) it read the name list and asked for schemas by exact name with select:. Zero keyword searches, so no ranking step sat between the names and the model.
It did pull a rival in once, on purpose. The task was "scan.pdf takes forever to show up on our website, make it display page by page while downloading", where pdf_optimize and pdf_compress sit next to each other. One of the two runs loaded both (pdf_optimize first), read both descriptions and called pdf_optimize. The other loaded pdf_optimize plus pdf_info. Every other extra load across the 36 runs was a read-only helper (pdf_info, image_info, zip_list), never a competing writer. Wrong first loads: 0 of 36.
The look-alike-first cases all came from the vague-name variant: 8 of 72 Claude runs (combine_files for "3 JPGs into one PDF" four times, optimize_image / convert_image for "shrink a PNG" three times), and 34 of 72 for Gemini. That is also where Claude fell back to keyword search, in 28 of 36 runs at 100 tools, so it is the only place in my setup where a matcher like yours would have been in the path. Your "rates" → "rat" bug is one layer below anything I measured: with honest names the list went straight to the model.
Your case is a different regime from mine, and an interesting one. My sweep stops at 100 tools on one server, with look-alikes inside one catalog. Seven servers in one domain adds something I never tested: the same operation under two prefixes, where the names can't settle it and the model has to read descriptions or try both. The only hint my data gives for 300 tools is that Codex's search output was already truncated at 100 in 34 of 36 runs, so at that size some ranker is in the path whether you build it or the client does. That setup, several servers with overlapping tools in one session, is on my list for a follow-up round.
The per-run queries are in the data: research/tools-count in the worklore-mcp repo, names.md and the toolsearch_queries column of results.csv.
0 of 36 wrong first loads says the name list did the work, which also explains why a ranker bug like our "rates" -> "rat" stemmer never shows up in your honest-name runs. Thank you for the exact breakdown.
The case I'd watch in your follow-up is the same operation under two prefixes: names can't settle it, so the model either reads both descriptions or picks by list order, and those fail differently. If you log which one it called first when both are loaded, that would tell the two apart.
If canli-mcp is useful to you, a star on github.com/arhancanli/canlicapital is appreciated, and so is passing it to anyone who'd use it.
"What search saves is context, not money" is a useful correction. Most of the advice I read treats those as the same thing. Your runs were short direct tasks, so I'm curious whether the gap flips in long sessions: tool schemas are a fixed cost, but tool results pile up turn after turn, and in my experience that's where most of the input goes. Did the per-run numbers show how much of the input was schemas vs. returned tool output?
good question, and honestly my runs can't answer it. the tools were fake and returned tiny results, so each extra turn added only ~130 tokens. nearly all the input was schemas + system prompt sent again each turn.
long sessions are where tool output piles up, but i don't think i can model that honestly. fake outputs would just give me whatever sizes i picked, and i'd feel bad burning that many tokens to measure my own guesses. it needs logs from real working sessions. if you (or anyone reading) have some and don't mind sharing, i'd be happy to dig into the schemas vs output split and write it up
That's a fair call, and more careful than most benchmarks I read. Fake outputs really would just measure the sizes you picked. I keep per-turn token usage from my own sessions in CodeCook, the VS Code agent I'm building, so I could pull schema vs. tool-output counts from a few longer ones, just the numbers, no code or file contents. Happy to send them if that's useful. One thing to watch for: long sessions usually get compacted at some point, so the split looks quite different before and after that.
yes please, that would be exactly the data i'm missing. a few longer sessions is plenty.
what would make it directly comparable with my runs: per turn, input tokens split into (a) system prompt + tool schemas, (b) accumulated tool output, (c) everything else, plus a marker on the turn where compaction happened. before/after compaction as two separate series is a great point, i'd keep them apart rather than average over it. if your counts are already in some other shape, send what you have and i'll adapt.
dev.to connect (DM) or X @YahhiMurr both work. i'll write it up as a follow-up with CodeCook credited as the source, and you get to read the draft before it goes out.
Perfect — I'll pull a few longer sessions and break each turn into (a) system + schemas, (b) accumulated tool output, (c) everything else, with a marker on the compaction turn, so you get before/after as separate series. I'll DM you on Connect (or X @YahhiMurr if that's easier) once the numbers are ready. Looking forward to the write-up, and happy to look over the draft before it goes out. Thanks for offering to credit CodeCook.
Quick preview from a real CodeCook session I already published numbers for on my site (codecook.makayis.com), until I can pull the finer split:
One long turn after smart compression:
I don't yet have the clean per-turn split into (a) system+schemas, (b) accumulated tool output, (c) everything else with a compaction marker from these public figures — I'll DM that once I export a few sessions in that shape. Happy for you to use these headline numbers in the meantime if they help frame the before/after.
Thanks for publishing the raw runs, that's rare in this debate. The bit I'd underline is that tool search makes names matter more than count: if the model picks from names before it ever sees a schema,
querytells it nothing. In the vague-names variant, did ToolSearch still surface the right tool first, or did it just take more searches to get there?Mostly first try, but it read a lot more to get there. With vague names Claude Code still picked the right tool in 72/72 runs. Its first search missed the needed tool in 8 of them (5 at 100 tools), and each of those searched again and found it. The real cost was reading: at 100 tools it ran keyword searches in 28 of 36 runs and loaded 7.6 schemas per run instead of 1.6, about +21% input tokens. So under tool search the descriptions still decide which tool gets picked, and the names decide how much the model reads first. Gemini in Antigravity had it worse: its first schema read was wrong in 34 of 72 runs.