As developers increasingly build autonomous AI agents designed to interact with databases, APIs, and enterprise software, a quiet architectural failure is eroding system reliability. An AI agent does not run its tools by itself. It reads a list of tool definitions, picks one, and fills in the arguments, leaving the underlying code to execute the actual call. When a system provides a modest selection of five tools, that choice is remarkably straightforward. However, when that same list balloons to sixty options pulled from four interconnected servers, the model begins to miscalculate, selecting the wrong tool with alarming frequency. Crucially, the wrong tool usually returns a normal, successful response, masking the error beneath a veneer of false confidence.
Recent research spanning the last three years illuminates the root causes of this phenomenon. Studies show that long prompts are read unevenly by large language models, semantically similar tools are difficult for models to tell apart, and selection accuracy degrades systematically as the pool of choices grows. To understand this challenge, developers must look closely at how tool definitions are structured. A typical definition consists of three distinct components: a name, a description, and a schema for its arguments. The model never actually sees the underlying code of a tool; instead, it relies entirely on the accompanying text injected inside its context window right alongside the system prompt and the conversation history.
Major artificial intelligence providers have long recognized this constraint. OpenAI’s function-calling guide explicitly instructs developers to clearly describe the purpose of the function and each parameter because that descriptive text serves as the absolute sole source of information the model possesses regarding the function.
This textual dependency carries a heavy computational and operational cost. In a November 2025 engineering post, AI developer Anthropic detailed a configuration utilizing five connected servers running on the Model Context Protocol, an open standard designed to plug external tools seamlessly into an agent. Across those five connected servers, the environment exposed 58 distinct tools that collectively consumed approximately 55,000 tokens before a single word of conversation had even commenced. In a separate internal case study, Anthropic reported that its own internal tool definitions once bloated to a staggering 134,000 tokens prior to optimization.
The same engineering documentation points directly to the core vulnerability plaguing modern agentic systems: the most common operational failures involve incorrect tool selection and flawed parameter assignment. Because a wrong choice frequently mimics a correct one, it easily slips past human monitoring. For instance, if an agent mistakenly invokes a function named search_orders when it should have executed search_invoices, the API call frequently succeeds and returns valid-looking data. The model then synthesizes a perfectly confident, yet fundamentally incorrect, answer derived from the wrong data source.
Long Prompts Are Read Unevenly
The first fundamental driver of this failure applies broadly to any extended input, extending far beyond tool definitions alone. Extensive research confirms that language models do not process lengthy inputs with uniform attention from beginning to end.
In a landmark study titled "Lost in the Middle," published in the Transactions of the Association for Computational Linguistics, researchers Nelson Liu and colleagues at Stanford University explored how models handle buried information by hiding a single relevant document among numerous distractors. Their findings revealed that retrieval accuracy peaked when the relevant document was positioned either at the very beginning or the very end of the prompt. Performance noticeably degraded when the critical document sat squarely in the middle, a limitation that persisted even in models specifically engineered to handle massive context windows.
Complementing these findings, a 2025 report titled "Context Rot," published by the vector database firm Chroma, tested eighteen prominent models on straightforward retrieval tasks while progressively increasing the volume of input text. The authors concluded that models fail to utilize their context uniformly and that performance grows increasingly erratic as input lengths expand.
A comprehensive tool list serves as a textbook example of this vulnerability. Sixty tool definitions represent sixty miniature documents, and the specific function an agent requires at any given moment is frequently trapped in the middle of that dense block of text.
Selection Accuracy Falls as the List Grows
Direct empirical measurements have begun to quantify the exact toll that tool bloat takes on model accuracy. A May 2025 preprint titled RAG-MCP, authored by Tiantian Gan and Qiyao Sun, tested a model’s ability to select the correct tool from a massive pool of Model Context Protocol tools. When every available tool definition was dumped directly into the prompt, the model selected the correct function a mere 13.62 percent of the time.
To combat this, the researchers introduced an intermediate retrieval step. By running a search over the tool descriptions to filter out low-probability candidates and feeding only a handful of likely options to the model, selection accuracy surged to 43.13 percent—more than three times higher—while simultaneously cutting the prompt length by more than half. Because the paper remains a preprint and has not undergone formal peer review, its exact figures warrant careful interpretation, yet the directional trend is undeniable.
Additional confirmation came from a peer-reviewed study accepted at the DATE 2025 conference. Titled "Less is More" and authored by Varatheepan Paramanayakam alongside fellow researchers, the work investigated small language models operating on edge hardware. The authors reported that selectively reducing the number of available tools significantly enhances function-calling performance while slashing execution times by up to seventy percent due to the reduced volume of text processed by the hardware.
Compounding the issue of sheer volume is the problem of functional overlap. The MetaTool benchmark, presented at ICLR 2024 by Yue Huang and colleagues, evaluated whether popular models could accurately determine when to deploy a tool and which specific option to choose. When tested against toolsets specifically engineered to include overlapping functions, the majority of eight popular models evaluated continued to struggle with effective tool selection.
Anthropic’s technical guidance on writing tools for agents echoes these academic insights, warning that excessive or overlapping tools can easily distract agents. When tools share functional boundaries or feature vague descriptions, agents frequently grow confused about which utility to execute. To mitigate this, developers are advised to group related tools under shared naming conventions, such as prefixing functions with asana_search or jira_search, ensuring that the identifier itself clarifies the originating system.
This overlap problem accelerates when engineering teams integrate third-party servers, as independent development teams rarely coordinate their naming schemas. Consequently, two distinct servers might each provide an unaliased function named search, forcing the model to arbitrate between them based solely on often ambiguous text descriptions.
Every Description Is an Instruction
Because large language models treat tool descriptions as actionable instructions rather than passive metadata, these fields introduce significant security vulnerabilities. In April 2025, security researchers at Invariant Labs detailed a tool poisoning attack targeting the Model Context Protocol. In their demonstrations, hidden instructions embedded inside a tool’s description—invisible to human users but fully legible to the parsing AI model—instructed the agent to exfiltrate private files to external servers.
Every added tool expands the attack surface by introducing untrusted text for the model to weigh, often originating from third-party repositories outside the developer’s direct control. A shorter, tightly curated tool list minimizes both processing noise and security exposure.
While state-of-the-art language models excel at single-turn calls—where a single request is answered by a single function execution—long-horizon reasoning and dynamic decision-making remain persistent challenges. Public benchmarks such as the Berkeley Function Calling Leaderboard, presented at ICML 2025 by Shishir Patil, Ion Stoica, Joseph Gonzalez, and their colleagues at UC Berkeley, demonstrate that models handle isolated tasks well but falter during extended agentic loops. In real-world deployments, an agent executes numerous sequential choices, with each decision drawing from the cumulative context. A single erroneous tool selection early in the workflow can propagate errors through every subsequent step.
What Model Providers and Platforms Recommend
Responding to these empirical findings, major AI providers have updated their design recommendations. OpenAI’s official documentation now advises developers to keep the initial pool of available functions small to maximize accuracy, offering a soft guideline of fewer than twenty functions at the start of a turn. Both OpenAI and Anthropic have introduced native tool search features. In this architecture, an agent launches with a generalized search capability and a minimal set of core tools, dynamically loading additional definitions only when a specific request demands them—mirroring the retrieval-augmented generation strategies tested by academic researchers.
Anthropic reports that its internal Tool Search feature elevated benchmark evaluation accuracy from 49 percent to 74 percent for Claude Opus 4, and from 79.5 percent to 88.1 percent for Claude Opus 4.5. However, independent evaluations suggest caution. In December 2025, tool platform Arcade loaded 4,027 tools into Anthropic’s tool search beta environment and executed 25 everyday tasks. The correct tool surfaced in search results for 56 percent of tasks utilizing regular expression search modes, rising to 64 percent under BM25 keyword-ranking configurations. Arcade’s evaluation measured only whether the correct tool was successfully retrieved, not whether the agent subsequently executed it correctly.
These findings suggest that retrieval merely shifts the selection bottleneck into a search operation, which itself remains prone to failure. Organizations managing large function catalogs must maintain concise per-agent tool lists, implement robust retrieval layers, and continuously measure the efficacy of both systems.
The underlying research points to a singular, critical engineering discipline: deliberately deciding which tools are allocated to each specific agent, centralizing that governance, and auditing every selection. Developers should actively avoid the common anti-pattern of maintaining a single, monolithic catalog of every available tool for every agent. A customer support agent requires order lookup and refund capabilities, but it has no operational need for the financial export tools utilized exclusively by accounting personnel. Frameworks like the OpenAI Agents SDK reflect this reality by isolating tools within agent-specific lists and permitting controlled handoffs between distinct conversational instances. Similarly, Model Context Protocol clients typically allow users to toggle individual servers on and off dynamically.
Ecosystem tooling is gradually adapting to this paradigm. LLMOps platforms vary widely in their handling of tool governance; some systems merely display tool invocations within execution traces, while platforms like AcruxCore store and version tools within a centralized catalog, tying individual prompts strictly to the specific tools they require. Regardless of the underlying infrastructure, debugging environments must capture the exact name of the tool selected by the model alongside the preceding user request. Without granular tracing, an incorrect tool invocation disguised by a successful API response remains virtually undetectable.
For engineering teams evaluating their current architectures, the immediate remedy involves auditing the number of tools loaded at the start of an agent’s turn. If that count exceeds OpenAI’s recommended threshold of twenty, developers should prune redundant capabilities, harmonize overlapping naming conventions, and establish dedicated evaluation harnesses to test tool selection independently of general conversation quality. As the industry matures, treating the tool list as an extension of the prompt itself will remain a foundational requirement for reliable agentic deployment.
Comments (0)