Why a long tool list slows everything down
A flat list of two thousand tools asks one model to hold all of them in mind. Bluwing narrows a tree first, so the model sees twelve or fewer.
The first version of most tool-calling systems is a flat list. You define search, read, send, book, refund, and twenty more, paste them into the prompt, and let the model choose. It works at ten tools. At a hundred it starts missing obvious matches. At a thousand it gets slower, more expensive and less accurate at the same time.
That is not a prompt-engineering problem. It is a search problem.
A flat list makes the model do the indexing
Every tool definition is context. The model has to read all of them before it can decide that book_table is the one it wants. For a short list that is cheap. For a long list it means thousands of tokens describing tools that have nothing to do with the request, and the relevant definition is buried somewhere in the middle.
The effect compounds: more context means more time per turn, more tokens billed, and a higher chance that a plausible-sounding tool wins over a correct one. The model is not failing because it is weak. It is failing because we asked it to search before it could reason.
Narrow first, then decide
Bluwing treats a tool set as a tree. Branches describe areas of work, leaves are the tools themselves. A request enters at the root, and each step only compares the request against the children of the branch it is already in.
- A request about a table booking goes to the dining branch, not the payments branch.
- The dining branch holds a dozen tools, not two thousand.
- The engine reasons over that small group, then returns the call.
The narrowing is a ranking step over short labels, not a generation step over full schemas. It is why the engine can afford to look at the whole tree on every turn without paying for it in context.
Twelve is a budget, not a magic number
We target twelve or fewer tools in the working set. That number is a design budget: small enough that the definitions fit comfortably in a prompt, large enough that a real branch is still useful. If a branch grows past it, the branch splits. If two leaves describe the same action, they merge.
The budget changes what you do with a large integration. Instead of trimming features to keep the list short, you group them. One billing branch can hold invoices, refunds, subscriptions and payment methods without making the invoice request read about subscriptions.
What this looks like at the API
Nothing changes in the shape of the response. You still get standard tool_calls, and your app still runs the function. The difference is the set of definitions the engine considered:
{
"finish_reason": "tool_calls",
"tool_calls": [{
"id": "call_8sKf2LmQ0aPz",
"type": "function",
"function": {
"name": "book_table",
"arguments": "{\"restaurant\":\"Nok\",\"party_size\":4,\"datetime\":\"2026-11-05T19:00\"}"
}
}]
}The response does not tell you how many tools were skipped. It does not need to. What matters is that book_table was in the final group, and that every value in the arguments came from the sentence.
The trade you are making
A tree is not free. Someone has to decide which branch a tool belongs to, and that decision is part of the version you publish. A tool filed under the wrong branch will be hard to find, exactly as a badly named function is hard to import.
The trade is worth it because the alternative fails in a way that is hard to debug. A flat list degrades gradually: quality drops, latency rises, and the cause is invisible in a single trace. A tree fails loudly: a request lands in the wrong branch and you can see it, fix the label and publish a new version.
Frequently asked questions
Does a tree change the tool schema?
No. Leaves are the same JSON Schema you already send to OpenAI or Anthropic. Optional hints such as aliases help the engine match vague language, but the schema is unchanged.
How many tools can one tree hold?
Thousands. The limit is on the working set per request, not on the tree. A large tree simply has more branches and deeper splits.
What happens when no branch fits?
The engine returns no tool call. It does not fall back to the closest leaf just to produce an answer. That case is covered in no tool call is a valid answer.