Bluwing
Blog/Engine/Why a long tool list slows everything down4 min left
Start free

On this page

01A flat list makes the model do the indexing02Narrow first, then decide03Twelve is a budget, not a magic number04What this looks like at the API05The trade you are making06Frequently asked questions

Bluwing

HomeBlogWhy it's cheaperFeaturesHow it worksPricing
  1. Blog
  2. Engine
  3. 4 min read

Why a long tool list slows everything down

A flat list of two thousand tools asks one model to hold all of them in mind. Bluwing narrows a tree first, so the model sees twelve or fewer.

Bluwing teamOctober 28, 2026
A request branching down a tree until a handful of tools remain.

On this page

  1. 01A flat list makes the model do the indexing
  2. 02Narrow first, then decide
  3. 03Twelve is a budget, not a magic number
  4. 04What this looks like at the API
  5. 05The trade you are making
  6. 06Frequently asked questions
Back to the top
Key takeaways
  • Long tool lists cost quality as well as time
  • Bluwing narrows a tree before any reasoning happens
  • The working set per request is twelve tools or fewer

The first version of most tool-calling systems is a flat list. You define search, read, send, book, refund, and twenty more, paste them into the prompt, and let the model choose. It works at ten tools. At a hundred it starts missing obvious matches. At a thousand it gets slower, more expensive and less accurate at the same time.

That is not a prompt-engineering problem. It is a search problem.

#A flat list makes the model do the indexing

Every tool definition is context. The model has to read all of them before it can decide that book_table is the one it wants. For a short list that is cheap. For a long list it means thousands of tokens describing tools that have nothing to do with the request, and the relevant definition is buried somewhere in the middle.

The effect compounds: more context means more time per turn, more tokens billed, and a higher chance that a plausible-sounding tool wins over a correct one. The model is not failing because it is weak. It is failing because we asked it to search before it could reason.

#Narrow first, then decide

Bluwing treats a tool set as a tree. Branches describe areas of work, leaves are the tools themselves. A request enters at the root, and each step only compares the request against the children of the branch it is already in.

  • A request about a table booking goes to the dining branch, not the payments branch.
  • The dining branch holds a dozen tools, not two thousand.
  • The engine reasons over that small group, then returns the call.

The narrowing is a ranking step over short labels, not a generation step over full schemas. It is why the engine can afford to look at the whole tree on every turn without paying for it in context.

Rule

the model should never see a tool it has no realistic chance of calling.

#Twelve is a budget, not a magic number

We target twelve or fewer tools in the working set. That number is a design budget: small enough that the definitions fit comfortably in a prompt, large enough that a real branch is still useful. If a branch grows past it, the branch splits. If two leaves describe the same action, they merge.

The budget changes what you do with a large integration. Instead of trimming features to keep the list short, you group them. One billing branch can hold invoices, refunds, subscriptions and payment methods without making the invoice request read about subscriptions.

#What this looks like at the API

Nothing changes in the shape of the response. You still get standard tool_calls, and your app still runs the function. The difference is the set of definitions the engine considered:

json
{
  "finish_reason": "tool_calls",
  "tool_calls": [{
    "id": "call_8sKf2LmQ0aPz",
    "type": "function",
    "function": {
      "name": "book_table",
      "arguments": "{\"restaurant\":\"Nok\",\"party_size\":4,\"datetime\":\"2026-11-05T19:00\"}"
    }
  }]
}

The response does not tell you how many tools were skipped. It does not need to. What matters is that book_table was in the final group, and that every value in the arguments came from the sentence.

#The trade you are making

A tree is not free. Someone has to decide which branch a tool belongs to, and that decision is part of the version you publish. A tool filed under the wrong branch will be hard to find, exactly as a badly named function is hard to import.

The trade is worth it because the alternative fails in a way that is hard to debug. A flat list degrades gradually: quality drops, latency rises, and the cause is invisible in a single trace. A tree fails loudly: a request lands in the wrong branch and you can see it, fix the label and publish a new version.

A request narrowing through the tree

#Frequently asked questions

Does a tree change the tool schema?

No. Leaves are the same JSON Schema you already send to OpenAI or Anthropic. Optional hints such as aliases help the engine match vague language, but the schema is unchanged.

How many tools can one tree hold?

Thousands. The limit is on the working set per request, not on the tree. A large tree simply has more branches and deeper splits.

What happens when no branch fits?

The engine returns no tool call. It does not fall back to the closest leaf just to produce an answer. That case is covered in no tool call is a valid answer.

EngineTool design
← PreviousEvery argument has to come from somewhere
Point your client at Bluwing.Same tool definitions, same tool_calls. One credit per turn, from $4.50 a month.
Start free

Keep reading

LatencyOct 124 min

What the fast path actually skips

The roughly 60 ms target is not a faster language model. It is a route that avoids calling one.Read →
GuaranteesOct 204 min

Every argument has to come from somewhere

Language models are good at plausible IDs and dates. Bluwing requires each argument value to appear in the request, the conversation, your facts or an earlier tool result.Read →
ProductOct 44 min

No tool call is a valid answer

When nothing fits, the most useful thing an engine can do is decline. Bluwing returns no tool call instead of reaching for the closest one.Read →
Start freeBluwing logoBluwing

Explore

  • Why it's cheaper
  • Features
  • How it works
  • Pricing
  • Documentation
  • API Reference
  • Comparisons
  • Blog & Guides
  • Privacy Policy
  • Terms of Service
  • System Status

Talk to us

© 2026 Bluwing · bluwing.si. The tool-calling engine for AI agents.

  • Blog→
  • Product Hunt
  • LinkedIn