Bluwing
Blog/Latency/What the fast path actually skips4 min left
Start free

On this page

01Two kinds of turn02What the fast path skips03The budget, stage by stage04Why not just use a smaller model?05When the engine slows down on purpose06Measuring it honestly07Frequently asked questions

Bluwing

HomeBlogWhy it's cheaperFeaturesHow it worksPricing
  1. Blog
  2. Latency
  3. 4 min read

What the fast path actually skips

The roughly 60 ms target is not a faster language model. It is a route that avoids calling one.

Bluwing teamOctober 12, 2026
Two lanes, the engine's fast path and the slow one it leaves behind.

On this page

  1. 01Two kinds of turn
  2. 02What the fast path skips
  3. 03The budget, stage by stage
  4. 04Why not just use a smaller model?
  5. 05When the engine slows down on purpose
  6. 06Measuring it honestly
  7. 07Frequently asked questions
Back to the top
Key takeaways
  • A 60 ms target is an engineering budget, not a benchmark
  • The fast path resolves common requests without a generation step
  • Slower work is reserved for requests that genuinely need it

Most latency advice for tool calling starts with the model: use a smaller one, cache the prompt, stream the response. All of that helps, and none of it gets you to tens of milliseconds. A generation step is measured in hundreds of milliseconds at best, and often in seconds.

Bluwing's fast path is fast because it does not generate. It resolves.

#Two kinds of turn

A request can be handled in one of two ways.

Resolve. The engine matches the request against the tree, checks that the required arguments are grounded, and emits the call. No model writes a sentence. The work is ranking, comparison and validation over short strings.

Reason. The request is ambiguous, spans several steps, or needs a value that is only available after another call. The engine falls back to a reasoning path, and that path has the latency you would expect.

The fast path is the common case by design. Bookings, lookups, status checks and simple sends are resolves. Multi-step tasks that need a plan are not, and pretending otherwise would trade correctness for a stopwatch.

#What the fast path skips

  • No generation of the tool name. The name comes from the leaf that won the ranking.
  • No generation of the arguments. Values are transcribed and normalised from their sources.
  • No full schema in context. The working set is twelve tools or fewer.
  • No round trip to an external model. The decision is made inside the engine.

What it does not skip is validation. A fast answer that calls the wrong tool is worse than a slow one that calls the right tool, so the checks run on every path.

Note

the 60 ms figure is a design target for the fast path, not a promise for every request. Reasoning turns take longer, and the response says which kind of turn it was.

#The budget, stage by stage

The target is a budget you can spend, not a single number to hit:

StageWhat it doesShare of the budget
NormaliseClean the message, split clausesSmall
NarrowWalk the tree to a branchLargest
GroundMatch values to argumentsMedium
ValidateCheck required fields and typesSmall
SerialiseBuild the tool_calls responseTiny

When a request misses the target, the table tells you where it went. That is more useful than a single end-to-end percentile, because it points at the stage that needs work.

#Why not just use a smaller model?

A small model is still a model. It still needs the tool definitions, still generates tokens, and still has a quality floor that drops as the list grows. You can get to a few hundred milliseconds that way, and a few hundred milliseconds is fine for a chat reply and slow for a booking button.

The other problem is determinism. A smaller model is still sampling. The same request can choose a different tool on a different day. The fast path is a lookup and a check, so the same request and the same version produce the same call.

#When the engine slows down on purpose

The fast path is not allowed to guess. If the ranking is close, or a required value is missing, or a clause needs another tool's result first, the engine takes the slower route. That is the correct trade: the user waits a little longer instead of getting a confident wrong answer.

The fast lane and the slow lane

#Measuring it honestly

Report the path, not just the average. A service that answers 90% of turns in 60 ms and 10% in 900 ms has a very different feel from one that answers everything in 150 ms, even though the averages are close.

Track at least:

  • the share of turns resolved on the fast path;
  • latency by path, not blended;
  • the reasons requests fall back to reasoning.

The last one is the most actionable. A spike in fallbacks usually means the tree labels drifted, not that the engine got slower.

#Frequently asked questions

Is 60 ms measured or targeted?

It is a design target for the fast path. Real numbers depend on region, tree size and request shape, and should be measured on your own workload.

Do reasoning turns cost more?

A turn costs one credit either way. The path affects latency, not price. See one credit per turn.

LatencyEngine
← PreviousNo tool call is a valid answerNext →Every argument has to come from somewhere
Point your client at Bluwing.Same tool definitions, same tool_calls. One credit per turn, from $4.50 a month.
Start free

Keep reading

EngineOct 284 min

Why a long tool list slows everything down

A flat list of two thousand tools asks one model to hold all of them in mind. Bluwing narrows a tree first, so the model sees twelve or fewer.Read →
GuaranteesOct 204 min

Every argument has to come from somewhere

Language models are good at plausible IDs and dates. Bluwing requires each argument value to appear in the request, the conversation, your facts or an earlier tool result.Read →
ProductOct 44 min

No tool call is a valid answer

When nothing fits, the most useful thing an engine can do is decline. Bluwing returns no tool call instead of reaching for the closest one.Read →
Start freeBluwing logoBluwing

Explore

  • Why it's cheaper
  • Features
  • How it works
  • Pricing
  • Documentation
  • API Reference
  • Comparisons
  • Blog & Guides
  • Privacy Policy
  • Terms of Service
  • System Status

Talk to us

© 2026 Bluwing · bluwing.si. The tool-calling engine for AI agents.

  • Blog→
  • Product Hunt
  • LinkedIn