Local AI Toolkit

Understanding token estimates

Русская версия

The useful question is: how much context would the cloud agent need for this request? The size of every file opened locally is not that amount. A local tool may inspect a large file to return three relevant lines; charging the entire file to the cloud alternative would exaggerate the estimate.

A worked example

For the runtime text comparison, let S be the explicitly declared necessary source input not already seen in the cloud, V the complete visible tool response, and R the tool request. The conservative comparison is S - V - 2R. The request is generated by the agent and subsequently appears as context. Local answer generation is not automatically credited as generation the cloud would otherwise perform.

Illustrative input Tokens
Necessary source context, S 4,000
Entire returned response, V 600
Request, R 100
Conditional estimated saving 3,200

These numbers are an example, not a benchmark or a promise. The interactive website lets you change them without submitting anything. A caller-provided task context is an explicit assumption, not proof that it is the smallest possible context.

Read the notice

Notice Meaning
Approximately N cloud tokens saved Positive conditional estimate under the recorded scenario
No savings; approximately N additional cloud tokens Request and response cost exceed the declared alternative
Cannot estimate Necessary context, provenance, overlap or returned size is not sufficiently known
Not applicable Administrative operation, not model delegation

AlreadyPresent excludes previously read source credit. Unknown is appropriate when the agent cannot establish what context was already present. Files are not credited merely because their paths were supplied; an explicit task-input comparison basis is required. Retrieval alone cannot establish how much a cloud agent would need to read.

Custom log-triage questions use an unknown task-context baseline (log_question_context_unknown) while preserving the caller’s sourceContext. Default full-log triage retains its declared whole-source baseline.

Practical decisions

  1. Delegate first reading of an unread, bounded task; send paths rather than copied files.
  2. Ask for a concise result with evidence locations.
  3. Read exact source fragments only where consequential verification or a safe edit needs them.
  4. Skip redundant local rereading when the source is already in cloud context and the remaining work cannot cover the request/response overhead.

This is not a meter of billed tokens, subscription limits, money or energy. Historical reports from a different runtime methodology are not interchangeable with this comparison. Check the installed runtime version when comparing a report with current source documentation.