Count Tokens and Estimate Cost

Text usage has units

A token is a piece of text processed by a model. A word may be one token or several, and punctuation also consumes tokens. Input tokens include instructions, conversation history, and retrieved context. Output tokens are generated by the model.

Hosted providers often price input and output separately per million tokens. Estimated cost is (input_tokens input_price + output_tokens output_price) / 1,000,000. Store the price-table version because prices change. Local inference has no per-call provider bill, but the same estimate teaches budgeting and enables comparisons.

Production systems should use the tokenizer or token counts reported by the actual model provider. Counting whitespace-separated pieces is only a transparent approximation; counting characters is not a valid substitute because words have different lengths.

Scenario: A team sees its daily bill double. Traces show output tokens stayed stable while retrieved context tripled, revealing that input growth - not user traffic - caused the increase.

Goal: Instrument the lab service so every request records internally consistent input, output, total-token, and estimated-cost fields.