Measure a Real Ollama Stream

Observe the first response event

Ollama's local generate API can return newline-delimited streaming JSON. Start the clock before the HTTP request, read response lines incrementally, and capture TTFT when the first event containing response text arrives. Stop the total timer when the event with done: true arrives.

Run more than once because the first request may load the model into memory. Report individual runs and a median - the middle value - which is less distorted by one slow outlier. Do not invent a universal pass threshold across different CPUs.

A non-streaming request returns only after the full answer is ready, so it cannot demonstrate interactive TTFT correctly. Buffering in an application proxy can similarly defeat streaming even when the model streams internally.

Tip: Benchmark through the same wrapper and network path users call. A direct model benchmark and an end-to-end application benchmark answer different questions.

Goal: Run two real requests against the pre-baked Qwen2.5 0.5B model and verify each first-token time is below its total time.