Tokens per second and time to first token, explained
Updated
Tokens per second (TPS) is how many output tokens a model generates each second once it starts writing, and time to first token (TTFT) is the wait between sending a request and receiving the first token.
Together they set how long a response takes: total time is about TTFT plus output tokens divided by TPS. For a chat, TTFT is what feels slow. For a coding agent writing a 2,000-token file, TPS matters more.
What each number measures
| Metric | Measures | Driven by |
|---|---|---|
| Time to first token | Wait before the first output token | Prompt length (the model reads it all first), cache hits, reasoning before visible text, network |
| Tokens per second | Output speed after the first token | Model size and architecture, serving hardware |
Reading the prompt (prefill) happens in parallel, so it is fast per token, but a 200,000-token prompt still takes time. Writing (decode) produces one token per step. A cached prefix skips part of the prefill, so caching shortens TTFT as well as cutting cost.
Real numbers
| Model | Tokens per second | 500 tokens | 2,000 tokens |
|---|---|---|---|
| DeepSeek V4.1 Flash | 200+ | 2.5 s | 10 s |
| GLM 5.3 Flash | 100+ | 5 s | 20 s |
| DeepSeek V4.1 Flash (fast) | 250+ | 2 s | 8 s |
The times are decode only, at the listed floor, so a real response is that plus TTFT. DeepSeek V4.1 Flash (fast) is the same model on faster serving, with about twice the output speed and a shorter wait for the first token, at $0.28 input and $1.10 output per 1M tokens.
Reasoning and perceived speed
A reasoning model thinks before it answers. If your client shows only the answer, the thinking time looks like a long TTFT, even though the model is generating at full speed. Lower the reasoning effort, or turn reasoning off where the model allows it, for tasks that do not need it. On OpenScale, DeepSeek V4.1 Flash takes low, high, max effort and GLM 5.3 Flash takes low, high, max effort. The API reference lists which models can turn reasoning off.
How to measure it yourself
- Stream the response (
"stream": true) and record the time of the first content or reasoning chunk: that is TTFT. - Divide the output tokens (
usage.completion_tokens) by the seconds between the first and last chunk: that is TPS. - Run at least 20 requests with your real prompt sizes and look at the median, not one run.
Questions
What is a good tokens per second?
It depends on the job. For chat, the answer only has to keep ahead of a reader. For an agent writing whole files, more is better: at 100 tokens per second, a 2,000-token file takes 20 s to write.
Does a longer prompt slow down output?
It mainly lengthens time to first token, because the model reads the whole prompt first. Output speed changes less. Caching a long, stable prefix shortens the wait.