Latency

The time between a request and the model's response. Locally it depends mostly on generation speed (tokens per second), which in turn depends on how large the weights that get re-read every token are: below ~15 tokens/s, interactive use starts to feel slow, whatever the answer quality.