Glossary: LLMs in Production

Latency

How long something takes. For an LLM answer it is time to first token plus output tokens divided by output speed, plus your own code.