Skip to content

Glossary: LLMs in Production

Streaming

Also written: stream, on_text

Receiving a model's answer piece by piece as it is written. It doesn't change the total time but changes what users feel, and the gateway always streams from the provider.

Where the book teaches it

Related terms

Server-sent events (SSE)Time to first token (TTFT)