Skip to content

Glossary: Running Open Models Locally

Streaming

Also written: stream, stream=True

Sending the reply piece by piece as it's written, so users see the first words straight away. It doesn't make the model faster.

Where the book teaches it

Related terms

ChunkTime to first token (TTFT)