Glossary: Running Open Models Locally
Streaming
Also written: stream, stream=True
Sending the reply piece by piece as it's written, so users see the first words straight away. It doesn't make the model faster.
Glossary: Running Open Models Locally
Also written: stream, stream=True
Sending the reply piece by piece as it's written, so users see the first words straight away. It doesn't make the model faster.