Glossary: Running Open Models Locally
Prefill
Also written: prompt processing
The phase where the model reads the whole prompt before writing the first token. It is limited by computing power, where GPUs are far faster than CPUs.
Glossary: Running Open Models Locally
Also written: prompt processing
The phase where the model reads the whole prompt before writing the first token. It is limited by computing power, where GPUs are far faster than CPUs.