Glossary: Running Open Models Locally
llama bench
Also written: llama-bench
A llama.cpp command that measures model files directly, without a server. It reports prompt processing (pp256) and text generation (tg32) in tokens per second.
Glossary: Running Open Models Locally
Also written: llama-bench
A llama.cpp command that measures model files directly, without a server. It reports prompt processing (pp256) and text generation (tg32) in tokens per second.