Glossary: Running Open Models Locally
Slots
Also written: -np, n_slots
The number of requests llama-server can work on at once, set with `-np`. The context is shared between slots, so give each enough.
Glossary: Running Open Models Locally
Also written: -np, n_slots
The number of requests llama-server can work on at once, set with `-np`. The context is shared between slots, so give each enough.