Glossary: Running Open Models Locally

GPU offloading (-ngl)

Also written: -ngl, layers on GPU

The llama.cpp setting for how many of a model's layers are placed on the GPU: `all`, a number, or `auto` (the default). Lower it if the GPU runs out of memory.