Skip to content

Glossary: Running Open Models Locally

Inference

Also written: inference engine

Running a trained model to get output. The inference engine is the program that does it, such as llama.cpp.

Where the book teaches it

Related terms

llama.cpp