Skip to content

Glossary: Serving LLMs at Scale

N-gram speculation

Also written: N-gram, --spec-type ngram-simple

A drafting method that copies the text that followed the same few tokens earlier in the prompt or answer. It helps only when answers repeat their input word for word.

Where the book teaches it

Related terms

Speculative decodingDrafter