Glossary: Serving LLMs at Scale
N-gram speculation
Also written: N-gram, --spec-type ngram-simple
A drafting method that copies the text that followed the same few tokens earlier in the prompt or answer. It helps only when answers repeat their input word for word.