Glossary: Serving LLMs at Scale

Thread

Also written: thread pool, ThreadPoolExecutor

A separate line of work in the same program. A thread pool runs a function several times in parallel.