Skip to content

Glossary: Serving LLMs at Scale

Probes

Also written: startup probe, readiness probe, liveness probe

Kubernetes checks on a container. The startup probe allows time for loading, the readiness probe keeps traffic away until the server is ready, and the liveness probe restarts a pod that stops answering.

Where the book teaches it

Related terms

KubernetesHealth checkCold start