Glossary: Serving LLMs at Scale
Probes
Also written: startup probe, readiness probe, liveness probe
Kubernetes checks on a container. The startup probe allows time for loading, the readiness probe keeps traffic away until the server is ready, and the liveness probe restarts a pod that stops answering.