Glossary: Serving LLMs at Scale
Prometheus
Also written: scrape, scrapes
The standard open-source monitoring system. It scrapes (fetches) each server's /metrics page every few seconds, stores time series and evaluates alert rules.
Glossary: Serving LLMs at Scale
Also written: scrape, scrapes
The standard open-source monitoring system. It scrapes (fetches) each server's /metrics page every few seconds, stores time series and evaluates alert rules.