API layer
The web service in front of the model (FastAPI here). It checks who is asking and what they sent, enforces limits, logs the request and streams the answer back.
The web service in front of the model (FastAPI here). It checks who is asking and what they sent, enforces limits, logs the request and streams the answer back.