Glossary: NLP with Transformers
The time a model takes to answer one request, measured here in milliseconds per message.
ThroughputBatching