Glossary: NLP with Transformers

Weight decay

Also written: weight_decay, AdamW

A small penalty on large weights, used here with AdamW, the default optimizer in `Trainer`.