Batch size

Also written: per_device_train_batch_size, batch

The number of examples processed together in one forward and backward pass. Larger batches need more memory, so a batch of 1 combined with gradient accumulation is a way to train on limited hardware.