Memory bandwidth

Also written: bandwidth

How many gigabytes per second memory can deliver. Since every generated token reads all the weights once, it caps writing speed at bandwidth divided by weight size.