Cross-entropy loss
Also written: cross-entropy, negative log-likelihood, NLL
The average of minus the log of the probability the model gave to the true next token. Lower is better; blind guessing among 72 characters scores ln 72 = 4.277.
Also written: cross-entropy, negative log-likelihood, NLL
The average of minus the log of the probability the model gave to the true next token. Lower is better; blind guessing among 72 characters scores ln 72 = 4.277.