Context length (context window)
Also written: context length, context window, context
The most tokens a model can look at when predicting the next one. Your tiny GPT has a context of 64 characters and GPT-2 has 1,024 tokens.
Also written: context length, context window, context
The most tokens a model can look at when predicting the next one. Your tiny GPT has a context of 64 characters and GPT-2 has 1,024 tokens.