UTF-8
Also written: bytes
The standard way to store text as bytes (numbers 0 to 255): English letters take one byte and other characters two to four. Byte-level BPE builds on it.
Also written: bytes
The standard way to store text as bytes (numbers 0 to 255): English letters take one byte and other characters two to four. Byte-level BPE builds on it.