Glossary: Embeddings and Vector Search
truncation
Also written: truncated
Silently ignoring text past a model's token limit. all-MiniLM-L6-v2 stops at 256 tokens, so two texts that differ only at the end can get the same vector.
Glossary: Embeddings and Vector Search
Also written: truncated
Silently ignoring text past a model's token limit. all-MiniLM-L6-v2 stops at 256 tokens, so two texts that differ only at the end can get the same vector.