Jaccard similarity
A measure of how alike two texts are: the number of words they share divided by the number of distinct words in both. It can be used to drop near-duplicate examples, such as emails with a similarity over 0.9 to an earlier one.
A measure of how alike two texts are: the number of words they share divided by the number of distinct words in both. It can be used to drop near-duplicate examples, such as emails with a similarity over 0.9 to an earlier one.