Document Similarity Metrics: Jaccard, Cosine, and Hamming Calculations
Posted by Anonymous and classified in Mathematics
Written on in 
English with a size of 128 KB
Document Similarity Metrics: Jaccard, Cosine, and Hamming Calculations (Q99)
Documents (after lowercasing and tokenizing words, removing punctuation):
- D1: “the night is dark and the moon is red”
- D2: “the moon in the night is red”
- D3: “i can see moon is red the night is dark”
Step A — Construct Word Sets / Vectors (Unified Vocabulary)
Vocabulary (unique words across D1–D3): {the, night, is, dark, and, moon, red, in, i, can, see} (11 words)
i) Jaccard Similarity (Set of Words)
Set(D1) = {the, night, is, dark, and, moon, red}
Set(D2) = {the, moon, in, night, is, red}
Set(D3) = {i, can, see, moon, is, red, the, night, dark}
Compute pairwise Jaccard:
- D1 ∩ D2 = {the, night, is, moon, red} → size 5; D1 ∪ D2 size = 8 → J(D1,D2)=5/8=0.625
- D1