spaCy/website/usage/_spacy-101/_similarity.jade

49 lines
2.2 KiB
Plaintext
Raw Normal View History

2017-05-24 00:16:31 +03:00
//- 💫 DOCS > USAGE > SPACY 101 > SIMILARITY
p
| spaCy is able to compare two objects, and make a prediction of
| #[strong how similar they are]. Predicting similarity is useful for
| building recommendation systems or flagging duplicates. For example, you
| can suggest a user content that's similar to what they're currently
2017-05-28 02:30:12 +03:00
| looking at, or label a support ticket as a duplicate if it's very
2017-05-24 00:16:31 +03:00
| similar to an already existing one.
p
| Each #[code Doc], #[code Span] and #[code Token] comes with a
| #[+api("token#similarity") #[code .similarity()]] method that lets you
| compare it with another object, and determine the similarity. Of course
| similarity is always subjective whether "dog" and "cat" are similar
| really depends on how you're looking at it. spaCy's similarity model
| usually assumes a pretty general-purpose definition of similarity.
+code.
tokens = nlp(u'dog cat banana')
for token1 in tokens:
for token2 in tokens:
print(token1.similarity(token2))
+aside
| #[strong #[+procon("neutral", "identical", false, 16)] similarity:] identical#[br]
| #[strong #[+procon("yes", "similar", false, 16)] similarity:] similar (higher is more similar) #[br]
| #[strong #[+procon("no", "dissimilar", false, 16)] similarity:] dissimilar (lower is less similar)
2017-05-24 00:16:31 +03:00
+table
+row("head")
for column in ["", "dog", "cat", "banana"]
+head-cell.u-text-center=column
each cells, label in {"dog": [1, 0.8, 0.24], "cat": [0.8, 1, 0.28], "banana": [0.24, 0.28, 1]}
2017-05-24 00:16:31 +03:00
+row
+cell.u-text-label.u-color-theme=label
for cell in cells
+cell.u-text-center
- var result = cell < 0.5 ? ["yes", "similar"] : cell != 1 ? ["no", "dissimilar"] : ["neutral", "identical"]
| #[code=cell.toFixed(2)] #[+procon(...result)]
2017-05-24 00:16:31 +03:00
p
| In this case, the model's predictions are pretty on point. A dog is very
| similar to a cat, whereas a banana is not very similar to either of them.
| Identical tokens are obviously 100% similar to each other (just not always
| exactly #[code 1.0], because of vector math and floating point
| imprecisions).