explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-11-11 12:18:04 +03:00
Code Issues Packages Projects Releases Wiki Activity
11,334 Commits 68 Branches 175 Tags 279 MiB
a6e521cd79
Commit Graph

2 Commits

Author SHA1 Message Date
Ines Montani
5e9849b60f Auto-format [ci skip] 2019-10-30 19:27:18 +01:00
adrianeboyd
f2bfaa1b38 Filter subtoken matches in merge_subtokens() (#4539) 
The `Matcher` in `merge_subtokens()` returns all possible subsequences
of `subtok`, so for sequences of two or more subtoks it's necessary to
filter the matches so that the retokenizer is only merging the longest
matches with no overlapping spans.
 2019-10-28 15:40:28 +01:00