explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-27 02:16:32 +03:00
Code Issues Packages Projects Releases Wiki Activity
fef768ef74
spaCy/spacy/tests/tokenizer
History
Adriane Boyd 4f441dfa24
Fix infix as prefix in Tokenizer.explain (#10140) 
* Fix infix as prefix in Tokenizer.explain

Update `Tokenizer.explain` to align with the `Tokenizer` algorithm:

* skip infix matches that are prefixes in the current substring

* Update tokenizer pseudocode in docs
2022-01-28 17:00:54 +01:00
..
__init__.py Revert #4334 2019-09-29 17:32:12 +02:00
sun.txt Revert #4334 2019-09-29 17:32:12 +02:00
test_exceptions.py Ignore prefix in suffix matches (#9155) 2021-10-27 13:02:25 +02:00
test_explain.py Update Tokenizer.explain with special matches (#7749) 2021-04-19 19:08:20 +10:00
test_naughty_strings.py Merge branch 'develop' into master-tmp 2020-09-04 13:15:36 +02:00
test_tokenizer.py Fix infix as prefix in Tokenizer.explain (#10140) 2022-01-28 17:00:54 +01:00
test_urls.py Merge branch 'develop' into master-tmp 2020-06-20 15:52:00 +02:00
test_whitespace.py Merge branch 'develop' into master-tmp 2020-09-04 13:15:36 +02:00