Adriane Boyd 2a558a7cdc Switch to mecab-ko as default Korean tokenizer (#11294) ... * Switch to mecab-ko as default Korean tokenizer Switch to the (confusingly-named) mecab-ko python module for default Korean tokenization. Maintain the previous `natto-py` tokenizer as `spacy.KoreanNattoTokenizer.v1`. * Temporarily run tests with mecab-ko tokenizer * Fix types * Fix duplicate test names * Update requirements test * Revert "Temporarily run tests with mecab-ko tokenizer" This reverts commit d2083e7044. * Add mecab_args setting, fix pickle for KoreanNattoTokenizer * Fix length check * Update docs * Formatting * Update natto-py error message Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com> Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com>		2022-08-26 10:11:18 +02:00
..
__init__.py	Sync thinc install dep in setup, fix test packaging (#9336)	2021-09-30 19:02:10 +02:00
test_requirements.py	Switch to mecab-ko as default Korean tokenizer (#11294)	2022-08-26 10:11:18 +02:00