adrianeboyd f162815f45 Handle empty and whitespace-only docs for Japanese (#5564) ... Handle empty and whitespace-only docs in the custom alignment method used by the Japanese tokenizer.		2020-06-08 21:09:23 +02:00
..
__init__.py	Revert #4334	2019-09-29 17:32:12 +02:00
test_lemmatization.py	Add Japanese Model (#5544)	2020-06-04 19:15:43 +02:00
test_serialize.py	Update Japanese tokenizer config and add serialization (#5562)	2020-06-08 16:29:05 +02:00
test_tokenizer.py	Handle empty and whitespace-only docs for Japanese (#5564)	2020-06-08 21:09:23 +02:00