spaCy/spacy/lang/ja
adrianeboyd 3bf111585d
Update Japanese tokenizer config and add serialization (#5562)
* Use `config` dict for tokenizer settings
* Add serialization of split mode setting
* Add tests for tokenizer split modes and serialization of split mode
setting

Based on #5561
2020-06-08 16:29:05 +02:00
..
__init__.py Update Japanese tokenizer config and add serialization (#5562) 2020-06-08 16:29:05 +02:00
bunsetu.py Add Japanese Model (#5544) 2020-06-04 19:15:43 +02:00
examples.py 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
stop_words.py 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
syntax_iterators.py Add Japanese Model (#5544) 2020-06-04 19:15:43 +02:00
tag_bigram_map.py Add Japanese Model (#5544) 2020-06-04 19:15:43 +02:00
tag_map.py Add Japanese Model (#5544) 2020-06-04 19:15:43 +02:00
tag_orth_map.py Add Japanese Model (#5544) 2020-06-04 19:15:43 +02:00