mirror of
https://github.com/explosion/spaCy.git
synced 2024-12-29 11:26:28 +03:00
43eedf73f2
Before this commit, the Mecab tokenizer had to be called twice when creating a Doc- once during tokenization and once during tagging. This creates a JapaneseDoc wrapper class for Doc that stashes the parsed tokenizer output to remove redundant processing. -POLM |
||
---|---|---|
.. | ||
__init__.py | ||
language_data.py | ||
stop_words.py | ||
tag_map.py |