mirror of
https://github.com/explosion/spaCy.git
synced 2025-03-14 15:12:15 +03:00
Before this commit, the Mecab tokenizer had to be called twice when creating a Doc- once during tokenization and once during tagging. This creates a JapaneseDoc wrapper class for Doc that stashes the parsed tokenizer output to remove redundant processing. -POLM |
||
---|---|---|
.. | ||
__init__.py | ||
language_data.py | ||
stop_words.py | ||
tag_map.py |