Reorganise English tokenizer exceptions (as discussed in #718)

Browse Source

Add logic to generate exceptions that follow a consistent pattern (like
verbs and pronouns) and allow certain tokens to be excluded explicitly.

...

This commit is contained in:

Ines Montani 2017-01-03 18:26:09 +01:00

parent fb9d3bb022

commit 35b39f53c3

1 changed files with 416 additions and 1781 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2197

spacy/en/tokenizer_exceptions.py

View File

File diff suppressed because it is too large Load Diff