Reorganise English tokenizer exceptions (as discussed in #718)

Browse Source

Add logic to generate exceptions that follow a consistent pattern (like
verbs and pronouns) and allow certain tokens to be excluded explicitly.

...

This commit is contained in:

Ines Montani 2017-01-03 18:17:57 +01:00

parent 1b82756cc7

commit b19cfcc144

1 changed files with 416 additions and 1781 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2197

spacy/en/tokenizer_exceptions.py

View File

File diff suppressed because it is too large Load Diff