explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-06-29 17:33:10 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add tokenizer exception for 'Ph.D.', to fix 592.

Browse Source
This commit is contained in:
Matthew Honnibal 2016-11-03 00:03:34 +01:00
parent 532318e80b
commit 41a90a7fbb
1 changed files with 4 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
spacy/en/language_data.py
View File

@ -112,6 +112,10 @@ TOKENIZER_INFIXES = (r'''\.\.\.+ (?<=[a-z])\.(?=[A-Z]) (?<=[a-zA-Z])-(?=[a-zA-z]
TOKENIZER_EXCEPTIONS = { TOKENIZER_EXCEPTIONS = {
"Ph.D.": [
{
"F": "Ph.D."
}],
"d.": [ "d.": [
{ {
"F": "d." "F": "d."