explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-25 01:16:28 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update base norm exceptions with more unicode characters

Browse Source 
e.g. unicode variations of punctuation used in Chinese
This commit is contained in:
ines 2017-10-14 14:58:52 +02:00
parent 3516aa0cea
commit 9d6c8eaa49
1 changed files with 10 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
spacy/lang/norm_exceptions.py
View File

@ -31,11 +31,21 @@ BASE_NORMS = {
"„": '"',
"»": '"',
"«": '"',
"‘‘": '"',
"’’": '"',
"？": "?",
"！": "!",
"，": ",",
"；": ";",
"：": ":",
"。": ".",
"।": ".",
"…": "...",
"—": "-",
"–": "-",
"--": "-",
"---": "-",
"——": "-",
"€": "$",
"£": "$",
"¥": "$",