explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-25 17:36:30 +03:00
Code Issues Packages Projects Releases Wiki Activity

Fix JSON in tokenizer

Browse Source
This commit is contained in:
Matthew Honnibal 2016-10-21 01:44:20 +02:00
parent 6eb73a095f
commit 8ce8803824
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
spacy/tokenizer.pyx
View File

@ -43,7 +43,7 @@ cdef class Tokenizer:
path = pathlib.Path(path)
if rules is None:
with (path / 'tokenizer' / 'specials.json').open() as file_:
with (path / 'tokenizer' / 'specials.json').open('r', encoding='utf8') as file_:
rules = json.load(file_)
if prefix_search in (None, True):
with (path / 'tokenizer' / 'prefix.txt').open() as file_: