mirror of
https://github.com/explosion/spaCy.git
synced 2024-12-24 17:06:29 +03:00
Fix JSON segmentation bug that affected French
Fix a bug in the JSON streaming code that GoldCorpus uses. Escaped slashes were being handled incorrectly. This bug caused low scores for French in the early v2.1.0 alphas, because most of the data was not being read in. Fittingly, the document that triggered the bug was a Wikipedia article about Perl. Parsing perl remains difficult!
This commit is contained in:
parent
6f36b6bc4e
commit
a338c6f8f6
|
@ -346,12 +346,12 @@ def _json_iterate(loc):
|
|||
cdef char close_curly = ord('}')
|
||||
for i in range(len(py_raw)):
|
||||
c = raw[i]
|
||||
if c == backslash:
|
||||
escape = True
|
||||
continue
|
||||
if escape:
|
||||
escape = False
|
||||
continue
|
||||
if c == backslash:
|
||||
escape = True
|
||||
continue
|
||||
if c == quote:
|
||||
inside_string = not inside_string
|
||||
continue
|
||||
|
|
Loading…
Reference in New Issue
Block a user