Merge pull request #1079 from yuvalpinter/patch-1

extend suffixes from first-by-character-order to last
This commit is contained in:
Matthew Honnibal 2017-05-23 10:59:49 +02:00 committed by GitHub
commit b885ab405d

View File

@ -113,7 +113,7 @@ p
else:
tokens.append(substring)
substring = ''
tokens.extend(suffixes)
tokens.extend(reversed(suffixes))
return tokens
p