explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-02-05 14:10:34 +03:00
Code Issues Packages Projects Releases Wiki Activity

* Use Jieba tokenizer in Chinese class

Browse Source
This commit is contained in:
Matthew Honnibal 2016-04-24 19:11:38 +02:00
parent 9bfe20cac9
commit b6ccd8d76a
1 changed files with 8 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
spacy/zh/__init__.py
View File

@ -1,5 +1,13 @@
from ..language import Language from ..language import Language
from .jieba import JiebaTokenizer
class Chinese(Language): class Chinese(Language):
lang = u'zh' lang = u'zh'
@classmethod
def default_tokenizer(cls, package, vocab):
'''Return Jieba-wrapper tokenizer.'''
return JiebaTokenizer.from_package(package, vocab)