explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-29 11:26:28 +03:00
Code Issues Packages Projects Releases Wiki Activity
16,086 Commits 70 Branches 177 Tags 279 MiB
f293386d3e
Commit Graph

2 Commits

Author SHA1 Message Date
Daniël de Kok
2468742cb8 isort all the things 2023-06-26 11:41:03 +02:00
Adriane Boyd
1d59fdbd39
Update Vietnamese tokenizer (#8099) 
* Adapt tokenization methods from `pyvi` to preserve text encoding and
whitespace
* Add serialization support similar to Chinese and Japanese

Note: as for Chinese and Japanese, some settings are duplicated in
`config.cfg` and `tokenizer/cfg`.
 2021-05-17 18:16:20 +10:00