explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-25 17:36:30 +03:00
Code Issues Packages Projects Releases Wiki Activity
2a558a7cdc
spaCy/spacy
History
Adriane Boyd 2a558a7cdc
Switch to mecab-ko as default Korean tokenizer (#11294) 
* Switch to mecab-ko as default Korean tokenizer

Switch to the (confusingly-named) mecab-ko python module for default Korean
tokenization.

Maintain the previous `natto-py` tokenizer as
`spacy.KoreanNattoTokenizer.v1`.

* Temporarily run tests with mecab-ko tokenizer

* Fix types

* Fix duplicate test names

* Update requirements test

* Revert "Temporarily run tests with mecab-ko tokenizer"

This reverts commit d2083e7044.

* Add mecab_args setting, fix pickle for KoreanNattoTokenizer

* Fix length check

* Update docs

* Formatting

* Update natto-py error message

Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com>

Co-authored-by: Paul O'Leary McCann <polm@dampfkraft.com>
2022-08-26 10:11:18 +02:00
..
cli Support full prerelease versions in the compat table (#11228) 2022-08-04 15:14:19 +02:00
displacy Docs: displaCy documentation - data types, parse_{deps,ents,spans}, spans example (#10950) 2022-08-16 11:23:34 -04:00
lang Switch to mecab-ko as default Korean tokenizer (#11294) 2022-08-26 10:11:18 +02:00
matcher Merge remote-tracking branch 'upstream/master' into merge-master-v4-20220728 2022-07-28 13:53:59 +02:00
ml precompute_hiddens/Parser: look up CPU ops once (v4) (#11068) 2022-07-29 15:12:19 +02:00
pipeline Rename modules for consistency (#11286) 2022-08-10 11:44:05 +02:00
tests Switch to mecab-ko as default Korean tokenizer (#11294) 2022-08-26 10:11:18 +02:00
tokens Merge remote-tracking branch 'upstream/master' into chore/update-v4-from-master 2022-08-24 07:15:41 +02:00
training Clean up warnings in the test suite (#11331) 2022-08-22 12:04:30 +02:00
__init__.pxd * Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags. 2014-10-24 02:23:42 +11:00
__init__.py enable argument for spacy.load() (#10784) 2022-06-17 20:24:13 +01:00
__main__.py Tidy up 2020-06-22 00:45:40 +02:00
about.py Set version to v3.4.1 (#11209) 2022-07-26 12:52:38 +02:00
attrs.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
attrs.pyx Remove intify_attrs(_do_deprecated) (#11319) 2022-08-17 12:13:54 +02:00
compat.py Custom component types in spacy.ty (#9469) 2021-10-21 15:31:06 +02:00
default_config_pretraining.cfg Add new parameter for saving every n epoch in pretraining (#8912) 2021-08-12 11:14:48 +02:00
default_config.cfg Add a few docs to the default_config.cfg (#9981) 2022-01-05 09:16:40 +01:00
errors.py Support custom attributes for tokens and spans in json conversion (#11125) 2022-08-23 10:05:02 +02:00
glossary.py Add glossary entry for root (#10821) 2022-05-20 09:56:32 +02:00
kb.pxd Replace cpdef variables with cdef (#7834) 2021-04-26 16:54:02 +02:00
kb.pyx Made _initialize_X() methods private. (#10978) 2022-06-17 15:55:34 +02:00
language.py Check for . in factory names (#11336) 2022-08-19 09:52:12 +02:00
lexeme.pxd Fix Lexeme.from_ptr 2020-08-10 16:43:37 +02:00
lexeme.pyi fix type of lexeme.rank (#9979) 2022-01-04 13:15:25 +01:00
lexeme.pyx Bugfix for similarity return types (#10051) 2022-01-20 11:40:46 +01:00
lookups.py Fix issues for Mypy 0.950 and Pydantic 1.9.0 (#10786) 2022-05-25 09:33:54 +02:00
morphology.pxd Morphology/Morphologizer optimizations and refactoring (#11024) 2022-07-15 11:14:08 +02:00
morphology.pyx Morphology/Morphologizer optimizations and refactoring (#11024) 2022-07-15 11:14:08 +02:00
parts_of_speech.pxd Add support for Universal Dependencies v2.0 2017-03-03 13:17:34 +01:00
parts_of_speech.pyx Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
pipe_analysis.py 🏷 Add Mypy check to CI and ignore all existing Mypy errors (#9167) 2021-10-14 15:21:40 +02:00
py.typed Add py.typed 2021-03-16 09:48:31 +01:00
schemas.py Support custom attributes for tokens and spans in json conversion (#11125) 2022-08-23 10:05:02 +02:00
scorer.py Alignment: use a simplified ragged type for performance (#10319) 2022-04-01 09:02:06 +02:00
strings.pxd StringStore-related optimizations (#10938) 2022-07-04 15:04:03 +02:00
strings.pyi Fix StringStore.__getitem__ return type depending on parameter types (#10741) 2022-05-03 17:57:07 +02:00
strings.pyx StringStore-related optimizations (#10938) 2022-07-04 15:04:03 +02:00
structs.pxd Morphology/Morphologizer optimizations and refactoring (#11024) 2022-07-15 11:14:08 +02:00
symbols.pxd introduce token.has_head and refer to MISSING_DEP_ (WIP) 2021-01-12 17:17:06 +01:00
symbols.pyx introduce token.has_head and refer to MISSING_DEP_ (WIP) 2021-01-12 17:17:06 +01:00
tokenizer.pxd Cleanup Cython structs (#11337) 2022-08-22 15:52:24 +02:00
tokenizer.pyx Cleanup Cython structs (#11337) 2022-08-22 15:52:24 +02:00
ty.py Custom component types in spacy.ty (#9469) 2021-10-21 15:31:06 +02:00
typedefs.pxd Merge remote-tracking branch 'upstream/master' into chore/update-develop-from-master 2020-11-25 11:49:34 +01:00
typedefs.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
util.py Support full prerelease versions in the compat table (#11228) 2022-08-04 15:14:19 +02:00
vectors.pyx vectors: avoid expensive comparisons between numpy ints and Python ints (#10992) 2022-06-29 12:58:31 +02:00
vocab.pxd Cleanup Cython structs (#11337) 2022-08-22 15:52:24 +02:00
vocab.pyi Cleanup Cython structs (#11337) 2022-08-22 15:52:24 +02:00
vocab.pyx Cleanup Cython structs (#11337) 2022-08-22 15:52:24 +02:00