spaCy/spacy
Adriane Boyd 11e195d3ed Update ChineseTokenizer
* Allow `pkuseg_model` to be set to `None` on initialization
* Don't save config within tokenizer
* Force convert pkuseg_model to use pickle protocol 4 by reencoding with
`pickle5` on serialization
* Update pkuseg serialization test
2020-09-27 14:00:18 +02:00
..
cli Merge branch 'develop' into fix/train-config-interpolation 2020-09-26 15:32:14 +02:00
displacy Refactor Docs.is_ flags (#6044) 2020-09-17 00:14:01 +02:00
lang Update ChineseTokenizer 2020-09-27 14:00:18 +02:00
matcher Add MORPH handling to Matcher (#6107) 2020-09-24 16:55:09 +02:00
ml state_type as Literal 2020-09-23 17:32:14 +02:00
pipeline Simplify string match IDs for AttributeRuler 2020-09-26 11:12:39 +02:00
tests Update ChineseTokenizer 2020-09-27 14:00:18 +02:00
tokens Make DocBin output stable data (set iteration) 2020-09-25 22:20:44 +02:00
training Fix iteration order 2020-09-25 21:47:22 +02:00
__init__.pxd * Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags. 2014-10-24 02:23:42 +11:00
__init__.py Support vocab arg in spacy.blank 2020-09-15 11:39:36 +02:00
__main__.py Tidy up 2020-06-22 00:45:40 +02:00
about.py Fix pretraining in train script (#6143) 2020-09-25 15:47:10 +02:00
attrs.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
attrs.pyx Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
compat.py Use Literal type for nr_feature_tokens 2020-09-23 16:00:03 +02:00
default_config_pretraining.cfg generalize corpora, dot notation for dev and train corpus 2020-09-17 11:38:59 +02:00
default_config.cfg Add [training.before_to_disk] callback 2020-09-24 12:40:25 +02:00
errors.py Update ChineseTokenizer 2020-09-27 14:00:18 +02:00
glossary.py unicode -> str consistency 2020-05-24 17:20:58 +02:00
kb.pxd fix KB IO (#6118) 2020-09-22 21:53:06 +02:00
kb.pyx updates to NEL functionality (#6132) 2020-09-24 16:53:59 +02:00
language.py Fix combined scores and update test 2020-09-24 10:42:47 +02:00
lexeme.pxd Fix Lexeme.from_ptr 2020-08-10 16:43:37 +02:00
lexeme.pyx Update docs links in codebase 2020-09-04 12:58:50 +02:00
lookups.py Update docs links in codebase 2020-09-04 12:58:50 +02:00
morphology.pxd Add Lemmatizer and simplify related components (#5848) 2020-08-07 15:27:13 +02:00
morphology.pyx Add Lemmatizer and simplify related components (#5848) 2020-08-07 15:27:13 +02:00
parts_of_speech.pxd Add support for Universal Dependencies v2.0 2017-03-03 13:17:34 +01:00
parts_of_speech.pyx Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
pipe_analysis.py Simplify pipe analysis 2020-08-01 13:40:06 +02:00
schemas.py Add MORPH handling to Matcher (#6107) 2020-09-24 16:55:09 +02:00
scorer.py Revert changes to Scorer.score_spans 2020-09-25 08:21:47 +02:00
strings.pxd Remove 'cleanup' of strings (#6007) 2020-09-01 16:12:15 +02:00
strings.pyx Update docs links in codebase 2020-09-04 12:58:50 +02:00
structs.pxd Clean up MorphAnalysisC struct (#6146) 2020-09-25 15:56:48 +02:00
symbols.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
symbols.pyx Remove PRON_LEMMA symbol (#5968) 2020-08-25 14:21:29 +02:00
tokenizer.pxd Simplify specials and cache checks (#6012) 2020-09-03 09:42:49 +02:00
tokenizer.pyx Fix token.idx for special cases with affixes (#6035) 2020-09-13 14:05:36 +02:00
typedefs.pxd Update spaCy for thinc 8.0.0 (#4920) 2020-01-29 17:06:46 +01:00
typedefs.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
util.py Fix combined scores if multiple components report it 2020-09-24 17:11:13 +02:00
vectors.pyx Update docs links in codebase 2020-09-04 12:58:50 +02:00
vocab.pxd Minor renaming / refactoring 2020-09-18 19:43:19 +02:00
vocab.pyx Minor renaming / refactoring 2020-09-18 19:43:19 +02:00