spaCy/spacy/lang
Ines Montani 43b960c01b
Refactor pipeline components, config and language data (#5759)
* Update with WIP

* Update with WIP

* Update with pipeline serialization

* Update types and pipe factories

* Add deep merge, tidy up and add tests

* Fix pipe creation from config

* Don't validate default configs on load

* Update spacy/language.py

Co-authored-by: Ines Montani <ines@ines.io>

* Adjust factory/component meta error

* Clean up factory args and remove defaults

* Add test for failing empty dict defaults

* Update pipeline handling and methods

* provide KB as registry function instead of as object

* small change in test to make functionality more clear

* update example script for EL configuration

* Fix typo

* Simplify test

* Simplify test

* splitting pipes.pyx into separate files

* moving default configs to each component file

* fix batch_size type

* removing default values from component constructors where possible (TODO: test 4725)

* skip instead of xfail

* Add test for config -> nlp with multiple instances

* pipeline.pipes -> pipeline.pipe

* Tidy up, document, remove kwargs

* small cleanup/generalization for Tok2VecListener

* use DEFAULT_UPSTREAM field

* revert to avoid circular imports

* Fix tests

* Replace deprecated arg

* Make model dirs require config

* fix pickling of keyword-only arguments in constructor

* WIP: clean up and integrate full config

* Add helper to handle function args more reliably

Now also includes keyword-only args

* Fix config composition and serialization

* Improve config debugging and add visual diff

* Remove unused defaults and fix type

* Remove pipeline and factories from meta

* Update spacy/default_config.cfg

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/default_config.cfg

* small UX edits

* avoid printing stack trace for debug CLI commands

* Add support for language-specific factories

* specify the section of the config which holds the model to debug

* WIP: add Language.from_config

* Update with language data refactor WIP

* Auto-format

* Add backwards-compat handling for Language.factories

* Update morphologizer.pyx

* Fix morphologizer

* Update and simplify lemmatizers

* Fix Japanese tests

* Port over tagger changes

* Fix Chinese and tests

* Update to latest Thinc

* WIP: xfail first Russian lemmatizer test

* Fix component-specific overrides

* fix nO for output layers in debug_model

* Fix default value

* Fix tests and don't pass objects in config

* Fix deep merging

* Fix lemma lookup data registry

Only load the lookups if an entry is available in the registry (and if spacy-lookups-data is installed)

* Add types

* Add Vocab.from_config

* Fix typo

* Fix tests

* Make config copying more elegant

* Fix pipe analysis

* Fix lemmatizers and is_base_form

* WIP: move language defaults to config

* Fix morphology type

* Fix vocab

* Remove comment

* Update to latest Thinc

* Add morph rules to config

* Tidy up

* Remove set_morphology option from tagger factory

* Hack use_gpu

* Move [pipeline] to top-level block and make [nlp.pipeline] list

Allows separating component blocks from component order – otherwise, ordering the config would mean a changed component order, which is bad. Also allows initial config to define more components and not use all of them

* Fix use_gpu and resume in CLI

* Auto-format

* Remove resume from config

* Fix formatting and error

* [pipeline] -> [components]

* Fix types

* Fix tagger test: requires set_morphology?

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
Co-authored-by: svlandeg <sofie.vanlandeghem@gmail.com>
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-07-22 13:42:59 +02:00
..
af Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ar Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
bg Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
bn Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ca Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
cs Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
da Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
de Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
el Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
en Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
es Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
et Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
eu Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
fa Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
fi Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
fr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ga Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
gu Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
he Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
hi Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
hr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
hu Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
hy Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
id Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
is Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
it Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ja Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
kn Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ko Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lb Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lij Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lt Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lv Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ml Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
mr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
nb Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ne Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
nl Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
pl Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
pt Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ro Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ru Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
si Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sk Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sl Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sq Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sv Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ta Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
te Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
th Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
tl Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
tr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
tt Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
uk Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ur Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
vi Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
xx Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
yo Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
zh Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
__init__.py Remove imports in /lang/__init__.py 2017-05-08 23:58:07 +02:00
char_classes.py Merge branch 'master' into develop 2020-02-18 14:47:23 +01:00
lex_attrs.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
norm_exceptions.py Tidy up and auto-format 2020-02-18 15:38:18 +01:00
punctuation.py Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
tag_map.py Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
tokenizer_exceptions.py Tidy up and auto-format 2020-06-21 22:38:04 +02:00