spaCy/spacy/tests/lang
Ines Montani 43b960c01b
Refactor pipeline components, config and language data (#5759)
* Update with WIP

* Update with WIP

* Update with pipeline serialization

* Update types and pipe factories

* Add deep merge, tidy up and add tests

* Fix pipe creation from config

* Don't validate default configs on load

* Update spacy/language.py

Co-authored-by: Ines Montani <ines@ines.io>

* Adjust factory/component meta error

* Clean up factory args and remove defaults

* Add test for failing empty dict defaults

* Update pipeline handling and methods

* provide KB as registry function instead of as object

* small change in test to make functionality more clear

* update example script for EL configuration

* Fix typo

* Simplify test

* Simplify test

* splitting pipes.pyx into separate files

* moving default configs to each component file

* fix batch_size type

* removing default values from component constructors where possible (TODO: test 4725)

* skip instead of xfail

* Add test for config -> nlp with multiple instances

* pipeline.pipes -> pipeline.pipe

* Tidy up, document, remove kwargs

* small cleanup/generalization for Tok2VecListener

* use DEFAULT_UPSTREAM field

* revert to avoid circular imports

* Fix tests

* Replace deprecated arg

* Make model dirs require config

* fix pickling of keyword-only arguments in constructor

* WIP: clean up and integrate full config

* Add helper to handle function args more reliably

Now also includes keyword-only args

* Fix config composition and serialization

* Improve config debugging and add visual diff

* Remove unused defaults and fix type

* Remove pipeline and factories from meta

* Update spacy/default_config.cfg

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/default_config.cfg

* small UX edits

* avoid printing stack trace for debug CLI commands

* Add support for language-specific factories

* specify the section of the config which holds the model to debug

* WIP: add Language.from_config

* Update with language data refactor WIP

* Auto-format

* Add backwards-compat handling for Language.factories

* Update morphologizer.pyx

* Fix morphologizer

* Update and simplify lemmatizers

* Fix Japanese tests

* Port over tagger changes

* Fix Chinese and tests

* Update to latest Thinc

* WIP: xfail first Russian lemmatizer test

* Fix component-specific overrides

* fix nO for output layers in debug_model

* Fix default value

* Fix tests and don't pass objects in config

* Fix deep merging

* Fix lemma lookup data registry

Only load the lookups if an entry is available in the registry (and if spacy-lookups-data is installed)

* Add types

* Add Vocab.from_config

* Fix typo

* Fix tests

* Make config copying more elegant

* Fix pipe analysis

* Fix lemmatizers and is_base_form

* WIP: move language defaults to config

* Fix morphology type

* Fix vocab

* Remove comment

* Update to latest Thinc

* Add morph rules to config

* Tidy up

* Remove set_morphology option from tagger factory

* Hack use_gpu

* Move [pipeline] to top-level block and make [nlp.pipeline] list

Allows separating component blocks from component order – otherwise, ordering the config would mean a changed component order, which is bad. Also allows initial config to define more components and not use all of them

* Fix use_gpu and resume in CLI

* Auto-format

* Remove resume from config

* Fix formatting and error

* [pipeline] -> [components]

* Fix types

* Fix tagger test: requires set_morphology?

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
Co-authored-by: svlandeg <sofie.vanlandeghem@gmail.com>
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-07-22 13:42:59 +02:00
..
ar Tidy up and auto-format 2020-02-18 15:38:18 +01:00
bn Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
ca Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
da Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
de Remove corpus-specific tag maps 2020-07-15 15:58:29 +02:00
el Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
en Test suite clean up (#5781) 2020-07-20 14:49:54 +02:00
es Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
eu Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
fa Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
fi Merge branch 'master' into tmp/sync 2020-03-26 13:38:14 +01:00
fr Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ga Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
gu Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
he Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
hu Tidy up and auto-format 2020-03-25 12:28:12 +01:00
hy Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
id Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
it Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
ja Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
ko Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
lb Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
lt Merge branch 'master' into tmp/sync 2020-03-26 13:38:14 +01:00
ml Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
nb Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
ne Add Nepali Language (#5622) 2020-06-22 10:25:46 +02:00
nl Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
pl Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
pt Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
ro Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
ru Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
sr Un-xfail passing tests 2019-12-25 18:02:20 +01:00
sv Remove unicode declarations and tidy up 2020-06-21 22:34:10 +02:00
th Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
tr Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
tt Merge branch 'master' into develop 2020-02-18 14:47:23 +01:00
uk Test suite clean up (#5781) 2020-07-20 14:49:54 +02:00
ur Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
yo Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
zh Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
__init__.py Revert #4334 2019-09-29 17:32:12 +02:00
test_attrs.py Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
test_initialize.py Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00