spaCy/spacy
Ines Montani 43b960c01b
Refactor pipeline components, config and language data (#5759)
* Update with WIP

* Update with WIP

* Update with pipeline serialization

* Update types and pipe factories

* Add deep merge, tidy up and add tests

* Fix pipe creation from config

* Don't validate default configs on load

* Update spacy/language.py

Co-authored-by: Ines Montani <ines@ines.io>

* Adjust factory/component meta error

* Clean up factory args and remove defaults

* Add test for failing empty dict defaults

* Update pipeline handling and methods

* provide KB as registry function instead of as object

* small change in test to make functionality more clear

* update example script for EL configuration

* Fix typo

* Simplify test

* Simplify test

* splitting pipes.pyx into separate files

* moving default configs to each component file

* fix batch_size type

* removing default values from component constructors where possible (TODO: test 4725)

* skip instead of xfail

* Add test for config -> nlp with multiple instances

* pipeline.pipes -> pipeline.pipe

* Tidy up, document, remove kwargs

* small cleanup/generalization for Tok2VecListener

* use DEFAULT_UPSTREAM field

* revert to avoid circular imports

* Fix tests

* Replace deprecated arg

* Make model dirs require config

* fix pickling of keyword-only arguments in constructor

* WIP: clean up and integrate full config

* Add helper to handle function args more reliably

Now also includes keyword-only args

* Fix config composition and serialization

* Improve config debugging and add visual diff

* Remove unused defaults and fix type

* Remove pipeline and factories from meta

* Update spacy/default_config.cfg

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/default_config.cfg

* small UX edits

* avoid printing stack trace for debug CLI commands

* Add support for language-specific factories

* specify the section of the config which holds the model to debug

* WIP: add Language.from_config

* Update with language data refactor WIP

* Auto-format

* Add backwards-compat handling for Language.factories

* Update morphologizer.pyx

* Fix morphologizer

* Update and simplify lemmatizers

* Fix Japanese tests

* Port over tagger changes

* Fix Chinese and tests

* Update to latest Thinc

* WIP: xfail first Russian lemmatizer test

* Fix component-specific overrides

* fix nO for output layers in debug_model

* Fix default value

* Fix tests and don't pass objects in config

* Fix deep merging

* Fix lemma lookup data registry

Only load the lookups if an entry is available in the registry (and if spacy-lookups-data is installed)

* Add types

* Add Vocab.from_config

* Fix typo

* Fix tests

* Make config copying more elegant

* Fix pipe analysis

* Fix lemmatizers and is_base_form

* WIP: move language defaults to config

* Fix morphology type

* Fix vocab

* Remove comment

* Update to latest Thinc

* Add morph rules to config

* Tidy up

* Remove set_morphology option from tagger factory

* Hack use_gpu

* Move [pipeline] to top-level block and make [nlp.pipeline] list

Allows separating component blocks from component order – otherwise, ordering the config would mean a changed component order, which is bad. Also allows initial config to define more components and not use all of them

* Fix use_gpu and resume in CLI

* Auto-format

* Remove resume from config

* Fix formatting and error

* [pipeline] -> [components]

* Fix types

* Fix tagger test: requires set_morphology?

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
Co-authored-by: svlandeg <sofie.vanlandeghem@gmail.com>
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-07-22 13:42:59 +02:00
..
cli Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
displacy Remove object subclassing 2020-07-12 14:03:23 +02:00
gold Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lang Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
matcher Remove object subclassing 2020-07-12 14:03:23 +02:00
ml Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
pipeline Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
syntax Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
tests Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
tokens Merge branch 'develop' into master-tmp 2020-07-20 14:58:04 +02:00
__init__.pxd * Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags. 2014-10-24 02:23:42 +11:00
__init__.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
__main__.py Tidy up 2020-06-22 00:45:40 +02:00
about.py Set version to 2.3.2 (#5756) 2020-07-13 14:55:56 +02:00
attrs.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
attrs.pyx Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
compat.py Merge branch 'develop' into refactor/remove-symlinks 2020-02-18 17:22:20 +01:00
default_config.cfg Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
errors.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
glossary.py unicode -> str consistency 2020-05-24 17:20:58 +02:00
gold.pyx Improve spacy.gold (no GoldParse, no json format!) (#5555) 2020-06-26 19:34:12 +02:00
kb.pxd Tidy up and avoid absolute spacy imports in core 2020-05-21 20:05:03 +02:00
kb.pyx Merge branch 'develop' into master-tmp 2020-06-20 15:52:00 +02:00
language.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lemmatizer.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
lexeme.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
lexeme.pyx Merge branch 'develop' into master-tmp 2020-07-20 14:58:04 +02:00
lookups.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
morphology.pxd Update Morphology to load exceptions as MORPH_RULES 2020-07-16 21:16:49 +02:00
morphology.pyx Merge branch 'develop' into pr/5767 2020-07-19 13:37:46 +02:00
parts_of_speech.pxd Add support for Universal Dependencies v2.0 2017-03-03 13:17:34 +01:00
parts_of_speech.pyx Drop Python 2.7 and 3.5 (#4828) 2019-12-22 01:53:56 +01:00
pipe_analysis.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
schemas.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
scorer.py Remove object subclassing 2020-07-12 14:03:23 +02:00
strings.pxd Tidy up compiler flags and imports (#5071) 2020-03-02 11:48:10 +01:00
strings.pyx unicode -> str consistency [ci skip] 2020-05-24 18:51:10 +02:00
structs.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
symbols.pxd Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
symbols.pyx Merge branch 'develop' into master-tmp 2020-05-21 18:39:06 +02:00
tokenizer.pxd Remove dead and/or deprecated code (#5710) 2020-07-06 13:06:25 +02:00
tokenizer.pyx Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
typedefs.pxd Update spaCy for thinc 8.0.0 (#4920) 2020-01-29 17:06:46 +01:00
typedefs.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
util.py Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
vectors.pyx Remove object subclassing 2020-07-12 14:03:23 +02:00
vocab.pxd Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00
vocab.pyx Refactor pipeline components, config and language data (#5759) 2020-07-22 13:42:59 +02:00