diff --git a/website/docs/images/layers-architectures.svg b/website/docs/images/layers-architectures.svg deleted file mode 100644 index 22e705ba1..000000000 --- a/website/docs/images/layers-architectures.svg +++ /dev/null @@ -1,97 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - diff --git a/website/docs/images/trainable_component.svg b/website/docs/images/trainable_component.svg new file mode 100644 index 000000000..621ff90ef --- /dev/null +++ b/website/docs/images/trainable_component.svg @@ -0,0 +1,55 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/website/docs/usage/layers-architectures.md b/website/docs/usage/layers-architectures.md index 24c7bf1cf..7fa60e0f1 100644 --- a/website/docs/usage/layers-architectures.md +++ b/website/docs/usage/layers-architectures.md @@ -646,7 +646,9 @@ get_candidates = model.attrs["get_candidates"] To use our new relation extraction model as part of a custom [trainable component](/usage/processing-pipelines#trainable-components), we -create a subclass of [`Pipe`](/api/pipe) that holds the model: +create a subclass of [`Pipe`](/api/pipe) that holds the model. + +![Illustration of Pipe methods](../images/trainable_component.svg) ```python ### Pipeline component skeleton @@ -826,7 +828,7 @@ def __call__(self, Doc doc): Once our `Pipe` subclass is fully implemented, we can [register](/usage/processing-pipelines#custom-components-factories) the -component with the [`@Language.factory`](/api/lnguage#factory) decorator. This +component with the [`@Language.factory`](/api/language#factory) decorator. This assigns it a name and lets you create the component with [`nlp.add_pipe`](/api/language#add_pipe) and via the [config](/usage/training#config). diff --git a/website/docs/usage/processing-pipelines.md b/website/docs/usage/processing-pipelines.md index c8224dfc9..8b4e39ee9 100644 --- a/website/docs/usage/processing-pipelines.md +++ b/website/docs/usage/processing-pipelines.md @@ -1172,13 +1172,15 @@ doc = nlp("This is a text...") spaCy's [`Pipe`](/api/pipe) class helps you implement your own trainable components that have their own model instance, make predictions over `Doc` objects and can be updated using [`spacy train`](/api/cli#train). This lets you -plug fully custom machine learning components into your pipeline. You'll need -the following: +plug fully custom machine learning components into your pipeline. + +![Illustration of Pipe methods](../images/trainable_component.svg) + +You'll need the following: 1. **Model:** A Thinc [`Model`](https://thinc.ai/docs/api-model) instance. This - can be a model implemented in - [Thinc](/usage/layers-architectures#thinc), or a - [wrapped model](/usage/layers-architectures#frameworks) implemented in + can be a model implemented in [Thinc](/usage/layers-architectures#thinc), or + a [wrapped model](/usage/layers-architectures#frameworks) implemented in PyTorch, TensorFlow, MXNet or a fully custom solution. The model must take a list of [`Doc`](/api/doc) objects as input and can have any type of output. 2. **Pipe subclass:** A subclass of [`Pipe`](/api/pipe) that implements at least @@ -1283,7 +1285,7 @@ loss is calculated and to add evaluation scores to the training output. For more details on how to implement your own trainable components and model architectures, and plug existing models implemented in PyTorch or TensorFlow into your spaCy pipeline, see the usage guide on -[layers and model architectures](/usage/layers-architectures). +[layers and model architectures](/usage/layers-architectures#components). diff --git a/website/docs/usage/spacy-101.md b/website/docs/usage/spacy-101.md index 5d7c7d7a5..c315c5f76 100644 --- a/website/docs/usage/spacy-101.md +++ b/website/docs/usage/spacy-101.md @@ -404,8 +404,73 @@ import Training101 from 'usage/101/\_training.md' To learn more about **training and updating** pipelines, how to create training -data and how to improve spaCy's named entity recognition models, see the usage -guides on [training](/usage/training). +data and how to improve spaCy's named models, see the usage guides on +[training](/usage/training). + + + +### Training config and lifecycle {#training-config} + +Training config files include all **settings and hyperparameters** for training +your pipeline. Instead of providing lots of arguments on the command line, you +only need to pass your `config.cfg` file to [`spacy train`](/api/cli#train). +This also makes it easy to integrate custom models and architectures, written in +your framework of choice. A pipeline's `config.cfg` is considered the "single +source of truth", both at **training** and **runtime**. + +> ```ini +> ### config.cfg (excerpt) +> [training] +> accumulate_gradient = 3 +> +> [training.optimizer] +> @optimizers = "Adam.v1" +> +> [training.optimizer.learn_rate] +> @schedules = "warmup_linear.v1" +> warmup_steps = 250 +> total_steps = 20000 +> initial_rate = 0.01 +> ``` + +![Illustration of pipeline lifecycle](../images/lifecycle.svg) + + + +For more details on spaCy's **configuration system** and how to use it to +customize your pipeline components, component models, training settings and +hyperparameters, see the [training config](/usage/training#config) usage guide. + + + +### Trainable components {#training-components} + +spaCy's [`Pipe`](/api/pipe) class helps you implement your own trainable +components that have their own model instance, make predictions over `Doc` +objects and can be updated using [`spacy train`](/api/cli#train). This lets you +plug fully custom machine learning components into your pipeline that can be +configured via a single training config. + +> #### config.cfg (excerpt) +> +> ```ini +> [components.my_component] +> factory = "my_component" +> +> [components.my_component.model] +> @architectures = "my_model.v1" +> width = 128 +> ``` + +![Illustration of Pipe methods](../images/trainable_component.svg) + + + +To learn more about how to implement your own **model architectures** and use +them to power custom **trainable components**, see the usage guides on the +[trainable component API](/usage/processing-pipelines#trainable-components) and +implementing [layers and architectures](/usage/layers-architectures#components) +for trainable components.