From c7c9b0451f9b0858d800fcb178d2f457564de986 Mon Sep 17 00:00:00 2001 From: Ines Montani Date: Sat, 22 Aug 2020 13:52:52 +0200 Subject: [PATCH] Update docs [ci skip] --- website/docs/api/architectures.md | 28 ++++++-- website/docs/usage/layers-architectures.md | 2 +- website/docs/usage/training.md | 82 ++-------------------- website/src/styles/layout.sass | 5 +- 4 files changed, 29 insertions(+), 88 deletions(-) diff --git a/website/docs/api/architectures.md b/website/docs/api/architectures.md index fd88434f1..3089fa1b3 100644 --- a/website/docs/api/architectures.md +++ b/website/docs/api/architectures.md @@ -11,9 +11,17 @@ menu: - ['Entity Linking', 'entitylinker'] --- -TODO: intro and how architectures work, link to -[`registry`](/api/top-level#registry), -[custom functions](/usage/training#custom-functions) usage etc. +A **model architecture** is a function that wires up a +[`Model`](https://thinc.ai/docs/api-model) instance, which you can then use in a +pipeline component or as a layer of a larger network. This page documents +spaCy's built-in architectures that are used for different NLP tasks. All +trainable [built-in components](/api#architecture-pipeline) expect a `model` +argument defined in the config and document their the default architecture. +Custom architectures can be registered using the +[`@spacy.registry.architectures`](/api/top-level#regsitry) decorator and used as +part of the [training config](/usage/training#custom-functions). Also see the +usage documentation on +[layers and model architectures](/usage/layers-architectures). ## Tok2Vec architectures {#tok2vec-arch source="spacy/ml/models/tok2vec.py"} @@ -284,8 +292,18 @@ on [static vectors](/usage/embeddings-transformers#static-vectors) for details. The following architectures are provided by the package [`spacy-transformers`](https://github.com/explosion/spacy-transformers). See the -[usage documentation](/usage/embeddings-transformers) for how to integrate the -architectures into your training config. +[usage documentation](/usage/embeddings-transformers#transformers) for how to +integrate the architectures into your training config. + + + +Note that in order to use these architectures in your config, you need to +install the +[`spacy-transformers`](https://github.com/explosion/spacy-transformers). See the +[installation docs](/usage/embeddings-transformers#transformers-installation) +for details and system requirements. + + ### spacy-transformers.TransformerModel.v1 {#TransformerModel} diff --git a/website/docs/usage/layers-architectures.md b/website/docs/usage/layers-architectures.md index 1ee0f4fae..eebcc4681 100644 --- a/website/docs/usage/layers-architectures.md +++ b/website/docs/usage/layers-architectures.md @@ -9,7 +9,7 @@ menu: next: /usage/projects --- -​ A **model architecture** is a function that wires up a +​A **model architecture** is a function that wires up a [Thinc `Model`](https://thinc.ai/docs/api-model) instance, which you can then use in a component or as a layer of a larger network. You can use Thinc as a thin wrapper around frameworks such as PyTorch, TensorFlow or MXNet, or you can diff --git a/website/docs/usage/training.md b/website/docs/usage/training.md index c04d3ca77..59766bada 100644 --- a/website/docs/usage/training.md +++ b/website/docs/usage/training.md @@ -6,8 +6,7 @@ menu: - ['Quickstart', 'quickstart'] - ['Config System', 'config'] - ['Custom Functions', 'custom-functions'] - - ['Transfer Learning', 'transfer-learning'] - - ['Parallel Training', 'parallel-training'] + # - ['Parallel Training', 'parallel-training'] - ['Internal API', 'api'] --- @@ -92,16 +91,6 @@ spaCy's binary `.spacy` format. You can either include the data paths in the $ python -m spacy train config.cfg --output ./output --paths.train ./train.spacy --paths.dev ./dev.spacy ``` - - ## Training config {#config} Training config files include all **settings and hyperparameters** for training @@ -400,13 +389,11 @@ recipe once the dish has already been prepared. You have to make a new one. spaCy includes a variety of built-in [architectures](/api/architectures) for different tasks. For example: - - | Architecture | Description | | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | [HashEmbedCNN](/api/architectures#HashEmbedCNN) | Build spaCy’s "standard" embedding layer, which uses hash embedding with subword features and a CNN with layer-normalized maxout. ~~Model[List[Doc], List[Floats2d]]~~ | | [TransitionBasedParser](/api/architectures#TransitionBasedParser) | Build a [transition-based parser](https://explosion.ai/blog/parsing-english-in-python) model used in the default [`EntityRecognizer`](/api/entityrecognizer) and [`DependencyParser`](/api/dependencyparser). ~~Model[List[Docs], List[List[Floats2d]]]~~ | -| [TextCatEnsemble](/api/architectures#TextCatEnsemble) | Stacked ensemble of a bag-of-words model and a neural network model with an internal CNN embedding layer. Used in the default [`TextCategorizer`](/api/textcategorizer). ~~Model~~ | +| [TextCatEnsemble](/api/architectures#TextCatEnsemble) | Stacked ensemble of a bag-of-words model and a neural network model with an internal CNN embedding layer. Used in the default [`TextCategorizer`](/api/textcategorizer). ~~Model[List[Doc], Floats2d]~~ | @@ -755,71 +742,10 @@ def filter_batch(size: int) -> Callable[[Iterable[Example]], Iterator[List[Examp return create_filtered_batches ``` - - ### Defining custom architectures {#custom-architectures} -## Transfer learning {#transfer-learning} - - - -### Using transformer models like BERT {#transformers} - -spaCy v3.0 lets you use almost any statistical model to power your pipeline. You -can use models implemented in a variety of frameworks. A transformer model is -just a statistical model, so the -[`spacy-transformers`](https://github.com/explosion/spacy-transformers) package -actually has very little work to do: it just has to provide a few functions that -do the required plumbing. It also provides a pipeline component, -[`Transformer`](/api/transformer), that lets you do multi-task learning and lets -you save the transformer outputs for later use. - - - -For more details on how to integrate transformer models into your training -config and customize the implementations, see the usage guide on -[training transformers](/usage/embeddings-transformers#transformers-training). - -### Pretraining with spaCy {#pretraining} - - - -## Parallel Training with Ray {#parallel-training} - - - ## Internal training API {#api} @@ -880,8 +806,8 @@ example = Example.from_dict(predicted, {"tags": tags}) Here's another example that shows how to define gold-standard named entities. The letters added before the labels refer to the tags of the [BILUO scheme](/usage/linguistic-features#updating-biluo) – `O` is a token -outside an entity, `U` a single entity unit, `B` the beginning of an entity, -`I` a token inside an entity and `L` the last token of an entity. +outside an entity, `U` a single entity unit, `B` the beginning of an entity, `I` +a token inside an entity and `L` the last token of an entity. ```python doc = Doc(nlp.vocab, words=["Facebook", "released", "React", "in", "2014"]) diff --git a/website/src/styles/layout.sass b/website/src/styles/layout.sass index b71eccd80..775523190 100644 --- a/website/src/styles/layout.sass +++ b/website/src/styles/layout.sass @@ -363,7 +363,7 @@ body [id]:target color: var(--color-red-medium) background: var(--color-red-transparent) - &.italic, &.comment + &.italic font-style: italic @@ -384,11 +384,9 @@ body [id]:target // Settings for ini syntax (config files) [class*="language-ini"] color: var(--syntax-comment) - font-style: italic !important .token color: var(--color-subtle) - font-style: normal !important .gatsby-highlight-code-line @@ -426,7 +424,6 @@ body [id]:target .cm-comment color: var(--syntax-comment) - font-style: italic .cm-keyword color: var(--syntax-keyword)