explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-26 18:06:29 +03:00
Code Issues Packages Projects Releases Wiki Activity

Update docs [ci skip]

Browse Source
This commit is contained in:
Ines Montani 2020-08-21 16:11:38 +02:00
parent f5bcc10268
commit 74cb6d39d0
5 changed files with 125 additions and 25 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

49
website/docs/api/architectures.md
View File

@ -114,7 +114,7 @@ argument that connects to the shared `tok2vec` component in the pipeline.
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `width` | The width of the vectors produced by the "upstream" [`Tok2Vec`](/api/tok2vec) component. ~~int~~ |
| `upstream` | A string to identify the "upstream" `Tok2Vec` component to communicate with. The upstream name should either be the wildcard string `"*"`, or the name of the `Tok2Vec` component. You'll almost never have multiple upstream `Tok2Vec` components, so the wildcard string will almost always be fine. ~~str~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
### spacy.MultiHashEmbed.v1 {#MultiHashEmbed}
@ -143,7 +143,7 @@ representation.
| `rows` | The number of rows for the embedding tables. Can be low, due to the hashing trick. Embeddings for prefix, suffix and word shape use half as many rows. Recommended values are between `2000` and `10000`. ~~int~~ |
| `also_embed_subwords` | Whether to use the `PREFIX`, `SUFFIX` and `SHAPE` features in the embeddings. If not using these, you may need more rows in your hash embeddings, as there will be increased chance of collisions. ~~bool~~ |
| `also_use_static_vectors` | Whether to also use static word vectors. Requires a vectors table to be loaded in the [Doc](/api/doc) objects' vocab. ~~bool~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
### spacy.CharacterEmbed.v1 {#CharacterEmbed}
@ -180,7 +180,7 @@ construct a single vector to represent the information.
| `rows` | The number of rows in the `NORM` hash embedding table. ~~int~~ |
| `nM` | The dimensionality of the character embeddings. Recommended values are between `16` and `64`. ~~int~~ |
| `nC` | The number of UTF-8 bytes to embed per word. Recommended values are between `3` and `8`, although it may depend on the length of words in the language. ~~int~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], List[Floats2d]]~~ |
### spacy.MaxoutWindowEncoder.v1 {#MaxoutWindowEncoder}
@ -273,8 +273,8 @@ on [static vectors](/usage/embeddings-transformers#static-vectors) for details.
| Name |  Description |
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `nO` | The output width of the layer, after the linear projection. ~~Optional[int]~~ |
| `nM` | The width of the static vectors. ~~Optional[int]~~ |
| `nO` | The output width of the layer, after the linear projection. ~~Optional[int]~~ |
| `nM` | The width of the static vectors. ~~Optional[int]~~ |
| `dropout` | Optional dropout rate. If set, it's applied per dimension over the whole batch. Defaults to `None`. ~~Optional[float]~~ |
| `init_W` | The [initialization function](https://thinc.ai/docs/api-initializers). Defaults to [`glorot_uniform_init`](https://thinc.ai/docs/api-initializers#glorot_uniform_init). ~~Callable[[Ops, Tuple[int, ...]]], FloatsXd]~~ |
| `key_attr` | Defaults to `"ORTH"`. ~~str~~ |
@ -303,22 +303,23 @@ architectures into your training config.
> stride = 96
> ```
Load and wrap a transformer model from the Huggingface transformers library.
You can any transformer that has pretrained weights and a PyTorch
implementation. The `name` variable is passed through to the underlying
library, so it can be either a string or a path. If it's a string, the
pretrained weights will be downloaded via the transformers library if they are
not already available locally.
In order to support longer documents, the `TransformerModel` layer allows you
to pass in a `get_spans` function that will divide up the `Doc` objects before
passing them through the transformer. Your spans are allowed to overlap or
exclude tokens.
This layer is usually used directly by the `Transformer` component, which
allows you to share the transformer weights across your pipeline. For a layer
that's configured for use in other components, see `Tok2VecTransformer`.
Load and wrap a transformer model from the
[HuggingFace `transformers`](https://huggingface.co/transformers) library. You
can any transformer that has pretrained weights and a PyTorch implementation.
The `name` variable is passed through to the underlying library, so it can be
either a string or a path. If it's a string, the pretrained weights will be
downloaded via the transformers library if they are not already available
locally.
In order to support longer documents, the
[TransformerModel](/api/architectures#TransformerModel) layer allows you to pass
in a `get_spans` function that will divide up the [`Doc`](/api/doc) objects
before passing them through the transformer. Your spans are allowed to overlap
or exclude tokens. This layer is usually used directly by the
[`Transformer`](/api/transformer) component, which allows you to share the
transformer weights across your pipeline. For a layer that's configured for use
in other components, see
[Tok2VecTransformer](/api/architectures#Tok2VecTransformer).
| Name | Description |
| ------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
@ -559,7 +560,7 @@ network has an internal CNN Tok2Vec layer and uses attention.
| `ngram_size` | Determines the maximum length of the n-grams in the BOW model. For instance, `ngram_size=3`would give unigram, trigram and bigram features. ~~int~~ |
| `dropout` | The dropout rate. ~~float~~ |
| `nO` | Output dimension, determined by the number of different labels. If not set, the [`TextCategorizer`](/api/textcategorizer) component will set it when `begin_training` is called. ~~Optional[int]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
### spacy.TextCatCNN.v1 {#TextCatCNN}
@ -591,7 +592,7 @@ architecture is usually less accurate than the ensemble, but runs faster.
| `exclusive_classes` | Whether or not categories are mutually exclusive. ~~bool~~ |
| `tok2vec` | The [`tok2vec`](#tok2vec) layer of the model. ~~Model~~ |
| `nO` | Output dimension, determined by the number of different labels. If not set, the [`TextCategorizer`](/api/textcategorizer) component will set it when `begin_training` is called. ~~Optional[int]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
### spacy.TextCatBOW.v1 {#TextCatBOW}
@ -615,7 +616,7 @@ others, but may not be as accurate, especially if texts are short.
| `ngram_size` | Determines the maximum length of the n-grams in the BOW model. For instance, `ngram_size=3`would give unigram, trigram and bigram features. ~~int~~ |
| `no_output_layer` | Whether or not to add an output layer to the model (`Softmax` activation if `exclusive_classes` is `True`, else `Logistic`. ~~bool~~ |
| `nO` | Output dimension, determined by the number of different labels. If not set, the [`TextCategorizer`](/api/textcategorizer) component will set it when `begin_training` is called. ~~Optional[int]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
## Entity linking architectures {#entitylinker source="spacy/ml/models/entity_linker.py"}
@ -664,7 +665,7 @@ The `EntityLinker` model architecture is a Thinc `Model` with a
| ----------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `tok2vec` | The [`tok2vec`](#tok2vec) layer of the model. ~~Model~~ |
| `nO` | Output dimension, determined by the length of the vectors encoding each entity in the KB. If the `nO` dimension is not set, the entity linking component will set it when `begin_training` is called. ~~Optional[int]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
| **CREATES** | The model using the architecture. ~~Model[List[Doc], Floats2d]~~ |
### spacy.EmptyKB.v1 {#EmptyKB}

92
website/docs/usage/architectures.md Normal file
View File

@ -0,0 +1,92 @@
---
title: Layers and Model Architectures
teaser: Power spaCy components with custom neural networks
menu:
- ['Type Signatures', 'type-sigs']
- ['Defining Sublayers', 'sublayers']
- ['PyTorch & TensorFlow', 'frameworks']
- ['Trainable Components', 'components']
---
​ A **model architecture** is a function that wires up a
[Thinc `Model`](https://thinc.ai/docs/api-model) instance, which you can then
use in a component or as a layer of a larger network. You can use Thinc as a
thin wrapper around frameworks such as PyTorch, TensorFlow or MXNet, or you can
implement your logic in Thinc directly. ​ spaCy's built-in components will never
construct their `Model` instances themselves, so you won't have to subclass the
component to change its model architecture. You can just **update the config**
so that it refers to a different registered function. Once the component has
been created, its model instance has already been assigned, so you cannot change
its model architecture. The architecture is like a recipe for the network, and
you can't change the recipe once the dish has already been prepared. You have to
make a new one. ​
## Type signatures {#type-sigs}
​ The Thinc `Model` class is a **generic type** that can specify its input and
output types. Python uses a square-bracket notation for this, so the type
~~Model[List, Dict]~~ says that each batch of inputs to the model will be a
list, and the outputs will be a dictionary. Both `typing.List` and `typing.Dict`
are also generics, allowing you to be more specific about the data. For
instance, you can write ~~Model[List[Doc], Dict[str, float]]~~ to specify that
the model expects a list of [`Doc`](/api/doc) objects as input, and returns a
dictionary mapping strings to floats. Some of the most common types you'll see
are: ​
| Type | Description |
| ------------------ | ---------------------------------------------------------------------------------------------------- |
| ~~List[Doc]~~ | A batch of [`Doc`](/api/doc) objects. Most components expect their models to take this as input. |
| ~~Floats2d~~ | A two-dimensional `numpy` or `cupy` array of floats. Usually 32-bit. |
| ~~Ints2d~~ | A two-dimensional `numpy` or `cupy` array of integers. Common dtypes include uint64, int32 and int8. |
| ~~List[Floats2d]~~ | A list of two-dimensional arrays, generally with one array per `Doc` and one row per token. |
| ~~Ragged~~ | A container to handle variable-length sequence data in an unpadded contiguous array. |
| ~~Padded~~ | A container to handle variable-length sequence data in a passed contiguous array. |
The model type-signatures help you figure out which model architectures and
components can fit together. For instance, the
[`TextCategorizer`](/api/textcaregorizer) class expects a model typed
~~Model[List[Doc], Floats2d]~~, because the model will predict one row of
category probabilities per `Doc`. In contrast, the `Tagger` class expects a
model typed ~~Model[List[Doc], List[Floats2d]]~~, because it needs to predict
one row of probabilities per token. ​ There's no guarantee that two models with
the same type-signature can be used interchangeably. There are many other ways
they could be incompatible. However, if the types don't match, they almost
surely _won't_ be compatible. This little bit of validation goes a long way,
especially if you configure your editor or other tools to highlight these errors
early. Thinc will also verify that your types match correctly when your config
file is processed at the beginning of training. ​
## Defining sublayers {#sublayers}
​ Model architecture functions often accept sublayers as arguments, so that you
can try substituting a different layer into the network. Depending on how the
architecture function is structured, you might be able to define your network
structure entirely through the [config system](/usage/training#config), using
layers that have already been defined. ​The
[transformers documentation](/usage/embeddings-transformers#transformers)
section shows a common example of swapping in a different sublayer. In most NLP
neural network models, the most important parts of the network are what we refer
to as the
[embed and encode](https://explosion.ai/blog/embed-encode-attend-predict) steps.
These steps together compute dense, context-sensitive representations of the
tokens. Most of spaCy's default architectures accept a `tok2vec` layer as an
argument, so you can control this important part of the network separately. This
makes it easy to switch between transformer, CNN, BiLSTM or other feature
extraction approaches. And if you want to define your own solution, all you need
to do is register a ~~Model[List[Doc], List[Floats2d]]~~ architecture function,
and you'll be able to try it out in any of spaCy components. ​
### Registering new architectures
- Recap concept, link to config docs. ​
## Wrapping PyTorch, TensorFlow and other frameworks {#frameworks}
- Explain concept
- Link off to notebook ​
## Models for trainable components {#components}
- Interaction with `predict`, `get_loss` and `set_annotations`
- Initialization life-cycle with `begin_training`.
- Link to relation extraction notebook.

5
website/meta/sidebars.json
View File

@ -24,6 +24,11 @@
"tag": "new"
},
{ "text": "Training Models", "url": "/usage/training", "tag": "new" },
{
"text": "Layers & Model Architectures",
"url": "/usage/architectures",
"tag": "new"
},
{ "text": "spaCy Projects", "url": "/usage/projects", "tag": "new" },
{ "text": "Saving & Loading", "url": "/usage/saving-loading" },
{ "text": "Visualizers", "url": "/usage/visualizers" }

2
website/meta/type-annotations.json
View File

@ -29,6 +29,8 @@
"Optimizer": "https://thinc.ai/docs/api-optimizers",
"Model": "https://thinc.ai/docs/api-model",
"Ragged": "https://thinc.ai/docs/api-types#ragged",
"Padded": "https://thinc.ai/docs/api-types#padded",
"Ints2d": "https://thinc.ai/docs/api-types#types",
"Floats2d": "https://thinc.ai/docs/api-types#types",
"Floats3d": "https://thinc.ai/docs/api-types#types",
"FloatsXd": "https://thinc.ai/docs/api-types#types",

2
website/src/styles/code.module.sass
View File

@ -67,7 +67,7 @@
border: 0
// Special style for types in API tables
td > &:last-child
td:not(:first-child) > &:last-child
display: block
border-top: 1px dotted var(--color-subtle)
border-radius: 0