spaCy/website/docs/api/architectures.md

title	teaser	source	menu
Model Architectures	Pre-defined model architectures included with the core library	spacy/ml/models	Tok2Vec tok2vec Transformers transformers Parser & NER parser Tagging tagger Text Classification textcat Entity Linking entitylinker

TODO: intro and how architectures work, link to registry, custom models usage etc.

Tok2Vec architectures

spacy.HashEmbedCNN.v1

Example Config

[model]
@architectures = "spacy.HashEmbedCNN.v1"
# TODO: ...

[model.tok2vec]
# ...

Name	Type	Description
width	int
depth	int
embed_size	int
window_size	int
maxout_pieces	int
subword_features	bool
dropout	float
pretrained_vectors	bool

spacy.HashCharEmbedCNN.v1

spacy.HashCharEmbedBiLSTM.v1

Transformer architectures

The following architectures are provided by the package spacy-transformers. See the usage documentation for how to integrate the architectures into your training config.

spacy-transformers.TransformerModel.v1

Example Config

[model]
@architectures = "spacy-transformers.TransformerModel.v1"
name = "roberta-base"
tokenizer_config = {"use_fast": true}

[model.get_spans]
@span_getters = "strided_spans.v1"
window = 128
stride = 96

Name	Type	Description
name	str	Any model name that can be loaded by transformers.AutoModel.
get_spans	Callable	Function that takes a batch of Doc object and returns lists of Span objects to process by the transformer. See here for built-in options and examples.
tokenizer_config	Dict[str, Any]	Tokenizer settings passed to transformers.AutoTokenizer.

spacy-transformers.Tok2VecListener.v1

Example Config

[model]
@architectures = "spacy-transformers.Tok2VecListener.v1"
grad_factor = 1.0

[model.pooling]
@layers = "reduce_mean.v1"

Name	Type	Description
grad_factor	float	Factor for weighting the gradient if multiple components listen to the same transformer model.
pooling	Model[Ragged, Floats2d]	Pooling layer to determine how the vector for each spaCy token will be computed.

Parser & NER architectures

spacy.TransitionBasedParser.v1

Example Config

[model]
@architectures = "spacy.TransitionBasedParser.v1"
nr_feature_tokens = 6
hidden_width = 64
maxout_pieces = 2

[model.tok2vec]
# ...

Name	Type	Description
tok2vec	Model
nr_feature_tokens	int
hidden_width	int
maxout_pieces	int
use_upper	bool
nO	int

Tagging architectures

spacy.Tagger.v1

Example Config

[model]
@architectures = "spacy.Tagger.v1"
nO = null

[model.tok2vec]
# ...

Name	Type	Description
tok2vec	Model
nO	int

Text classification architectures

spacy.TextCatEnsemble.v1

spacy.TextCatBOW.v1

spacy.TextCatCNN.v1

spacy.TextCatLowData.v1

Entity linking architectures

spacy.EntityLinker.v1

Example Config

[model]
@architectures = "spacy.EntityLinker.v1"
nO = null

[model.tok2vec]
# ...

Name	Type	Description
tok2vec	Model
nO	int