From a5627673367282904143ffcd0c1b1621585d6774 Mon Sep 17 00:00:00 2001 From: Adriane Boyd Date: Mon, 3 Apr 2023 10:57:36 +0200 Subject: [PATCH] Update website/docs/usage/embeddings-transformers.mdx --- website/docs/usage/embeddings-transformers.mdx | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/website/docs/usage/embeddings-transformers.mdx b/website/docs/usage/embeddings-transformers.mdx index 068db29f2..5f1e5b817 100644 --- a/website/docs/usage/embeddings-transformers.mdx +++ b/website/docs/usage/embeddings-transformers.mdx @@ -745,12 +745,12 @@ To benefit from pretraining, your training step needs to know to initialize its this by setting `initialize.init_tok2vec` to the filename of the `.bin` file that you want to use from pretraining. -Similar to training, pretraining produces a `model-last.bin` file which is the -last iteration of the trained weights which you can use to initialize your -`tok2vec` layer. Additionally, you can configure `n_save_epoch` to tell -pretraining in which epoch interval it should save the current training -progress. To make use of the final output, you could fill in this value in your -config file: +A pretraining step that runs for 5 epochs with an output path of `pretrain/`, as +an example, produces `pretrain/model0.bin` through `pretrain/model4.bin` plus a +copy of the last iteration as `pretrain/model-last.bin`. Additionally, you can +configure `n_save_epoch` to tell pretraining in which epoch interval it should +save the current training progress. To use the final output to initialize your +`tok2vec` layer, you could fill in this value in your config file: ```ini {title="config.cfg"}