spaCy/website/docs/usage/training-ner.jade
2017-06-01 11:56:02 +02:00

67 lines
2.8 KiB
Plaintext

include ../../_includes/_mixins
p
| All #[+a("/docs/usage/models") spaCy models] support online learning, so
| you can update a pre-trained model with new examples. You can even add
| new classes to an existing model, to recognise a new entity type,
| part-of-speech, or syntactic relation. Updating an existing model is
| particularly useful as a "quick and dirty solution", if you have only a
| few corrections or annotations.
+under-construction
+h(2, "improving-accuracy") Improving accuracy on existing entity types
p
| To update the model, you first need to create an instance of
| #[+api("goldparse") #[code spacy.gold.GoldParse]], with the entity labels
| you want to learn. You will then pass this instance to the
| #[+api("entityrecognizer#update") #[code EntityRecognizer.update()]]
| method.
p
| You'll usually need to provide many examples to meaningfully improve the
| system — a few hundred is a good start, although more is better. You
| should avoid iterating over the same few examples multiple times, or the
| model is likely to "forget" how to annotate other examples. If you
| iterate over the same few examples, you're effectively changing the loss
| function. The optimizer will find a way to minimize the loss on your
| examples, without regard for the consequences on the examples it's no
| longer paying attention to.
p
| One way to avoid this "catastrophic forgetting" problem is to "remind"
| the model of other examples by augmenting your annotations with sentences
| annotated with entities automatically recognised by the original model.
| Ultimately, this is an empirical process: you'll need to
| #[strong experiment on your own data] to find a solution that works best
| for you.
+h(2, "saving-loading") Saving and loading
p
| After training our model, you'll usually want to save its state, and load
| it back later. You can do this with the
| #[+api("language#to_disk") #[code Language.to_disk()]] method:
+code.
nlp.to_disk('/home/me/data/en_technology')
p
| To make the model more convenient to deploy, we recommend wrapping it as
| a Python package, so that you can install it via pip and load it as a
| module. spaCy comes with a handy #[+api("cli#package") #[code package]]
| CLI command to create all required files and directories.
+code(false, "bash").
python -m spacy package /home/me/data/en_technology /home/me/my_models
p
| To build the package and create a #[code .tar.gz] archive, run
| #[code python setup.py sdist] from within its directory.
+infobox("Saving and loading models")
| For more information and a detailed guide on how to package your model,
| see the documentation on
| #[+a("/docs/usage/saving-loading#models") saving and loading models].