mirror of
https://github.com/explosion/spaCy.git
synced 2024-11-11 04:08:09 +03:00
Update NER training examples
This commit is contained in:
ines
parent
8116d1a077
commit
281f88a59c
|
|
@ -24,6 +24,58 @@ p
|
|||
| #[strong experiment on your own data] to find a solution that works best
|
||||
| for you.
|
||||
|
||||
+h(3, "example-train-ner") Updating the Named Entity Recognizer
|
||||
|
||||
p
|
||||
| This example shows how to update spaCy's entity recognizer
|
||||
| with your own examples, starting off with an existing, pre-trained
|
||||
| model, or from scratch using a blank #[code Language] class. To do
|
||||
| this, you'll need #[strong example texts] and the
|
||||
| #[strong character offsets] and #[strong labels] of each entity contained
|
||||
| in the texts.
|
||||
|
||||
+github("spacy", "examples/training/train_ner.py")
|
||||
|
||||
+h(4) Step by step guide
|
||||
|
||||
+list("numbers")
|
||||
+item
|
||||
| #[strong Reformat the training data] to match spaCy's
|
||||
| #[+a("/api/annotation#json-input") JSON format]. The built-in
|
||||
| #[+api("goldparse#biluo_tags_from_offsets") #[code biluo_tags_from_offsets]]
|
||||
| function can help you with this.
|
||||
|
||||
+item
|
||||
| #[strong Load the model] you want to start with, or create an
|
||||
| #[strong empty model] using
|
||||
| #[+api("spacy#blank") #[code spacy.blank]] with the ID of your
|
||||
| language. If you're using a blank model, don't forget to add the
|
||||
| entity recognizer to the pipeline. If you're using an existing model,
|
||||
| make sure to disable all other pipeline components during training
|
||||
| using #[+api("language#disable_pipes") #[code nlp.disable_pipes]].
|
||||
| This way, you'll only be training the entity recognizer.
|
||||
|
||||
+item
|
||||
| #[strong Shuffle and loop over] the examples and create a
|
||||
| #[code Doc] and #[code GoldParse] object for each example.
|
||||
|
||||
+item
|
||||
| For each example, #[strong update the model]
|
||||
| by calling #[+api("language#update") #[code nlp.update]], which steps
|
||||
| through the words of the input. At each word, it makes a
|
||||
| #[strong prediction]. It then consults the annotations provided on the
|
||||
| #[code GoldParse] instance, to see whether it was
|
||||
| right. If it was wrong, it adjusts its weights so that the correct
|
||||
| action will score higher next time.
|
||||
|
||||
+item
|
||||
| #[strong Save] the trained model using
|
||||
| #[+api("language#to_disk") #[code nlp.to_disk]].
|
||||
|
||||
+item
|
||||
| #[strong Test] the model to make sure the entities in the training
|
||||
| data are recognised correctly.
|
||||
|
||||
+h(3, "example-new-entity-type") Training an additional entity type
|
||||
|
||||
p
|
||||
|
|
@ -38,22 +90,22 @@ p
|
|||
|
||||
+github("spacy", "examples/training/train_new_entity_type.py")
|
||||
|
||||
p Training a new entity type requires the following steps:
|
||||
+h(4) Step by step guide
|
||||
|
||||
+list("numbers")
|
||||
+item
|
||||
| Create #[+api("doc") #[code Doc]] and
|
||||
| #[+api("goldparse") #[code GoldParse]] objects for
|
||||
| Create #[code Doc] and #[code GoldParse] objects for
|
||||
| #[strong each example in your training data].
|
||||
|
||||
+item
|
||||
| #[strong Load the model] you want to start with, or create an
|
||||
| #[strong empty model] using
|
||||
| #[+api("spacy#blank") #[code spacy.blank()]] with the ID of your
|
||||
| language. If you're using an existing model, make sure to disable
|
||||
| all other pipeline components during training using
|
||||
| #[+api("language#disable_pipes") #[code nlp.disable_pipes]]. This way,
|
||||
| you'll only be training the entity recognizer.
|
||||
| #[+api("spacy#blank") #[code spacy.blank]] with the ID of your
|
||||
| language. If you're using a blank model, don't forget to add the
|
||||
| entity recognizer to the pipeline. If you're using an existing model,
|
||||
| make sure to disable all other pipeline components during training
|
||||
| using #[+api("language#disable_pipes") #[code nlp.disable_pipes]].
|
||||
| This way, you'll only be training the entity recognizer.
|
||||
|
||||
+item
|
||||
| #[strong Add the new entity label] to the entity recognizer using the
|
||||
|
|
@ -66,28 +118,14 @@ p Training a new entity type requires the following steps:
|
|||
| #[+api("language#update") #[code nlp.update]], which steps through
|
||||
| the words of the input. At each word, it makes a
|
||||
| #[strong prediction]. It then consults the annotations provided on the
|
||||
| #[+api("goldparse") #[code GoldParse]] instance, to see whether it was
|
||||
| right. If it was wrong, it adjusts its weights so that the correct
|
||||
| action will score higher next time.
|
||||
| #[code GoldParse] instance, to see whether it was right. If it was
|
||||
| wrong, it adjusts its weights so that the correct action will score
|
||||
| higher next time.
|
||||
|
||||
+item
|
||||
| #[strong Save] the trained model using
|
||||
| #[+api("language#to_disk") #[code nlp.to_disk()]].
|
||||
| #[+api("language#to_disk") #[code nlp.to_disk]].
|
||||
|
||||
+item
|
||||
| #[strong Test] the model to make sure the new entity is recognized
|
||||
| #[strong Test] the model to make sure the new entity is recognised
|
||||
| correctly.
|
||||
|
||||
+h(3, "example-ner-from-scratch") Example: Training an NER system from scratch
|
||||
|
||||
p
|
||||
| This example is written to be self-contained and reasonably transparent.
|
||||
| To achieve that, it duplicates some of spaCy's internal functionality.
|
||||
| Specifically, in this example, we don't use spaCy's built-in
|
||||
| #[+api("language") #[code Language]] class to wire together the
|
||||
| #[+api("vocab") #[code Vocab]], #[+api("tokenizer") #[code Tokenizer]]
|
||||
| and #[+api("entityrecognizer") #[code EntityRecognizer]]. Instead, we
|
||||
| write our own simle #[code Pipeline] class, so that it's easier to see
|
||||
| how the pieces interact.
|
||||
|
||||
+github("spacy", "examples/training/train_ner_standalone.py")
|
||||
|
|
|
|||
|
|
@ -61,6 +61,15 @@ include ../_includes/_mixins
|
|||
+github("spacy", "examples/phrase_matcher.py")
|
||||
|
||||
+section("training")
|
||||
+h(3, "training-ner") Training spaCy's Named Entity Recognizer
|
||||
|
||||
p
|
||||
| This example shows how to update spaCy's entity recognizer
|
||||
| with your own examples, starting off with an existing, pre-trained
|
||||
| model, or from scratch using a blank #[code Language] class.
|
||||
|
||||
+github("spacy", "examples/training/train_ner.py")
|
||||
|
||||
+h(3, "new-entity-type") Training an additional entity type
|
||||
|
||||
p
|
||||
|
|
@ -71,15 +80,6 @@ include ../_includes/_mixins
|
|||
|
||||
+github("spacy", "examples/training/train_new_entity_type.py")
|
||||
|
||||
+h(3, "ner-standalone") Training an NER system from scratch
|
||||
|
||||
p
|
||||
| This example is written to be self-contained and reasonably
|
||||
| transparent. To achieve that, it duplicates some of spaCy's internal
|
||||
| functionality.
|
||||
|
||||
+github("spacy", "examples/training/train_ner_standalone.py")
|
||||
|
||||
+h(3, "textcat") Training spaCy's text classifier
|
||||
+tag-new(2)
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user