explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-26 18:06:29 +03:00
Code Issues Packages Projects Releases Wiki Activity
67928036f2
spaCy/website/docs/api/scorer.md
Sofie Van Landeghem c0f4a1e43b
train is from-config by default (#5575) 
* verbose and tag_map options

* adding init_tok2vec option and only changing the tok2vec that is specified

* adding omit_extra_lookups and verifying textcat config

* wip

* pretrain bugfix

* add replace and resume options

* train_textcat fix

* raw text functionality

* improve UX when KeyError or when input data can't be parsed

* avoid unnecessary access to goldparse in TextCat pipe

* save performance information in nlp.meta

* add noise_level to config

* move nn_parser's defaults to config file

* multitask in config - doesn't work yet

* scorer offering both F and AUC options, need to be specified in config

* add textcat verification code from old train script

* small fixes to config files

* clean up

* set default config for ner/parser to allow create_pipe to work as before

* two more test fixes

* small fixes

* cleanup

* fix NER pickling + additional unit test

* create_pipe as before
2020-06-12 02:02:07 +02:00

4.5 KiB
Raw Blame History

title teaser tag source
Scorer Compute evaluation scores class spacy/scorer.py

The Scorer computes and stores evaluation scores. It's typically created by Language.evaluate.

Scorer.__init__

Create a new Scorer.

Example

from spacy.scorer import Scorer

scorer = Scorer()
Name Type Description
eval_punct bool Evaluate the dependency attachments to and from punctuation.
RETURNS Scorer The newly created object.

Scorer.score

Update the evaluation scores from a single Doc / GoldParse pair.

Example

scorer = Scorer()
scorer.score(doc, gold)
Name Type Description
doc Doc The predicted annotations.
gold GoldParse The correct annotations.
verbose bool Print debugging information.
punct_labels tuple Dependency labels for punctuation. Used to evaluate dependency attachments to punctuation if eval_punct is True.

Properties

Name Type Description
token_acc float Tokenization accuracy.
tags_acc float Part-of-speech tag accuracy (fine grained tags, i.e. Token.tag).
uas float Unlabelled dependency score.
las float Labelled dependency score.
ents_p float Named entity accuracy (precision).
ents_r float Named entity accuracy (recall).
ents_f float Named entity accuracy (F-score).
ents_per_type 2.1.5 dict Scores per entity label. Keyed by label, mapped to a dict of p, r and f scores.
textcat_f 3.0 float F-score on positive label for binary classification, macro-averaged F-score otherwise.
textcat_auc <Tag variant="new"3.0 float Macro-averaged AUC ROC score for multilabel classification (-1 if undefined).
textcats_f_per_cat 3.0 dict F-scores per textcat label, keyed by label.
textcats_auc_per_cat 3.0 dict ROC AUC scores per textcat label, keyed by label.
las_per_type 2.2.3 dict Labelled dependency scores, keyed by label.
scores dict All scores, keyed by type.