mirror of
https://github.com/explosion/spaCy.git
synced 2025-10-23 20:24:28 +03:00
b5d999e510
* Add doc.cats to spacy.gold at the paragraph level
Support `doc.cats` as `"cats": [{"label": string, "value": number}]` in
the spacy JSON training format at the paragraph level.
* `spacy.gold.docs_to_json()` writes `docs.cats`
* `GoldCorpus` reads in cats in each `GoldParse`
* Update instances of gold_tuples to handle cats
Update iteration over gold_tuples / gold_parses to handle addition of
cats at the paragraph level.
* Add textcat to train CLI
* Add textcat options to train CLI
* Add textcat labels in `TextCategorizer.begin_training()`
* Add textcat evaluation to `Scorer`:
* For binary exclusive classes with provided label: F1 for label
* For 2+ exclusive classes: F1 macro average
* For multilabel (not exclusive): ROC AUC macro average (currently
relying on sklearn)
* Provide user info on textcat evaluation settings, potential
incompatibilities
* Provide pipeline to Scorer in `Language.evaluate` for textcat config
* Customize train CLI output to include only metrics relevant to current
pipeline
* Add textcat evaluation to evaluate CLI
* Fix handling of unset arguments and config params
Fix handling of unset arguments and model confiug parameters in Scorer
initialization.
* Temporarily add sklearn requirement
* Remove sklearn version number
* Improve Scorer handling of models without textcats
* Fixing Scorer handling of models without textcats
* Update Scorer output for python 2.7
* Modify inf in Scorer for python 2.7
* Auto-format
Also make small adjustments to make auto-formatting with black easier and produce nicer results
* Move error message to Errors
* Update documentation
* Add cats to annotation JSON format [ci skip]
* Fix tpl flag and docs [ci skip]
* Switch to internal roc_auc_score
Switch to internal `roc_auc_score()` adapted from scikit-learn.
* Add AUCROCScore tests and improve errors/warnings
* Add tests for AUCROCScore and roc_auc_score
* Add missing error for only positive/negative values
* Remove unnecessary warnings and errors
* Make reduced roc_auc_score functions private
Because most of the checks and warnings have been stripped for the
internal functions and access is only intended through `ROCAUCScore`,
make the functions for roc_auc_score adapted from scikit-learn private.
* Check that data corresponds with multilabel flag
Check that the training instances correspond with the multilabel flag,
adding the multilabel flag if required.
* Add textcat score to early stopping check
* Add more checks to debug-data for textcat
* Add example training data for textcat
* Add more checks to textcat train CLI
* Check configuration when extending base model
* Fix typos
* Update textcat example data
* Provide licensing details and licenses for data
* Remove two labels with no positive instances from jigsaw-toxic-comment
data.
Co-authored-by: Ines Montani <ines@ines.io>
|
||
|---|---|---|
| .. | ||
| CC_BY-SA-3.0.txt | ||
| CC_BY-SA-4.0.txt | ||
| CC0.txt | ||
| cooking.json | ||
| cooking.jsonl | ||
| jigsaw-toxic-comment.json | ||
| jigsaw-toxic-comment.jsonl | ||
| README.md | ||
| textcatjsonl_to_trainjson.py | ||
Examples of textcat training data
spacy JSON training files were generated from JSONL with:
python textcatjsonl_to_trainjson.py -m en file.jsonl .
cooking.json is an example with mutually-exclusive classes with two labels:
bakingnot_baking
jigsaw-toxic-comment.json is an example with multiple labels per instance:
insultobscenesevere_toxictoxic
Data Sources
cooking.jsonl: https://cooking.stackexchange.com. The meta IDs link to the original question ashttps://cooking.stackexchange.com/questions/ID, e.g.,https://cooking.stackexchange.com/questions/2for the first instance.jigsaw-toxic-comment.jsonl: Jigsaw Toxic Comments Classification Challenge
Data Licenses
cooking.jsonl: CC BY-SA 4.0 (CC_BY-SA-4.0.txt)jigsaw-toxic-comment.jsonl:- text: CC BY-SA 3.0 (
CC_BY-SA-3.0.txt) - annotation: CC0 (
CC0.txt)
- text: CC BY-SA 3.0 (