mirror of
https://github.com/explosion/spaCy.git
synced 2024-09-21 03:19:13 +03:00
563f46f026
The TextCategorizer class is supposed to support multi-label text classification, and allow training data to contain missing values. For this to work, the gradient of the loss should be 0 when labels are missing. Instead, there was no way to actually denote "missing" in the GoldParse class, and so the TextCategorizer class treated the label set within gold.cats as complete. To fix this, we change GoldParse.cats to be a dict instead of a list. The GoldParse.cats dict should map to floats, with 1. denoting 'present' and 0. denoting 'absent'. Gradients are zeroed for categories absent from the gold.cats dict. A nice bonus is that you can also set values between 0 and 1 for partial membership. You can also set numeric values, if you're using a text classification model that uses an appropriate loss function. Unfortunately this is a breaking change; although the functionality was only recently introduced and hasn't been properly documented yet. I've updated the example script accordingly. |
||
|---|---|---|
| .. | ||
| inventory_count | ||
| keras_parikh_entailment | ||
| training | ||
| _handler.py | ||
| deep_learning_keras.py | ||
| get_parse_subregions.py | ||
| information_extraction.py | ||
| matcher_example.py | ||
| nn_text_class.py | ||
| parallel_parse.py | ||
| phrase_matcher.py | ||
| pos_tag.py | ||
| README.md | ||
| twitter_filter.py | ||
| vectors_fast_text.py | ||
spaCy examples
The examples are Python scripts with well-behaved command line interfaces. For a full list of spaCy tutorials and code snippets, see the documentation.
How to run an example
For example, to run the nn_text_class.py script, do:
$ python examples/nn_text_class.py
usage: nn_text_class.py [-h] [-d 3] [-H 300] [-i 5] [-w 40000] [-b 24]
[-r 0.3] [-p 1e-05] [-e 0.005]
data_dir
nn_text_class.py: error: too few arguments
You can print detailed help with the -h argument.
While we try to keep the examples up to date, they are not currently exercised by the test suite, as some of them require significant data downloads or take time to train. If you find that an example is no longer running, please tell us! We know there's nothing worse than trying to figure out what you're doing wrong, and it turns out your code was never the problem.