spaCy/website/docs/api
Daniël de Kok efdbb722c5
Store activations in Docs when save_activations is enabled (#11002)
* Store activations in Doc when `store_activations` is enabled

This change adds the new `activations` attribute to `Doc`. This
attribute can be used by trainable pipes to store their activations,
probabilities, and guesses for downstream users.

As an example, this change modifies the `tagger` and `senter` pipes to
add an `store_activations` option. When this option is enabled, the
probabilities and guesses are stored in `set_annotations`.

* Change type of `store_activations` to `Union[bool, List[str]]`

When the value is:

- A bool: all activations are stored when set to `True`.
- A List[str]: the activations named in the list are stored

* Formatting fixes in Tagger

* Support store_activations in spancat and morphologizer

* Make Doc.activations type visible to MyPy

* textcat/textcat_multilabel: add store_activations option

* trainable_lemmatizer/entity_linker: add store_activations option

* parser/ner: do not currently support returning activations

* Extend tagger and senter tests

So that they, like the other tests, also check that we get no
activations if no activations were requested.

* Document `Doc.activations` and `store_activations` in the relevant pipes

* Start errors/warnings at higher numbers to avoid merge conflicts

Between the master and v4 branches.

* Add `store_activations` to docstrings.

* Replace store_activations setter by set_store_activations method

Setters that take a different type than what the getter returns are still
problematic for MyPy. Replace the setter by a method, so that type inference
works everywhere.

* Use dict comprehension suggested by @svlandeg

* Revert "Use dict comprehension suggested by @svlandeg"

This reverts commit 6e7b958f70.

* EntityLinker: add type annotations to _add_activations

* _store_activations: make kwarg-only, remove doc_scores_lens arg

* set_annotations: add type annotations

* Apply suggestions from code review

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* TextCat.predict: return dict

* Make the `TrainablePipe.store_activations` property a bool

This means that we can also bring back `store_activations` setter.

* Remove `TrainablePipe.activations`

We do not need to enumerate the activations anymore since `store_activations` is
`bool`.

* Add type annotations for activations in predict/set_annotations

* Rename `TrainablePipe.store_activations` to `save_activations`

* Error E1400 is not used anymore

This error was used when activations were still `Union[bool, List[str]]`.

* Change wording in API docs after store -> save change

* docs: tag (save_)activations as new in spaCy 4.0

* Fix copied line in morphologizer activations test

* Don't train in any test_save_activations test

* Rename activations

- "probs" -> "probabilities"
- "guesses" -> "label_ids", except in the edit tree lemmatizer, where
  "guesses" -> "tree_ids".

* Remove unused W400 warning.

This warning was used when we still allowed the user to specify
which activations to save.

* Formatting fixes

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Replace "kb_ids" by a constant

* spancat: replace a cast by an assertion

* Fix EOF spacing

* Fix comments in test_save_activations tests

* Do not set RNG seed in activation saving tests

* Revert "spancat: replace a cast by an assertion"

This reverts commit 0bd5730d16.

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-09-13 09:51:12 +02:00
..
architectures.md Remove simply (#11017) 2022-06-27 09:45:22 +02:00
attributeruler.md Rename modules for consistency (#11286) 2022-08-10 11:44:05 +02:00
attributes.md Add API docs for token attribute symbols (#10836) 2022-06-23 08:16:38 +02:00
cli.md Add a way to get the URL to download a pipeline to the CLI (#11175) 2022-09-02 11:58:21 +02:00
corpus.md Remove NBSP's across tables in the docs (#10842) 2022-05-25 09:48:39 +02:00
cython-classes.md Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
cython-structs.md Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
cython.md Update docs [ci skip] 2020-09-12 17:05:10 +02:00
data-formats.md adding spans to doc_annotation in Example.to_dict (#11261) 2022-08-05 12:26:38 +02:00
dependencymatcher.md add additional REL_OP (#10371) 2022-07-27 13:16:44 +02:00
dependencyparser.md Update docs for pipeline initialize() methods (#11221) 2022-08-03 16:53:02 +02:00
doc.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
docbin.md Fix point typo on docbin docs (#9097) 2021-08-31 10:55:44 +02:00
edittreelemmatizer.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
entitylinker.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
entityrecognizer.md Update docs for pipeline initialize() methods (#11221) 2022-08-03 16:53:02 +02:00
entityruler.md Rename modules for consistency (#11286) 2022-08-10 11:44:05 +02:00
example.md Extend score_spans for overlapping & non-labeled spans (#7209) 2021-04-08 12:19:17 +02:00
index.md Update v3 docs 2020-07-03 16:48:21 +02:00
kb.md Tidy up docs 2021-06-28 12:08:15 +02:00
language.md Allow string argument for disable/enable/exclude (#11406) 2022-08-31 09:02:34 +02:00
legacy.md Add ConsoleLogger.v2 (#11214) 2022-08-29 10:23:05 +02:00
lemmatizer.md Switch ru and uk lemmatizers to pymorphy3 (#11345) 2022-08-22 11:27:14 +02:00
lexeme.md fix 's typo's across code base (#8384) 2021-06-15 10:57:08 +02:00
lookups.md Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
matcher.md Update/remove old Matcher syntax (#11370) 2022-08-30 15:40:31 +02:00
morphologizer.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
morphology.md Document Assigned Attributes of Pipeline Components (#9041) 2021-09-01 12:09:39 +02:00
phrasematcher.md Update/remove old Matcher syntax (#11370) 2022-08-30 15:40:31 +02:00
pipe.md Document scorers in registry and components from #8766 (#8929) 2021-08-12 12:50:03 +02:00
pipeline-functions.md add doc cleaner to menu (#10862) 2022-05-30 08:51:19 +02:00
scorer.md Add micro PRF for morph scoring (#9546) 2021-10-29 10:29:29 +02:00
sentencerecognizer.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
sentencizer.md Update overwrite and scorer in API docs (#9384) 2021-10-11 10:35:07 +02:00
span.md Make Span/Doc.ents more consistent for ent_kb_id and ent_id (#11328) 2022-08-22 20:28:57 +02:00
spancategorizer.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
spangroup.md SpanGroup(s)-related optimizations (#11380) 2022-08-31 09:03:20 +02:00
spanruler.md the 'new' indicator wants a 'number' (#10997) 2022-06-21 22:01:16 +02:00
stringstore.md Fix misspelt keyword in StringStore example 2022-05-29 10:49:19 +01:00
tagger.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
textcategorizer.md Store activations in Docs when save_activations is enabled (#11002) 2022-09-13 09:51:12 +02:00
tok2vec.md Update docs for pipeline initialize() methods (#11221) 2022-08-03 16:53:02 +02:00
token.md Make Span/Doc.ents more consistent for ent_kb_id and ent_id (#11328) 2022-08-22 20:28:57 +02:00
tokenizer.md Add tokenizer option to allow Matcher handling for all rules (#10452) 2022-03-24 13:21:32 +01:00
top-level.md Allow string argument for disable/enable/exclude (#11406) 2022-08-31 09:02:34 +02:00
transformer.md Update docs for pipeline initialize() methods (#11221) 2022-08-03 16:53:02 +02:00
vectors.md Docs for v3.3 (#10628) 2022-04-28 14:09:35 +02:00
vocab.md Add vector deduplication (#10551) 2022-03-30 08:54:23 +02:00