Add size increase info about vocab and stringstore

This commit is contained in:
thomashacker 2023-03-15 13:16:13 +01:00
parent 520279ff7c
commit 17f2dfdd70
2 changed files with 14 additions and 0 deletions

View File

@ -8,6 +8,13 @@ Look up strings by 64-bit hashes. As of v2.0, spaCy uses hash values instead of
integer IDs. This ensures that strings always map to the same ID, even from integer IDs. This ensures that strings always map to the same ID, even from
different `StringStores`. different `StringStores`.
<Infobox variant="warning">
Please note that the `StringStore` size is not static and increases as texts are
processed and new tokens are seen.
</Infobox>
## StringStore.\_\_init\_\_ {id="init",tag="method"} ## StringStore.\_\_init\_\_ {id="init",tag="method"}
Create the `StringStore`. Create the `StringStore`.

View File

@ -10,6 +10,13 @@ The `Vocab` object provides a lookup table that allows you to access
[`StringStore`](/api/stringstore). It also owns underlying C-data that is shared [`StringStore`](/api/stringstore). It also owns underlying C-data that is shared
between `Doc` objects. between `Doc` objects.
<Infobox variant="warning">
Please note that the `Vocab` size is not static and increases as texts are
processed and new tokens are seen.
</Infobox>
## Vocab.\_\_init\_\_ {id="init",tag="method"} ## Vocab.\_\_init\_\_ {id="init",tag="method"}
Create the vocabulary. Create the vocabulary.