explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-07-14 18:22:27 +03:00
Code Issues Packages Projects Releases Wiki Activity

Add size increase info about vocab and stringstore

Browse Source
This commit is contained in:
thomashacker 2023-03-15 13:16:13 +01:00
parent 520279ff7c
commit 17f2dfdd70
2 changed files with 14 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
website/docs/api/stringstore.mdx
View File

@ -8,6 +8,13 @@ Look up strings by 64-bit hashes. As of v2.0, spaCy uses hash values instead of
integer IDs. This ensures that strings always map to the same ID, even from integer IDs. This ensures that strings always map to the same ID, even from
different `StringStores`. different `StringStores`.
<Infobox variant="warning">
Please note that the `StringStore` size is not static and increases as texts are
processed and new tokens are seen.
</Infobox>
## StringStore.\_\_init\_\_ {id="init",tag="method"} ## StringStore.\_\_init\_\_ {id="init",tag="method"}
Create the `StringStore`. Create the `StringStore`.

7
website/docs/api/vocab.mdx
View File

@ -10,6 +10,13 @@ The `Vocab` object provides a lookup table that allows you to access
[`StringStore`](/api/stringstore). It also owns underlying C-data that is shared [`StringStore`](/api/stringstore). It also owns underlying C-data that is shared
between `Doc` objects. between `Doc` objects.
<Infobox variant="warning">
Please note that the `Vocab` size is not static and increases as texts are
processed and new tokens are seen.
</Infobox>
## Vocab.\_\_init\_\_ {id="init",tag="method"} ## Vocab.\_\_init\_\_ {id="init",tag="method"}
Create the vocabulary. Create the vocabulary.