mirror of
https://github.com/explosion/spaCy.git
synced 2025-11-19 09:16:01 +03:00
c362006cb9
Data in the JSON format is split into sentences, and each sentence is saved with is_sent_start flags. Currently the flags are 1 for the first token and 0 for the others. When deserialized this results in a pattern of True, None, None, None... which makes single-sentence documents look as though they haven't had sentence boundaries set. Since items saved in JSON format have been split into sentences already, the is_sent_start values should all be True or False. |
||
|---|---|---|
| .. | ||
| converters | ||
| __init__.pxd | ||
| __init__.py | ||
| align.pyx | ||
| alignment.py | ||
| augment.py | ||
| batchers.py | ||
| corpus.py | ||
| example.pxd | ||
| example.pyx | ||
| gold_io.pyx | ||
| initialize.py | ||
| iob_utils.py | ||
| loggers.py | ||
| loop.py | ||
| pretrain.py | ||