mirror of
https://github.com/explosion/spaCy.git
synced 2024-12-24 00:46:28 +03:00
72e4d3782a
The doc.retokenize() context manager wasn't resizing doc.tensor, leading to a mismatch between the number of tokens in the doc and the number of rows in the tensor. We fix this by deleting rows from the tensor. Merged spans are represented by the vector of their last token. * Add test for resizing doc.tensor when merging * Add test for resizing doc.tensor when merging. Closes #1963 * Update get_lca_matrix test for develop * Fix retokenize if tensor unset |
||
|---|---|---|
| .. | ||
| __init__.pxd | ||
| __init__.py | ||
| _retokenize.pyx | ||
| _serialize.py | ||
| doc.pxd | ||
| doc.pyx | ||
| span.pxd | ||
| span.pyx | ||
| token.pxd | ||
| token.pyx | ||
| underscore.py | ||