mirror of
https://github.com/explosion/spaCy.git
synced 2024-11-15 14:17:58 +03:00
28299644fc
* Speed up the StateC::L feature function This function gets the n-th most-recent left-arc with a particular head. Before this change, StateC::L would construct a vector of all left-arcs with the given head and then pick the n-th most recent from that vector. Since the number of left-arcs strongly correlates with the doc length and the feature is constructed for every transition, this can make transition-parsing quadratic. With this change StateC::L: - Searches left-arcs backwards. - Stops early when the n-th matching transition is found. - Does not construct a vector (reducing memory pressure). This change doesn't avoid the linear search when the transition that is queried does not occur in the left-arcs. Regardless, performance is improved quite a bit with very long docs: Before: N Time 400 3.3 800 5.4 1600 11.6 3200 30.7 After: N Time 400 3.2 800 5.0 1600 9.5 3200 23.2 We can probably do better with more tailored data structures, but I first wanted to make a low-impact PR. Found while investigating #9858. * StateC::L: simplify loop |
||
|---|---|---|
| .. | ||
| __init__.pxd | ||
| __init__.py | ||
| _beam_utils.pxd | ||
| _beam_utils.pyx | ||
| _state.pxd | ||
| _state.pyx | ||
| arc_eager.pxd | ||
| arc_eager.pyx | ||
| ner.pxd | ||
| ner.pyx | ||
| nonproj.pxd | ||
| nonproj.pyx | ||
| stateclass.pxd | ||
| stateclass.pyx | ||
| transition_system.pxd | ||
| transition_system.pyx | ||