spaCy/spacy/pipeline/_parser_internals/stateclass.pxd

from cymem.cymem cimport Pool

from ...structs cimport TokenC, SpanC
from ...typedefs cimport attr_t
from ...tokens.doc cimport Doc

from ._state cimport StateC


cdef class StateClass:
    cdef StateC* c
    cdef readonly Doc doc
    cdef int _borrowed

    @staticmethod
    cdef inline StateClass borrow(StateC* ptr, Doc doc):
        cdef StateClass self = StateClass()
        self.c = ptr
        self._borrowed = 1
        self.doc = doc
        return self

    @staticmethod
    cdef inline StateClass init_offset(const TokenC* sent, int length, int
                                       offset):
        cdef StateClass self = StateClass()
        self.c = new StateC(sent, length)
        self.c.offset = offset
        return self
* Add StateClass, to replace/refactor the mess in _state 2015-06-09 02:39:54 +03:00			`from cymem.cymem cimport Pool`

The Parser is now a Pipe (2) (#5844) * moving syntax folder to _parser_internals * moving nn_parser and transition_system * move nn_parser and transition_system out of internals folder * moving nn_parser code into transition_system file * rename transition_system to transition_parser * moving parser_model and _state to ml * move _state back to internals * The Parser now inherits from Pipe! * small code fixes * removing unnecessary imports * remove link_vectors_to_models * transition_system to internals folder * little bit more cleanup * newlines 2020-07-31 00:30:54 +03:00			`from ...structs cimport TokenC, SpanC`
			`from ...typedefs cimport attr_t`
Add beam_parser and beam_ner components for v3 (#6369) * Get basic beam tests working * Get basic beam tests working * Compile _beam_utils * Remove prints * Test beam density * Beam parser seems to train * Draft beam NER * Upd beam * Add hypothesis as dev dependency * Implement missing is-gold-parse method * Implement early update * Fix state hashing * Fix test * Fix test * Default to non-beam in parser constructor * Improve oracle for beam * Start refactoring beam * Update test * Refactor beam * Update nn * Refactor beam and weight by cost * Update ner beam settings * Update test * Add __init__.pxd * Upd test * Fix test * Upd test * Fix test * Remove ring buffer history from StateC * WIP change arc-eager transitions * Add state tests * Support ternary sent start values * Fix arc eager * Fix NER * Pass oracle cut size for beam * Fix ner test * Fix beam * Improve StateC.clone * Improve StateClass.borrow * Work directly with StateC, not StateClass * Remove print statements * Fix state copy * Improve state class * Refactor parser oracles * Fix arc eager oracle * Fix arc eager oracle * Use a vector to implement the stack * Refactor state data structure * Fix alignment of sent start * Add get_aligned_sent_starts method * Add test for ae oracle when bad sentence starts * Fix sentence segment handling * Avoid Reduce that inserts illegal sentence * Update preset SBD test * Fix test * Remove prints * Fix sent starts in Example * Improve python API of StateClass * Tweak comments and debug output of arc eager * Upd test * Fix state test * Fix state test 2020-12-13 04:08:32 +03:00			`from ...tokens.doc cimport Doc`
* Add StateClass, to replace/refactor the mess in _state 2015-06-09 02:39:54 +03:00
* Shadow StateClass with StateC, to start proxying 2016-02-01 03:16:14 +03:00			`from ._state cimport StateC`

* Add StateClass, to replace/refactor the mess in _state 2015-06-09 02:39:54 +03:00
			`cdef class StateClass:`
* Shadow StateClass with StateC, to start proxying 2016-02-01 03:16:14 +03:00			`cdef StateC* c`
Add beam_parser and beam_ner components for v3 (#6369) * Get basic beam tests working * Get basic beam tests working * Compile _beam_utils * Remove prints * Test beam density * Beam parser seems to train * Draft beam NER * Upd beam * Add hypothesis as dev dependency * Implement missing is-gold-parse method * Implement early update * Fix state hashing * Fix test * Fix test * Default to non-beam in parser constructor * Improve oracle for beam * Start refactoring beam * Update test * Refactor beam * Update nn * Refactor beam and weight by cost * Update ner beam settings * Update test * Add __init__.pxd * Upd test * Fix test * Upd test * Fix test * Remove ring buffer history from StateC * WIP change arc-eager transitions * Add state tests * Support ternary sent start values * Fix arc eager * Fix NER * Pass oracle cut size for beam * Fix ner test * Fix beam * Improve StateC.clone * Improve StateClass.borrow * Work directly with StateC, not StateClass * Remove print statements * Fix state copy * Improve state class * Refactor parser oracles * Fix arc eager oracle * Fix arc eager oracle * Use a vector to implement the stack * Refactor state data structure * Fix alignment of sent start * Add get_aligned_sent_starts method * Add test for ae oracle when bad sentence starts * Fix sentence segment handling * Avoid Reduce that inserts illegal sentence * Update preset SBD test * Fix test * Remove prints * Fix sent starts in Example * Improve python API of StateClass * Tweak comments and debug output of arc eager * Upd test * Fix state test * Fix state test 2020-12-13 04:08:32 +03:00			`cdef readonly Doc doc`
Fix memory leak in beam parser 2017-11-14 04:11:40 +03:00			`cdef int _borrowed`
* Add StateClass, to replace/refactor the mess in _state 2015-06-09 02:39:54 +03:00
* Greedy parsing working with new StateClass. Beam parsing broken 2015-06-10 05:20:23 +03:00			`@staticmethod`
Add beam_parser and beam_ner components for v3 (#6369) * Get basic beam tests working * Get basic beam tests working * Compile _beam_utils * Remove prints * Test beam density * Beam parser seems to train * Draft beam NER * Upd beam * Add hypothesis as dev dependency * Implement missing is-gold-parse method * Implement early update * Fix state hashing * Fix test * Fix test * Default to non-beam in parser constructor * Improve oracle for beam * Start refactoring beam * Update test * Refactor beam * Update nn * Refactor beam and weight by cost * Update ner beam settings * Update test * Add __init__.pxd * Upd test * Fix test * Upd test * Fix test * Remove ring buffer history from StateC * WIP change arc-eager transitions * Add state tests * Support ternary sent start values * Fix arc eager * Fix NER * Pass oracle cut size for beam * Fix ner test * Fix beam * Improve StateC.clone * Improve StateClass.borrow * Work directly with StateC, not StateClass * Remove print statements * Fix state copy * Improve state class * Refactor parser oracles * Fix arc eager oracle * Fix arc eager oracle * Use a vector to implement the stack * Refactor state data structure * Fix alignment of sent start * Add get_aligned_sent_starts method * Add test for ae oracle when bad sentence starts * Fix sentence segment handling * Avoid Reduce that inserts illegal sentence * Update preset SBD test * Fix test * Remove prints * Fix sent starts in Example * Improve python API of StateClass * Tweak comments and debug output of arc eager * Upd test * Fix state test * Fix state test 2020-12-13 04:08:32 +03:00			`cdef inline StateClass borrow(StateC* ptr, Doc doc):`
Improve integration of NN parser, to support unified training API 2017-05-15 22:46:08 +03:00			`cdef StateClass self = StateClass()`
Fix memory leak in beam parser 2017-11-14 04:11:40 +03:00			`self.c = ptr`
			`self._borrowed = 1`
Add beam_parser and beam_ner components for v3 (#6369) * Get basic beam tests working * Get basic beam tests working * Compile _beam_utils * Remove prints * Test beam density * Beam parser seems to train * Draft beam NER * Upd beam * Add hypothesis as dev dependency * Implement missing is-gold-parse method * Implement early update * Fix state hashing * Fix test * Fix test * Default to non-beam in parser constructor * Improve oracle for beam * Start refactoring beam * Update test * Refactor beam * Update nn * Refactor beam and weight by cost * Update ner beam settings * Update test * Add __init__.pxd * Upd test * Fix test * Upd test * Fix test * Remove ring buffer history from StateC * WIP change arc-eager transitions * Add state tests * Support ternary sent start values * Fix arc eager * Fix NER * Pass oracle cut size for beam * Fix ner test * Fix beam * Improve StateC.clone * Improve StateClass.borrow * Work directly with StateC, not StateClass * Remove print statements * Fix state copy * Improve state class * Refactor parser oracles * Fix arc eager oracle * Fix arc eager oracle * Use a vector to implement the stack * Refactor state data structure * Fix alignment of sent start * Add get_aligned_sent_starts method * Add test for ae oracle when bad sentence starts * Fix sentence segment handling * Avoid Reduce that inserts illegal sentence * Update preset SBD test * Fix test * Remove prints * Fix sent starts in Example * Improve python API of StateClass * Tweak comments and debug output of arc eager * Upd test * Fix state test * Fix state test 2020-12-13 04:08:32 +03:00			`self.doc = doc`
Fix memory leak in beam parser 2017-11-14 04:11:40 +03:00			`return self`

Improve integration of NN parser, to support unified training API 2017-05-15 22:46:08 +03:00			`@staticmethod`
			`cdef inline StateClass init_offset(const TokenC* sent, int length, int`
			`offset):`
			`cdef StateClass self = StateClass()`
			`self.c = new StateC(sent, length)`
			`self.c.offset = offset`
			`return self`