spaCy/spacy/matcher/matcher.pxd

from libc.stdint cimport int32_t
from libcpp.vector cimport vector
from cymem.cymem cimport Pool

from ..vocab cimport Vocab
from ..typedefs cimport attr_t, hash_t
from ..structs cimport TokenC
from ..lexeme cimport attr_id_t


cdef enum action_t:
    REJECT = 0000
    MATCH = 1000
    ADVANCE = 0100
    RETRY = 0010
    RETRY_EXTEND = 0011
    RETRY_ADVANCE = 0110
    MATCH_EXTEND = 1001
    MATCH_REJECT = 2000
    MATCH_DOUBLE = 3000


cdef enum quantifier_t:
    ZERO
    ZERO_ONE
    ZERO_PLUS
    ONE
    ONE_PLUS
    FINAL_ID


cdef struct AttrValueC:
    attr_id_t attr
    attr_t value

cdef struct IndexValueC:
    int32_t index
    attr_t value

cdef struct TokenPatternC:
    AttrValueC* attrs
    int32_t* py_predicates
    IndexValueC* extra_attrs
    int32_t nr_attr
    int32_t nr_extra_attr
    int32_t nr_py
    quantifier_t quantifier
    hash_t key


cdef struct PatternStateC:
    TokenPatternC* pattern
    int32_t start
    int32_t length


cdef struct MatchC:
    attr_t pattern_id
    int32_t start
    int32_t length


cdef class Matcher:
    cdef Pool mem
    cdef vector[TokenPatternC*] patterns
    cdef readonly Vocab vocab
    cdef public object validate
    cdef public object _patterns
    cdef public object _callbacks
    cdef public object _filter
    cdef public object _extensions
    cdef public object _extra_predicates
    cdef public object _seen_attrs
💫 Break up large matcher.pyx (#3236) * Break up large matcher.pyx * Remove unused function 2019-02-07 11:42:25 +03:00			`from libc.stdint cimport int32_t`
			`from libcpp.vector cimport vector`
			`from cymem.cymem cimport Pool`

			`from ..vocab cimport Vocab`
			`from ..typedefs cimport attr_t, hash_t`
			`from ..structs cimport TokenC`
			`from ..lexeme cimport attr_id_t`


			`cdef enum action_t:`
			`REJECT = 0000`
			`MATCH = 1000`
			`ADVANCE = 0100`
			`RETRY = 0010`
			`RETRY_EXTEND = 0011`
			`RETRY_ADVANCE = 0110`
			`MATCH_EXTEND = 1001`
			`MATCH_REJECT = 2000`
adding double match for optional operator at the end (#4166) 2019-08-21 23:46:56 +03:00			`MATCH_DOUBLE = 3000`
💫 Break up large matcher.pyx (#3236) * Break up large matcher.pyx * Remove unused function 2019-02-07 11:42:25 +03:00

			`cdef enum quantifier_t:`
			`ZERO`
			`ZERO_ONE`
			`ZERO_PLUS`
			`ONE`
			`ONE_PLUS`
Identify final Matcher pattern node by quantifier (#6317) Modify the internal pattern representation in `Matcher` patterns to identify the final ID state using a unique quantifier rather than a combination of other attributes. It was insufficient to identify the final ID node based on an uninitialized `quantifier` (coincidentally being the same as the `ZERO`) with `nr_attr` as 0. (In addition, it was potentially bug-prone that `nr_attr` was set to 0 even though attrs were allocated.) In the case of `{"OP": "!"}` (a valid, if pointless, pattern), `nr_attr` is 0 and the quantifier is ZERO, so the previous methods for incrementing to the ID node at the end of the pattern weren't able to distinguish the final ID node from the `{"OP": "!"}` pattern. 2020-10-31 14:18:48 +03:00			`FINAL_ID`
💫 Break up large matcher.pyx (#3236) * Break up large matcher.pyx * Remove unused function 2019-02-07 11:42:25 +03:00

			`cdef struct AttrValueC:`
			`attr_id_t attr`
			`attr_t value`

			`cdef struct IndexValueC:`
			`int32_t index`
			`attr_t value`

			`cdef struct TokenPatternC:`
			`AttrValueC* attrs`
			`int32_t* py_predicates`
			`IndexValueC* extra_attrs`
			`int32_t nr_attr`
			`int32_t nr_extra_attr`
			`int32_t nr_py`
			`quantifier_t quantifier`
			`hash_t key`


			`cdef struct PatternStateC:`
			`TokenPatternC* pattern`
			`int32_t start`
			`int32_t length`


			`cdef struct MatchC:`
			`attr_t pattern_id`
			`int32_t start`
			`int32_t length`


			`cdef class Matcher:`
			`cdef Pool mem`
			`cdef vector[TokenPatternC*] patterns`
			`cdef readonly Vocab vocab`
Add better schemas and validation using Pydantic (#4831) * Remove unicode declarations * Remove Python 3.5 and 2.7 from CI * Don't require pathlib * Replace compat helpers * Remove OrderedDict * Use f-strings * Set Cython compiler language level * Fix typo * Re-add OrderedDict for Table * Update setup.cfg * Revert CONTRIBUTING.md * Add better schemas and validation using Pydantic * Revert lookups.md * Remove unused import * Update spacy/schemas.py Co-Authored-By: Sebastián Ramírez <tiangolo@gmail.com> * Various small fixes * Fix docstring Co-authored-by: Sebastián Ramírez <tiangolo@gmail.com> 2019-12-25 14:39:49 +03:00			`cdef public object validate`
💫 Break up large matcher.pyx (#3236) * Break up large matcher.pyx * Remove unused function 2019-02-07 11:42:25 +03:00			`cdef public object _patterns`
			`cdef public object _callbacks`
Option for returning only greedy matches (#5771) * add "greedy" option for match pattern * distinction between greedy FIRST or LONGEST * check for proper values, throw custom warning otherwise * unxfail one more test * add comment in docstring * add test that LONGEST also prefers first match if equal length * use c arrays for more efficient processing * rename 'greediness' to 'greedy' 2020-07-29 12:04:43 +03:00			`cdef public object _filter`
💫 Break up large matcher.pyx (#3236) * Break up large matcher.pyx * Remove unused function 2019-02-07 11:42:25 +03:00			`cdef public object _extensions`
			`cdef public object _extra_predicates`
Check for is_tagged/is_parsed for Matcher attrs (#4163) Check for relevant components in the pipeline when Matcher is called, similar to the checks for PhraseMatcher in #4105. * keep track of attributes seen in patterns * when Matcher is called on a Doc, check for is_tagged for LEMMA, TAG, POS and for is_parsed for DEP 2019-08-21 21:52:36 +03:00			`cdef public object _seen_attrs`