spaCy/spacy/tokens/doc.pxd

from cymem.cymem cimport Pool
cimport numpy as np
from preshed.counter cimport PreshCounter

from ..vocab cimport Vocab
from ..structs cimport TokenC, LexemeC
from ..typedefs cimport attr_t
from ..attrs cimport attr_id_t

from spacy.syntax.iterators cimport DocIterator


cdef attr_t get_token_attr(const TokenC* token, attr_id_t feat_name) nogil


ctypedef const LexemeC* const_Lexeme_ptr
ctypedef const TokenC* const_TokenC_ptr

ctypedef fused LexemeOrToken:
    const_Lexeme_ptr
    const_TokenC_ptr


cdef int token_by_start(const TokenC* tokens, int length, int start_char) except -2


cdef int token_by_end(const TokenC* tokens, int length, int end_char) except -2


cdef class Doc:
    cdef readonly Pool mem
    cdef readonly Vocab vocab

    cdef public object _vector
    cdef public object _vector_norm

    cdef TokenC* c

    cdef public bint is_tagged
    cdef public bint is_parsed

    cdef public list _py_tokens

    cdef int length
    cdef int max_length

    cdef DocIterator noun_chunks_iterator

    cdef int push_back(self, LexemeOrToken lex_or_tok, bint trailing_space) except -1

    cpdef np.ndarray to_array(self, object features)

    cdef void set_parse(self, const TokenC* parsed) nogil
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00			`from cymem.cymem cimport Pool`
			`cimport numpy as np`
			`from preshed.counter cimport PreshCounter`

			`from ..vocab cimport Vocab`
			`from ..structs cimport TokenC, LexemeC`
* Gazetteer stuff working, now need to wire up to API 2015-08-06 01:35:40 +03:00			`from ..typedefs cimport attr_t`
			`from ..attrs cimport attr_id_t`

add baseclass DocIterator for iterators over documents add classes for English and German noun chunks the respective iterators are set for the document when created by the parser as they depend on the annotation scheme of the parsing model 2016-03-16 17:53:35 +03:00			`from spacy.syntax.iterators cimport DocIterator`

* Gazetteer stuff working, now need to wire up to API 2015-08-06 01:35:40 +03:00
			`cdef attr_t get_token_attr(const TokenC* token, attr_id_t feat_name) nogil`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00

			`ctypedef const LexemeC* const_Lexeme_ptr`
* More work on language-generic parsing 2015-08-28 03:02:33 +03:00			`ctypedef const TokenC* const_TokenC_ptr`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00
			`ctypedef fused LexemeOrToken:`
			`const_Lexeme_ptr`
* More work on language-generic parsing 2015-08-28 03:02:33 +03:00			`const_TokenC_ptr`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00

* Rework the Span-merge patch, to avoid extending the interface of Doc, and avoid virtualizing the Span.start and Span.end indices, to keep Span usage efficient 2015-11-07 00:55:34 +03:00			`cdef int token_by_start(const TokenC* tokens, int length, int start_char) except -2`


			`cdef int token_by_end(const TokenC* tokens, int length, int end_char) except -2`


* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00			`cdef class Doc:`
* Make mem and vocab python-visible in Doc 2015-07-28 21:46:59 +03:00			`cdef readonly Pool mem`
			`cdef readonly Vocab vocab`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00
* Try giving Doc and Span objects vector and vector_norm attributes, and .similarity functions. Turns out to be bad idea. 2015-09-17 04:50:11 +03:00			`cdef public object _vector`
			`cdef public object _vector_norm`

* Rename Doc.data to Doc.c 2015-11-03 16:15:14 +03:00			`cdef TokenC* c`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00
			`cdef public bint is_tagged`
			`cdef public bint is_parsed`

* Restore _py_tokens cache, to handle orphan tokens. 2015-07-13 23:28:10 +03:00			`cdef public list _py_tokens`

* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00			`cdef int length`
			`cdef int max_length`

add baseclass DocIterator for iterators over documents add classes for English and German noun chunks the respective iterators are set for the document when created by the parser as they depend on the annotation scheme of the parsing model 2016-03-16 17:53:35 +03:00			`cdef DocIterator noun_chunks_iterator`

* Refactor tokenizer, to set the 'spacy' field on TokenC instead of passing a string 2015-07-13 22:46:02 +03:00			`cdef int push_back(self, LexemeOrToken lex_or_tok, bint trailing_space) except -1`
* Add spacy/tokens/doc.pyx, for Doc class in its own file 2015-07-13 20:58:26 +03:00
			`cpdef np.ndarray to_array(self, object features)`

* Make set_parse nogil 2016-01-30 22:27:52 +03:00			`cdef void set_parse(self, const TokenC* parsed) nogil`