spaCy/spacy/tokens.pxd

from libcpp.vector cimport vector
from spacy.lexeme cimport LexID
from spacy.lexeme cimport Lexeme

from cython.operator cimport dereference as deref
from spacy.spacy cimport Language


cdef class Tokens:
    cdef Language lang
    cdef vector[LexID]* vctr
    cdef size_t length
    
    cpdef int append(self, LexID token)
    cpdef int extend(self, Tokens other) except -1
    
    cpdef object group_by(self, size_t attr)
    cpdef dict count_by(self, size_t attr)
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`from libcpp.vector cimport vector`
* Fixed group_by, removed idea of general attr_of function. 2014-08-22 02:02:37 +04:00			`from spacy.lexeme cimport LexID`
* Refactor spacy so that chunks return arrays of lexemes, so that there is properly one lexeme per word. 2014-08-18 21:14:00 +04:00			`from spacy.lexeme cimport Lexeme`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00
			`from cython.operator cimport dereference as deref`
			`from spacy.spacy cimport Language`


			`cdef class Tokens:`
			`cdef Language lang`
* Fixed group_by, removed idea of general attr_of function. 2014-08-22 02:02:37 +04:00			`cdef vector[LexID]* vctr`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cdef size_t length`

* Fixed group_by, removed idea of general attr_of function. 2014-08-22 02:02:37 +04:00			`cpdef int append(self, LexID token)`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cpdef int extend(self, Tokens other) except -1`

* Broken version being refactored for docs 2014-08-20 15:39:39 +04:00			`cpdef object group_by(self, size_t attr)`
			`cpdef dict count_by(self, size_t attr)`