spaCy/spacy/lang.pxd

from libc.stdint cimport uint32_t
from libc.stdint cimport uint64_t
from spacy.word cimport Lexeme


cdef class Lexicon:
    cdef list string_features
    cdef list flag_features

    cdef dict _dict

    cpdef Lexeme lookup(self, unicode string)


cdef class Language:
    cdef object name
    cdef dict cache
    cpdef readonly Lexicon lexicon

    cpdef list tokenize(self, unicode text)

    cdef list _tokenize(self, unicode string)
    cpdef list _split(self, unicode string)
    cpdef int _split_one(self, unicode word)
* Switch to 32bit hash for strings 2014-08-03 00:51:52 +04:00			`from libc.stdint cimport uint32_t`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00			`from libc.stdint cimport uint64_t`
* More refactoring 2014-08-25 18:42:22 +04:00			`from spacy.word cimport Lexeme`
* Refactor for string view features. Working on setting up flags and enums. 2014-07-07 18:58:48 +04:00
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef class Lexicon:`
* Redesign proceeding 2014-08-28 21:45:09 +04:00			`cdef list string_features`
			`cdef list flag_features`
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00
* Redesign proceeding 2014-08-28 21:45:09 +04:00			`cdef dict _dict`
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00
			`cpdef Lexeme lookup(self, unicode string)`


* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cdef class Language:`
			`cdef object name`
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef dict cache`
			`cpdef readonly Lexicon lexicon`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00
* Moving to Word objects in place of the Lexeme struct. 2014-08-22 19:28:23 +04:00			`cpdef list tokenize(self, unicode text)`
* Refactoring tokenizer 2014-08-16 05:22:03 +04:00
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef list _tokenize(self, unicode string)`
			`cpdef list _split(self, unicode string)`
			`cpdef int _split_one(self, unicode word)`