spaCy/spacy/lang.pxd

from libc.stdint cimport uint32_t
from libc.stdint cimport uint64_t
from spacy.word cimport Lexeme


cdef class Lexicon:
    cpdef Lexeme lookup(self, unicode string)
    
    cdef dict _dict
    
    cdef list _string_features
    cdef list _flag_features


cdef class Language:
    cdef unicode name
    cdef dict cache
    cpdef readonly Lexicon lexicon

    cpdef list tokenize(self, unicode text)
    cpdef Lexeme lookup(self, unicode text)

    cdef list _tokenize(self, unicode string)
    cdef list _split(self, unicode string)
    cdef int _split_one(self, unicode word)
* Switch to 32bit hash for strings 2014-08-03 00:51:52 +04:00			`from libc.stdint cimport uint32_t`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00			`from libc.stdint cimport uint64_t`
* More refactoring 2014-08-25 18:42:22 +04:00			`from spacy.word cimport Lexeme`
* Refactor for string view features. Working on setting up flags and enums. 2014-07-07 18:58:48 +04:00
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef class Lexicon:`
			`cpdef Lexeme lookup(self, unicode string)`
* Docs coming together 2014-08-29 03:59:23 +04:00
			`cdef dict _dict`

			`cdef list _string_features`
			`cdef list _flag_features`
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00

* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cdef class Language:`
* Docs coming together 2014-08-29 03:59:23 +04:00			`cdef unicode name`
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef dict cache`
			`cpdef readonly Lexicon lexicon`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00
* Moving to Word objects in place of the Lexeme struct. 2014-08-22 19:28:23 +04:00			`cpdef list tokenize(self, unicode text)`
* Docs coming together 2014-08-29 03:59:23 +04:00			`cpdef Lexeme lookup(self, unicode text)`
* Refactoring tokenizer 2014-08-16 05:22:03 +04:00
* Refactoring with Lexeme as a class now compiles. Basic design seems to work 2014-08-27 19:15:39 +04:00			`cdef list _tokenize(self, unicode string)`
* Docs coming together 2014-08-29 03:59:23 +04:00			`cdef list _split(self, unicode string)`
			`cdef int _split_one(self, unicode word)`