spaCy/spacy/lang.pxd

from libc.stdint cimport uint32_t
from libc.stdint cimport uint64_t
from spacy.word cimport Lexeme


cdef class Language:
    cdef object name
    cdef dict blobs
    cdef dict lexicon

    cpdef list tokenize(self, unicode text)

    cdef Word lookup(self, unicode string)
    cdef list lookup_chunk(self, unicode chunk)
    
    cdef list new_chunk(self, unicode string, list substrings)
    cdef Word new_lexeme(self, unicode lex)
    
    cpdef list find_substrings(self, unicode chunk)
    cdef int find_split(self, unicode word)
* Switch to 32bit hash for strings 2014-08-03 00:51:52 +04:00			`from libc.stdint cimport uint32_t`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00			`from libc.stdint cimport uint64_t`
* More refactoring 2014-08-25 18:42:22 +04:00			`from spacy.word cimport Lexeme`
* Refactor for string view features. Working on setting up flags and enums. 2014-07-07 18:58:48 +04:00
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00
			`cdef class Language:`
			`cdef object name`
* More refactoring 2014-08-25 18:42:22 +04:00			`cdef dict blobs`
			`cdef dict lexicon`
* Reorganized, moving language-independent stuff to spacy. The functions in spacy ask for the dictionaries and split function on input, but the language-specific modules are curried versions that use the globals 2014-07-07 06:21:06 +04:00
* Moving to Word objects in place of the Lexeme struct. 2014-08-22 19:28:23 +04:00			`cpdef list tokenize(self, unicode text)`
* Refactoring tokenizer 2014-08-16 05:22:03 +04:00
* Moving to Word objects in place of the Lexeme struct. 2014-08-22 19:28:23 +04:00			`cdef Word lookup(self, unicode string)`
			`cdef list lookup_chunk(self, unicode chunk)`
* Refactoring tokenizer 2014-08-16 05:22:03 +04:00
* Moving to Word objects in place of the Lexeme struct. 2014-08-22 19:28:23 +04:00			`cdef list new_chunk(self, unicode string, list substrings)`
			`cdef Word new_lexeme(self, unicode lex)`
* Refactoring tokenizer 2014-08-16 05:22:03 +04:00
* Broken version being refactored for docs 2014-08-20 15:39:39 +04:00			`cpdef list find_substrings(self, unicode chunk)`
* Refactor spacy so that chunks return arrays of lexemes, so that there is properly one lexeme per word. 2014-08-18 21:14:00 +04:00			`cdef int find_split(self, unicode word)`