spaCy/spacy/ptb3.pxd

from libcpp.vector cimport vector

from spacy.spacy cimport StringHash
from spacy.spacy cimport Language
from spacy.spacy cimport Lexeme
from spacy.spacy cimport Lexeme_addr
from spacy.tokens cimport Tokens


cdef class EnglishPTB(Language):
    cdef int find_split(self, unicode word)
    

cdef EnglishPTB EN_PTB

cpdef Lexeme_addr lookup(unicode word) except 0
cpdef Tokens tokenize(unicode string)
cpdef unicode unhash(StringHash hash_value)
* Begin work on full PTB-compatible English tokenization 2014-07-07 06:29:24 +04:00			`from libcpp.vector cimport vector`

			`from spacy.spacy cimport StringHash`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`from spacy.spacy cimport Language`
* Refactor for string view features. Working on setting up flags and enums. 2014-07-07 18:58:48 +04:00			`from spacy.spacy cimport Lexeme`
			`from spacy.spacy cimport Lexeme_addr`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`from spacy.tokens cimport Tokens`
* Begin work on full PTB-compatible English tokenization 2014-07-07 06:29:24 +04:00

* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cdef class EnglishPTB(Language):`
* Refactor spacy so that chunks return arrays of lexemes, so that there is properly one lexeme per word. 2014-08-18 21:14:00 +04:00			`cdef int find_split(self, unicode word)`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00
* Begin work on full PTB-compatible English tokenization 2014-07-07 06:29:24 +04:00
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cdef EnglishPTB EN_PTB`
* Begin work on full PTB-compatible English tokenization 2014-07-07 06:29:24 +04:00
			`cpdef Lexeme_addr lookup(unicode word) except 0`
* Reorganized code to accomodate Tokens class. Need string views before group_by and count_by can be done well. 2014-07-07 14:47:21 +04:00			`cpdef Tokens tokenize(unicode string)`
* Begin work on full PTB-compatible English tokenization 2014-07-07 06:29:24 +04:00			`cpdef unicode unhash(StringHash hash_value)`