7 Commits

Author	SHA1	Message	Date
Andrew Ongko	81564cc4e8	Update Indonesian model (#2752) ... * adding e-KTP in tokenizer exceptions list * add exception token * removing lines with containing space as it won't matter since we use .split() method in the end, added new tokens in exception * add tokenizer exceptions list * combining base_norms with norm_exceptions * adding norm_exception * fix double key in lemmatizer * remove unused import on punctuation.py * reformat stop_words to reduce number of lines, improve readibility * updating tokenizer exception * implement is_currency for lang/id * adding orth_first_upper in tokenizer_exceptions * update the norm_exception list * remove bunch of abbreviations * adding contributors file	2018-09-14 12:30:32 +02:00
Aristo Rinjuang	432ede04af	adding more words and rephrasing (#2351) ... * adding more words and rephrasing * adding a contributor * tokenizer bugs solved	2018-05-24 11:40:57 +02:00
Kit	9bc524982e	Find lowercased forms of numeric words	2018-01-08 03:25:08 +01:00
ines	0c2343d73a	Tidy up language data	2017-10-11 02:22:49 +02:00
Jim Geovedi	7d96d477ea	updated like_num	2017-07-29 17:44:46 +07:00
Jim Geovedi	3cca4ed798	added lex attrs rules	2017-07-29 17:22:21 +07:00
Jim Geovedi	f6f15678fb	added lex_attrs	2017-07-23 22:55:22 +07:00