5 Commits

Author	SHA1	Message	Date
Andrew Ongko	81564cc4e8	Update Indonesian model (#2752) ... * adding e-KTP in tokenizer exceptions list * add exception token * removing lines with containing space as it won't matter since we use .split() method in the end, added new tokens in exception * add tokenizer exceptions list * combining base_norms with norm_exceptions * adding norm_exception * fix double key in lemmatizer * remove unused import on punctuation.py * reformat stop_words to reduce number of lines, improve readibility * updating tokenizer exception * implement is_currency for lang/id * adding orth_first_upper in tokenizer_exceptions * update the norm_exception list * remove bunch of abbreviations * adding contributors file	2018-09-14 12:30:32 +02:00
Aristo Rinjuang	432ede04af	adding more words and rephrasing (#2351) ... * adding more words and rephrasing * adding a contributor * tokenizer bugs solved	2018-05-24 11:40:57 +02:00
Jim Geovedi	37f19f5ed2	added more currencies based on corpus data	2017-08-03 13:03:25 +07:00
Jim Geovedi	d5fd32a572	added known currencies	2017-07-23 22:56:48 +07:00
Jim Geovedi	b5de329ea3	added norm_exceptions	2017-07-23 22:54:19 +07:00