2 Commits

Author	SHA1	Message	Date
Duygu Altinok	0e55f806dd	Turkish tokenization improvements (#6268) ... * added single and paired orth variants * added token match * added long text tokenization test * inverted init * normalized lemmas to lowercase * more abbrevs * tests for ordinals and abbrevs * separated period abbvrevs to another list * fiex typo * added ordinal and abbrev tests * added number tests for dates * minor refinement * added inflected abbrevs regex * added percentage and inflection * cosmetics * added token match * added url inflection tests * excluded url tokens from custom pattern * removed url match import	2020-10-29 09:43:17 +01:00
Duygu Altinok	b95a11dd95	Ordinal numbers for Turkish (#6142) ... * minor ordinal number addition * fixed typo * added corresponding lexical test	2020-10-07 10:25:37 +02:00