5 Commits

Author	SHA1	Message	Date
Bart Broere	e4a45ae55f	Very minor documentation fix	2017-06-12 12:28:51 +02:00
Yuval Pinter	af3d121ec9	extend suffixes from first to last ... reverse suffix list in `tokenizer_pseudo_code()` so the order of returned tokens matches input order	2017-05-22 10:56:03 -04:00
Kevin Gao	7ec710af0e	Fix Custom Tokenizer docs ... - Fix mismatched quotations - Make it more clear where ORTH, LEMMA, and POS symbols come from - Make strings consistent - Fix lemma_ assertion s/-PRON-/me/	2017-01-17 10:38:14 -08:00
Ines Montani	ce8bf08223	Fix formatting	2016-12-18 17:40:20 +01:00
Ines Montani	c20abc8a6d	Add customizing tokenizer and training workflow	2016-11-05 20:40:11 +01:00