Commit Graph

21 Commits

Author SHA1 Message Date
Matthew Honnibal
7441fce7ba Fix undefined variable in conllu script 2018-02-26 14:59:56 +01:00
Matthew Honnibal
f0478635df Fix Japanese tokenizer flag 2018-02-26 10:32:12 +01:00
Matthew Honnibal
5faae803c6 Add option to not use Janome for Japanese tokenization 2018-02-26 09:39:46 +01:00
Matthew Honnibal
9b406181cd Add Chinese.Defaults.use_jieba setting, for UD 2018-02-25 15:12:38 +01:00
Matthew Honnibal
9e960d24fc Refactor conllu script, fix interface, generalize 2018-02-25 14:54:47 +01:00
Matthew Honnibal
551c93fe01 Shuffle data after each epoch. Improve script 2018-02-25 13:35:32 +01:00
Matthew Honnibal
bdb0174571 Update conllu training script 2018-02-25 13:12:39 +01:00
Matthew Honnibal
e09070eca7 Refactor conllu script 2018-02-25 12:50:29 +01:00
Matthew Honnibal
44e496a82e Refactor conllu script 2018-02-25 12:48:22 +01:00
Matthew Honnibal
c388833ca6 Minibatch by number of tokens, support other vectors, refactor CoNLL printing 2018-02-25 10:38:06 +01:00
Matthew Honnibal
dd78ef066a Unset data size limit in conll script 2018-02-24 18:14:57 +01:00
Matthew Honnibal
8adeea3746 Generalize conllu script. Now handling Chinese (maybe badly) 2018-02-24 16:04:27 +01:00
Matthew Honnibal
329b14c9e6 Clean up conllu script 2018-02-24 10:31:53 +01:00
Matthew Honnibal
5be092ee72 CONLLU scoring 80.9% UAS with no oracle segments 2018-02-23 23:49:17 +01:00
Matthew Honnibal
23236340f4 Update CoNLL script. Don't preset SBD. Set batch size to 8, avoid writing twice 2018-02-22 21:35:50 +01:00
Matthew Honnibal
a26e399f84 Update conllu script 2018-02-22 19:43:54 +01:00
Matthew Honnibal
001e2ec6d6 Refactor CoNLL training script 2018-02-22 16:00:34 +01:00
Matthew Honnibal
6a27a4f77c Set accelerating batch size in CONLL train script 2018-02-21 21:02:41 +01:00
Matthew Honnibal
4dc0fc9954 Replace labels that didn't make freq cutoff 2018-02-21 15:59:22 +01:00
Matthew Honnibal
97164b1763 Fix conllu script 2018-02-21 14:46:54 +01:00
Matthew Honnibal
24fb2c246f Add script to do conllu training 2018-02-21 13:53:59 +01:00