3 Commits

Author	SHA1	Message	Date
Ines Montani	3d8fd4b461	Revert #4334	2019-09-29 17:32:12 +02:00
Ines Montani	c9cd516d96	Move tests out of package (#4334) ... * Move tests out of package * Fix typo	2019-09-28 18:05:00 +02:00
Matthew Honnibal	bb911e5f4e	Fix #3830: 'subtok' label being added even if learn_tokens=False (#4188) ... * Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training	2019-08-23 17:54:00 +02:00