Matthew Honnibal bb911e5f4e Fix #3830: 'subtok' label being added even if learn_tokens=False (#4188) ... * Prevent subtok label if not learning tokens The parser introduces the subtok label to mark tokens that should be merged during post-processing. Previously this happened even if we did not have the --learn-tokens flag set. This patch passes the config through to the parser, to prevent the problem. * Make merge_subtokens a parser post-process if learn_subtokens * Fix train script * Add test for 3830: subtok problem * Fix handlign of non-subtok in parser training		2019-08-23 17:54:00 +02:00
..
__init__.py	Merge branch 'master' into feature/el-framework	2019-03-26 11:00:02 +01:00
entityruler.py	Fix typo in docstrings [ci skip]	2019-08-22 16:24:15 +02:00
functions.py	Tidy up and improve docs and docstrings (#3370)	2019-03-08 11:42:26 +01:00
hooks.py	💫 Add better and serializable sentencizer (#3471)	2019-03-23 15:45:02 +01:00
pipes.pyx	Fix #3830: 'subtok' label being added even if learn_tokens=False (#4188)	2019-08-23 17:54:00 +02:00