Matthew Honnibal 95aaea0d3f Refactor so that the tokenizer data is read from Python data, rather than from disk		2016-09-25 14:49:53 +02:00
..
data	* Add rule to ensure ordinals are preserved as single tokens	2015-09-22 12:26:05 +10:00
__init__.py	Refactor so that the tokenizer data is read from Python data, rather than from disk	2016-09-25 14:49:53 +02:00
download.py	first idea for supporting multiple langs in download script	2016-03-24 11:19:43 +01:00