spaCy/lang_data/en/infix.txt

\.\.\.+
(?<=[a-z])\.(?=[A-Z])
(?<=[a-zA-Z])-(?=[a-zA-z])
(?<=[a-zA-Z])--(?=[a-zA-z])
(?<=[0-9])-(?=[0-9])
(?<=[A-Za-z]),(?=[A-Za-z])
* Allow longer ellipses to be treated as a single token, e.g. Hello......there 2016-05-09 14:22:53 +03:00			`\.\.\.+`
* Revert bad infix change 2015-07-26 18:32:37 +03:00			`(?<=[a-z])\.(?=[A-Z])`
* Fix ... tokenization, and correct orth inconsistencies in specials.json 2015-07-20 13:09:34 +03:00			`(?<=[a-zA-Z])-(?=[a-zA-z])`
* Add infix rule for double hyphens, re Issue #302 2016-03-29 05:03:44 +03:00			`(?<=[a-zA-Z])--(?=[a-zA-z])`
* Add tokenizer rule to fix numeric range tokenization 2015-10-17 07:49:51 +03:00			`(?<=[0-9])-(?=[0-9])`
* Fix infixed commas in tokenizer, re Issue #326. Need to benchmark on empirical data, to make sure this doesn't break other cases. 2016-04-14 12:36:03 +03:00			`(?<=[A-Za-z]),(?=[A-Za-z])`