Adriane Boyd 5861308910 Generalize handling of tokenizer special cases ... Handle tokenizer special cases more generally by using the Matcher internally to match special cases after the affix/token_match tokenization is complete. Instead of only matching special cases while processing balanced or nearly balanced prefixes and suffixes, this recognizes special cases in a wider range of contexts: * Allows arbitrary numbers of prefixes/affixes around special cases * Allows special cases separated by infixes Existing tests/settings that couldn't be preserved as before: * The emoticon '")' is no longer a supported special case * The emoticon ':)' in "example:)" is a false positive again When merged with #4258 (or the relevant cache bugfix), the affix and token_match properties should be modified to flush and reload all special cases to use the updated internal tokenization with the Matcher.		2019-09-08 20:35:16 +02:00
..
__init__.py	Add __init__.py file for regression tests	2016-11-01 13:45:06 +01:00
test_issue1-1000.py	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
test_issue1001-1500.py	Generalize handling of tokenizer special cases	2019-09-08 20:35:16 +02:00
test_issue1501-2000.py	Make flaky test test_issue_1971_4 more explicit	2019-08-31 14:08:05 +02:00
test_issue2001-2500.py	Merge regression tests	2019-07-10 12:49:18 +02:00
test_issue2501-3000.py	Merge regression tests	2019-02-24 21:03:39 +01:00
test_issue3001-3500.py	Merge regression tests	2019-07-10 12:49:18 +02:00
test_issue3521.py	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
test_issue3526.py	Fix handling of old entity ruler files	2019-07-10 12:14:12 +02:00
test_issue3531.py	Don't make "settings" or "title" required in displaCy data (closes #3531)	2019-04-03 10:13:16 +02:00
test_issue3540.py	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
test_issue3549.py	Ensure match pattern error isn't raised on empty errors (closes #3549)	2019-04-09 12:50:43 +02:00
test_issue3555.py	Add xfailing test for #3555	2019-04-09 11:07:14 +02:00
test_issue3611.py	Auto-format [ci skip]	2019-07-17 12:34:13 +02:00
test_issue3625.py	Auto-format [ci skip]	2019-07-17 12:34:13 +02:00
test_issue3803.py	Tidy up [ci skip]	2019-06-12 13:38:23 +02:00
test_issue3830.py	Fix #3830: 'subtok' label being added even if learn_tokens=False (#4188)	2019-08-23 17:54:00 +02:00
test_issue3839.py	Auto-format [ci skip]	2019-07-17 12:34:13 +02:00
test_issue3869.py	Auto-format [ci skip]	2019-07-17 12:34:13 +02:00
test_issue3879.py	use states[q] in while retry loop (#4162)	2019-08-21 21:58:04 +02:00
test_issue3880.py	Tidy up and auto-format	2019-07-11 12:02:25 +02:00
test_issue3882.py	Exclude user_data when copying doc in displaCy (closes #3882)	2019-06-26 14:37:05 +02:00
test_issue3951.py	use states[q] in while retry loop (#4162)	2019-08-21 21:58:04 +02:00
test_issue3959.py	Serialize POS attribute when doc.is_tagged (#4092)	2019-08-21 21:59:30 +02:00
test_issue3962.py	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
test_issue3972.py	Matcher ID fixes (#4179)	2019-08-22 17:17:07 +02:00
test_issue4002.py	Tidy up and auto-format	2019-08-18 15:09:16 +02:00
test_issue4030.py	Resolve edge case when calling textcat.predict with empty doc (#4035)	2019-07-30 14:58:01 +02:00
test_issue4054.py	ensure the lang of vocab and nlp stay consistent (#4057)	2019-08-01 17:13:01 +02:00
test_issue4104.py	Tidy up and auto-format	2019-08-18 15:09:16 +02:00
test_issue4120.py	adding double match for optional operator at the end (#4166)	2019-08-21 22:46:56 +02:00
test_issue4133.py	Serialize POS attribute when doc.is_tagged (#4092)	2019-08-21 21:59:30 +02:00
test_issue4190.py	failing unit test for Issue 4190	2019-08-28 14:16:34 +02:00