Matthew Honnibal
85485f5c2b
Fix inconsistencies in generate_specials.py
...
Re Issue #321 , fix inconsistencies in the script that generates specials.json. The result still isn't so satisfying --- we need to revise this as we move to parse more morphologically rich languages.
2016-04-07 11:21:52 +10:00
Matthew Honnibal
4b4eec8b47
* Fix Issue #201 : Tokenization of there'll
2015-12-29 18:09:09 +01:00
Matthew Honnibal
e8bd92f1e7
* Fix lemma of let's, re Issue #177
2015-11-13 06:42:23 +11:00
Matthew Honnibal
726bb648da
* Fix non-breaking space in specials.json
2015-10-19 12:46:11 +11:00
Matthew Honnibal
e39095da82
* Fix designation of non-breaking space in specials.json.
2015-10-19 12:39:03 +11:00
Matthew Honnibal
bdcb8d695c
* Add non-breaking space to specials.json
2015-10-10 15:54:06 +11:00
Matthew Honnibal
7e7f28e1fd
* Add smart-quote possessive marker in generate_specials
2015-10-10 14:06:09 +11:00
Matthew Honnibal
0e92e8574a
* Fix pos tag in em-dash in specials
2015-10-09 11:06:37 +02:00
Matthew Honnibal
393a13d1af
* Add unicode em dash to specials.json, so that we can control what POS tag it gets. This way we can prevent sentence boundary detection errors, to address Issue #130 .
2015-10-09 19:24:33 +11:00
Matthew Honnibal
1490feda29
* Make generate_specials pretty-print the specials.json file
2015-10-09 19:23:47 +11:00
Henning Peters
911de2ae49
add overseen (?) char
2015-09-22 12:29:47 +02:00
jxs8172
85f01c5e16
Add contributor agreement. Add exception to 'it' so that 'its' and 'Its' isn't generated (its =/= it's)
2015-08-24 18:20:06 -04:00
jxs8172
5876248109
Add missing we've and hardcoded 's and 'S
2015-08-21 22:57:47 -04:00
jxs8172
a5e0a0073b
Add a script to generate the specials.json file, to take care of handling uppercase and missing apostrophe contractions
2015-08-21 22:39:33 -04:00
