Commit Graph

2 Commits

Author SHA1 Message Date
Matthew Honnibal
677e111ee7 * Revise tokenization rules to match PTB. Rules are pretty messy around periods, need better support for these. 2014-12-07 22:04:47 +11:00
Matthew Honnibal
c4cd3bc57a * Add prefix and suffix data files 2014-09-25 18:24:52 +02:00