Matthew Honnibal
|
8f84e8a78b
|
* Neaten oracle
|
2014-11-12 23:38:07 +11:00 |
|
Matthew Honnibal
|
7e0a9077dd
|
* Add context files
|
2014-11-12 23:22:36 +11:00 |
|
Matthew Honnibal
|
3b0b902384
|
* IOB-style parsing working. Accuracy down from BILOU, form 87-88 to 85-86
|
2014-11-12 23:21:09 +11:00 |
|
Matthew Honnibal
|
e6bb8aa3a9
|
* Move moves to bilou_moves. Refactor context, returning to the simpler giant-enum style
|
2014-11-12 00:54:50 +11:00 |
|
Matthew Honnibal
|
c788633429
|
* Add tokens_from_list method to Language
|
2014-11-11 23:43:14 +11:00 |
|
Matthew Honnibal
|
95282d4993
|
* Use the dynamic oracle 'follow' strategy
|
2014-11-11 21:11:17 +11:00 |
|
Matthew Honnibal
|
5aaf7a024d
|
* Move ner features to ner subdir
|
2014-11-11 21:09:03 +11:00 |
|
Matthew Honnibal
|
ff8989b63c
|
* Use greedy NER parser
|
2014-11-11 21:08:35 +11:00 |
|
Matthew Honnibal
|
0d943ab358
|
* Fixed greedy NER parsing. With static oracle, replicates accuracy from tagger.
|
2014-11-11 17:17:54 +11:00 |
|
Matthew Honnibal
|
399239760b
|
* Fix moves for new State struct
|
2014-11-10 22:16:05 +11:00 |
|
Matthew Honnibal
|
82247169f2
|
* Implement validation and oracle on pystate, for testing
|
2014-11-10 22:15:32 +11:00 |
|
Matthew Honnibal
|
3709ed9d6d
|
* Add curr field to State, to handle entity being built
|
2014-11-10 22:14:36 +11:00 |
|
Matthew Honnibal
|
af9ed18cf1
|
* Bug fixes to NER
|
2014-11-10 17:39:23 +11:00 |
|
Matthew Honnibal
|
9f2587f5ec
|
* Work on shift-reduce NER
|
2014-11-10 16:28:56 +11:00 |
|
Matthew Honnibal
|
f307eb2e36
|
* Refactor context extraction, and start breaking out gold standards into their own functions
|
2014-11-09 15:43:07 +11:00 |
|
Matthew Honnibal
|
602f993af9
|
* Moving tagger to accept multiple correct answers
|
2014-11-09 15:18:33 +11:00 |
|
Matthew Honnibal
|
f37d896a42
|
* Upd NER feats. With adadelta learner, getting 76.9 on NER
|
2014-11-07 04:43:54 +11:00 |
|
Matthew Honnibal
|
68d1cdad62
|
* When encoding POS/NER tags, accept '-' as a missing value
|
2014-11-07 04:42:31 +11:00 |
|
Matthew Honnibal
|
949a6245f9
|
* Increase default number of iterations from 5 to 10
|
2014-11-07 04:42:04 +11:00 |
|
Matthew Honnibal
|
3cab1d9a29
|
* Refine word_shape feature, by trimming the max sequence length
|
2014-11-07 04:41:29 +11:00 |
|
Matthew Honnibal
|
b4454cf036
|
* Add extra context tokens
|
2014-11-07 04:40:36 +11:00 |
|
Matthew Honnibal
|
50309e6e49
|
* Fix context vector, importing all features
|
2014-11-05 22:11:39 +11:00 |
|
Matthew Honnibal
|
07a23768de
|
* Play with NER feats a bit. Up to 82.00 training on MUC7.
|
2014-11-05 21:47:17 +11:00 |
|
Matthew Honnibal
|
4ecbe8c893
|
* Complete refactor of Tagger features, to use a generic list of context names.
|
2014-11-05 20:45:29 +11:00 |
|
Matthew Honnibal
|
0a8c84625d
|
* Moving feature context stuff to a generalized place
|
2014-11-05 19:55:10 +11:00 |
|
Matthew Honnibal
|
3733444101
|
* Generalize tagger code, in preparation for NER and supersense tagging.
|
2014-11-05 03:42:14 +11:00 |
|
Matthew Honnibal
|
abbe3e44b0
|
* Move spacy.pos tagger to spacy.tagger, and generalize it so that it can take on other tagging tasks, given a different set of feature templates.
|
2014-11-05 00:37:59 +11:00 |
|
Matthew Honnibal
|
954c970415
|
* Add __iter__ method to tokens
|
2014-11-04 01:07:08 +11:00 |
|
Matthew Honnibal
|
f07457a91f
|
* Remove POS alignment stuff. Now use training data based on raw text, instead of clumsy detokenization stuff
|
2014-11-04 01:06:43 +11:00 |
|
Matthew Honnibal
|
ae52f9f38c
|
* Remove vocab10k from tokens
|
2014-11-03 00:23:20 +11:00 |
|
Matthew Honnibal
|
32fb50dc35
|
* Remove non_sparse method --- features wanting this can do it easily enough.
|
2014-11-03 00:15:47 +11:00 |
|
Matthew Honnibal
|
b5ae1471db
|
* Fiddle with POS tag features
|
2014-11-03 00:15:03 +11:00 |
|
Matthew Honnibal
|
70ea862703
|
* Remove vocab10k field, and add flags for gazetteers
|
2014-11-03 00:13:51 +11:00 |
|
Matthew Honnibal
|
711ed0f636
|
* Whitespace
|
2014-11-02 14:22:32 +11:00 |
|
Matthew Honnibal
|
fcd9490d56
|
* Add pos_tag method to Language
|
2014-11-02 14:21:43 +11:00 |
|
Matthew Honnibal
|
829bb2bdbe
|
* Add mappings to Twitter POS tag corpus
|
2014-11-02 13:21:19 +11:00 |
|
Matthew Honnibal
|
437cd2217d
|
* Fix strings i/o, removing use of ujson library in favour of plain text file. Allows better control of codecs.
|
2014-11-02 13:20:37 +11:00 |
|
Matthew Honnibal
|
3352e89e21
|
* Use LIKE_URL and LIKE_NUMBER flag features. Seems to improve accuracy on onto web
|
2014-11-02 13:19:54 +11:00 |
|
Matthew Honnibal
|
8335706321
|
* Add LIKE_URL and LIKE_NUMBER flag features
|
2014-11-02 13:19:23 +11:00 |
|
Matthew Honnibal
|
5484fbea69
|
* Implement is_number
|
2014-11-01 19:13:24 +11:00 |
|
Matthew Honnibal
|
f685218e21
|
* Add is_urlish function
|
2014-11-01 17:39:34 +11:00 |
|
Matthew Honnibal
|
09a3e54176
|
* Delete print statements from stringstore
|
2014-10-31 17:45:26 +11:00 |
|
Matthew Honnibal
|
b186a66bae
|
* Rename Token.lex_pos to Token.postype, and Token.lex_supersense to Token.sensetype
|
2014-10-31 17:44:39 +11:00 |
|
Matthew Honnibal
|
a8ca078b24
|
* Restore lexemes field to lexicon
|
2014-10-31 17:43:25 +11:00 |
|
Matthew Honnibal
|
6c807aa45f
|
* Restore id attribute to lexeme, and rename pos field to postype, to store clustered tag dictionaries
|
2014-10-31 17:43:00 +11:00 |
|
Matthew Honnibal
|
aaf6953fe0
|
* Add count_tags functionto pos.pyx, which should probably live in another file. Feature set achieves 97.9 on wsj19-21, 95.85 on onto web.
|
2014-10-31 17:42:15 +11:00 |
|
Matthew Honnibal
|
f67cb9a5a3
|
* Add count_tags functionto pos.pyx, which should probably live in another file. Feature set achieves 97.9 on wsj19-21, 95.85 on onto web.
|
2014-10-31 17:42:04 +11:00 |
|
Matthew Honnibal
|
ea8f1e7053
|
* Tighten interfaces
|
2014-10-30 18:14:42 +11:00 |
|
Matthew Honnibal
|
ea85bf3a0a
|
* Tighten the interface to Language
|
2014-10-30 18:01:27 +11:00 |
|
Matthew Honnibal
|
c6fcd03692
|
* Small efficiency tweak to lexeme init
|
2014-10-30 17:56:11 +11:00 |
|