Commit Graph

1276 Commits

Author SHA1 Message Date
Matthew Honnibal
7fc24821bc * Experiment with Zipfian corruptions when calculating prediction 2015-05-26 22:17:15 +02:00
Matthew Honnibal
32ae2cdabe * In prepare_treebank, move ner into the token descriptions 2015-05-26 19:52:39 +02:00
Matthew Honnibal
61885aee76 * Work on prepare_treebank script, adding NER to it 2015-05-26 19:28:29 +02:00
Matthew Honnibal
15bbbf4901 * Remove cruft from train.py 2015-05-25 07:54:10 +02:00
Matthew Honnibal
eba7b34f66 * Add flag to disable loading of word vectors 2015-05-25 01:02:42 +02:00
Matthew Honnibal
89c3364041 * Update tests, preventing the parser from being loaded if possible 2015-05-25 01:02:03 +02:00
Matthew Honnibal
a9c70c9447 * Add tests for ontonotes sgml extraction 2015-05-24 21:52:12 +02:00
Matthew Honnibal
f460a8d2b6 * Comment out failing test in test_conjuncts 2015-05-24 21:51:41 +02:00
Matthew Honnibal
cc7439a16b * Don't use alignment.pyx file, move functionality to spacy.gold 2015-05-24 21:51:15 +02:00
Matthew Honnibal
3593babd35 * Add functions for Levenshtein distance alignment 2015-05-24 21:50:48 +02:00
Matthew Honnibal
744f06abf5 * Add script to read OntoNotes source documents 2015-05-24 21:49:58 +02:00
Matthew Honnibal
13a8595a4b * Add tests for Levenshtein alignment of training data 2015-05-24 21:46:11 +02:00
Matthew Honnibal
fc75210941 * Move spacy.syntax.conll to spacy.gold 2015-05-24 21:35:02 +02:00
Matthew Honnibal
765b61cac4 * Update spacy.scorer, to use P/R/F to support tokenization errors 2015-05-24 20:07:18 +02:00
Matthew Honnibal
efe7a7d7d6 * Clean unused functions from spacy.syntax.conll 2015-05-24 20:06:46 +02:00
Matthew Honnibal
78487f3e66 * Update parser oracle for missing heads 2015-05-24 20:05:58 +02:00
Matthew Honnibal
541c62c126 * Remove import of removed read_docparse_file function 2015-05-24 20:05:13 +02:00
Matthew Honnibal
1044a13413 * Begin refactoring scorer to use recall over gold dependencies 2015-05-24 17:40:15 +02:00
Matthew Honnibal
acd1245ad4 * Remove cruft from conll.pyx --- unused stuff about evlauation, which now lives in spacy.scorer 2015-05-24 17:35:49 +02:00
Matthew Honnibal
bfeb29ebd1 * Tmp commit 2015-05-24 02:50:14 +02:00
Matthew Honnibal
20f1d868a3 * Tmp commit. Working on whole document parsing 2015-05-24 02:49:56 +02:00
Matthew Honnibal
983d954ef4 * Tmp commit, while switch to new format that assumes alignment happens during training 2015-05-23 17:39:04 +02:00
Matthew Honnibal
f35503018e * Tmp commit of train, while I move to better alignment in gold standard 2015-05-23 17:21:25 +02:00
Matthew Honnibal
bdaddc4103 * Add PTB file read tests 2015-05-20 17:29:27 +02:00
Matthew Honnibal
f2ee9c4feb * Comment out constituency parsing stuff, so that code compiles 2015-05-20 16:55:05 +02:00
Matthew Honnibal
8ee7c541f1 * Update Constituent definition 2015-05-20 16:03:26 +02:00
Matthew Honnibal
9dfc9c039c * Work on constituency parsing. 2015-05-20 16:02:51 +02:00
Matthew Honnibal
8e7d5fce90 * Add members to Tokens documentation 2015-05-13 21:53:41 +02:00
Matthew Honnibal
5a5710e711 * Fix Span.subtree property 2015-05-13 21:53:15 +02:00
Matthew Honnibal
badf030b6c * Add parse navigation to Span objects 2015-05-13 21:45:19 +02:00
Matthew Honnibal
ca320afe86 * Add docstring for ents attribute 2015-05-13 21:20:47 +02:00
Matthew Honnibal
7c8bf0eba5 * Add example JSON-formatted training file 2015-05-12 22:42:37 +02:00
Matthew Honnibal
4230467947 * Update fabfile.py for JSON-formatted training 2015-05-12 22:34:07 +02:00
Matthew Honnibal
ba07b925a7 * Fix compile error in conll.pyx 2015-05-12 22:33:47 +02:00
Matthew Honnibal
f1e0272b18 * Disable c-parsing transitions 2015-05-12 22:33:25 +02:00
Matthew Honnibal
03a6626545 * Tmp commit 2015-05-12 20:27:56 +02:00
Matthew Honnibal
9568ebed08 * Fix off-by-one in head reading 2015-05-12 20:27:56 +02:00
Matthew Honnibal
3d6b3fc6fb * Restore shuffling, and remove print statements from train.py 2015-05-12 20:27:56 +02:00
Matthew Honnibal
e167355505 * Use JSON docs for training and evaluation. Currently a bug that is costing 0.6 acc 2015-05-12 20:27:56 +02:00
Matthew Honnibal
69840d8cc3 * Tweak verbose output printing in scorer.py 2015-05-12 20:27:56 +02:00
Matthew Honnibal
e0ef6b6992 * Fix alignment in prepare_treebank 2015-05-12 20:27:56 +02:00
Matthew Honnibal
0605af6838 * Fix head misalignment in read_conll, when periods are ignored 2015-05-12 20:27:56 +02:00
Matthew Honnibal
d2ac8d8007 * Add ctnt field to State, in preparation for constituency parsing 2015-05-12 20:27:56 +02:00
Matthew Honnibal
ab67693393 * Add read_json_file to conll.pyx 2015-05-12 20:27:55 +02:00
Matthew Honnibal
aff9359a8d * Update ner.pyx to expect brackets from gold_tuples 2015-05-12 20:27:55 +02:00
Matthew Honnibal
0ad72a77ce * Write JSON files, with both dependency and PSG parses 2015-05-12 20:27:55 +02:00
Matthew Honnibal
5078a32213 * Work on script to format training data as a JSON file. 2015-05-12 20:27:55 +02:00
Matthew Honnibal
d48218f4b2 * Add left_edge and right_edge properties 2015-05-12 20:27:55 +02:00
Matthew Honnibal
bdb56497b5 * Add test for right_edge and left_edge 2015-05-12 20:27:55 +02:00
Matthew Honnibal
53cf77e1c8 * Bug fix: when non-monotonically correct a dependency, make sure to delete the old one from the child list 2015-05-12 20:26:41 +02:00