Commit Graph

  • 89b8775887 * Fix output from _min_edit_path when inputs match. Matthew Honnibal 2015-06-06 05:58:53 +0200
  • 27c8dc3db2 * Run tests one file at a time, as the teardown isn't cleaning up objects in global namespace properly Matthew Honnibal 2015-06-06 05:58:16 +0200
  • 4126ef3b8c * Restore hyphenation test to test_infix Matthew Honnibal 2015-06-06 05:57:36 +0200
  • 98cfd84123 * Remove hyphenation from main tokenizer loop: do it in infix.txt instead. This lets emoticons work Matthew Honnibal 2015-06-06 05:57:03 +0200
  • 45ec92243a * Add hyphenation rule to infix.txt for tokenizer Matthew Honnibal 2015-06-06 05:56:00 +0200
  • 4073533e28 * Upd munge_ewtb for the new json format Matthew Honnibal 2015-06-06 02:10:33 +0200
  • 6a1341b29e * Add tb pre-process script Matthew Honnibal 2015-06-06 01:59:44 +0200
  • a57ced0ead * Pin cython packages to particular versions, so that the current version works even if updates to them are pushed. Matthew Honnibal 2015-06-05 23:51:39 +0200
  • 1736fc5a67 * Add more options to bin/parser/train Matthew Honnibal 2015-06-05 23:49:26 +0200
  • 1fee7ade61 * Tweak to ner Matthew Honnibal 2015-06-05 23:48:43 +0200
  • 362f87dc3a * Update input corruption method to work with lists as well as trings Matthew Honnibal 2015-06-05 19:33:32 +0200
  • 33e70b167f * Remove dead code from ner.pyx Matthew Honnibal 2015-06-05 17:12:47 +0200
  • 88ac5c6e98 * Send beam_width < 0 to greedy parser Matthew Honnibal 2015-06-05 17:12:06 +0200
  • 0114e7600d * Fix NER oracle Matthew Honnibal 2015-06-05 17:11:26 +0200
  • c04e6ebca6 * Allow user to load different sized vectors. Matthew Honnibal 2015-06-05 16:26:39 +0200
  • 0aed9c9a33 * Fix train.py Matthew Honnibal 2015-06-05 15:50:24 +0200
  • 23e2f26535 * Require thinc 1.76 Matthew Honnibal 2015-06-05 15:50:05 +0200
  • 8466600add * Clean up train.py, removing unused tag jackknifing code Matthew Honnibal 2015-06-05 15:01:28 +0200
  • e772b48dcd * Skip sentences of length 1 in training Matthew Honnibal 2015-06-05 02:29:03 +0200
  • 6bf35cecc3 * Refactor transition system to use classes with staticmethods. Matthew Honnibal 2015-06-05 02:27:17 +0200
  • 36a34d544b * Refactoring arc_eager, grouping oracle functions into transitions Matthew Honnibal 2015-06-04 22:43:03 +0200
  • 4433396005 * Impove efficiency of dynamic oracle, making beam training faster Matthew Honnibal 2015-06-04 21:15:14 +0200
  • 079dad28a7 * Update for faster beam training Matthew Honnibal 2015-06-04 19:32:32 +0200
  • 0067f2e3ac add downloaded data for the tokenizer Pawel Jankiewicz 2015-06-03 13:40:30 +0200
  • f8843906ad Merge branch 'constituency' Matthew Honnibal 2015-06-03 06:07:24 +0200
  • ae653b850a * Remove unused import from gold.pyx Matthew Honnibal 2015-06-03 06:07:15 +0200
  • a2627b6102 * Fix bug in refactored init_transition Matthew Honnibal 2015-06-03 06:01:26 +0200
  • dd0867645d * Remove stray const from State header Matthew Honnibal 2015-06-03 00:10:04 +0200
  • 6c47b10a6e * Make optimization to children_in_buffer: stop searching when we would cross a bracket. Matthew Honnibal 2015-06-02 21:05:24 +0200
  • a513ec500f * Have oracle functions take a struct instead of a Python object Matthew Honnibal 2015-06-02 20:01:06 +0200
  • d1b55310a1 * Refactor _advance_beam function Matthew Honnibal 2015-06-02 18:38:41 +0200
  • 0786d9b3c7 * Refactor TransitionSystem, adding set_valid method Matthew Honnibal 2015-06-02 18:38:07 +0200
  • bd82a49994 * Add set_scores method to Model Matthew Honnibal 2015-06-02 18:37:10 +0200
  • a3964957f6 * Add profiling for _state.pyx Matthew Honnibal 2015-06-02 18:36:27 +0200
  • e822df0867 * Fix bugs in new greedy/beam parser Matthew Honnibal 2015-06-02 02:01:33 +0200
  • 66dfa95847 * Revise greedy_parse/beam_parse ownership goof Matthew Honnibal 2015-06-02 01:34:19 +0200
  • 70a7ad89ca * Removed unused imports from train.py Matthew Honnibal 2015-06-02 00:59:09 +0200
  • 75658b2ed3 * Remove use of new beam.loss property, to maintain compatibility with older versions of thinc for now. Matthew Honnibal 2015-06-02 00:57:09 +0200
  • a3de20118e * Wire up beam-width command line argument Matthew Honnibal 2015-06-02 00:54:12 +0200
  • 7c29362d60 * Rename parser class in parser.pxd, now that beam parsing is supported Matthew Honnibal 2015-06-02 00:53:49 +0200
  • 58d5ac0944 * Add beam search capabilities to Parser. Rename GreedyParser to Parser. Matthew Honnibal 2015-06-02 00:28:02 +0200
  • 62424e6c76 * Remove unused regularize argument from _ml.Model Matthew Honnibal 2015-06-02 00:27:07 +0200
  • adeb57cb1e * Fix long line Matthew Honnibal 2015-06-01 23:07:00 +0200
  • e09a08bd00 * Add copy_state function Matthew Honnibal 2015-06-01 23:06:30 +0200
  • c7876aa8b6 * Add get_valid method Matthew Honnibal 2015-06-01 23:05:25 +0200
  • d82f9d958d * Remove regularization cruft from _ml, move score from .pxd file to .pyx Matthew Honnibal 2015-05-31 18:48:05 +0200
  • 08044ea70c * Remove try/except around parser.train Matthew Honnibal 2015-05-31 15:21:56 +0200
  • c8a553fe91 * Fix cluster initialization Matthew Honnibal 2015-05-31 15:21:28 +0200
  • 5e99ff94c8 * Edits to arc eager oracle. Couldn't figure out how the non-monotonic lines made sense. They seem covered by children_in_stack Matthew Honnibal 2015-05-31 15:14:37 +0200
  • 6c5632b71c * Roll back proposed change to Break transition while investigate effect Matthew Honnibal 2015-05-31 06:49:52 +0200
  • d7cc2338e7 * Fix bug in train.py Matthew Honnibal 2015-05-31 06:49:06 +0200
  • 6bba793df3 * Disable the Zipf-reweighting thing while investigate effect Matthew Honnibal 2015-05-31 06:48:43 +0200
  • c037f80638 * Add case expansion to Brown clusters Matthew Honnibal 2015-05-31 05:50:50 +0200
  • 5ab0f233a1 * Ensure words in Brown clusters make it into the vocab, even if they're not in our probs list Matthew Honnibal 2015-05-31 05:46:16 +0200
  • e77940565d * Add length cap to distance feature Matthew Honnibal 2015-05-31 05:25:30 +0200
  • fd596351ba * Fix valency features Matthew Honnibal 2015-05-31 05:24:33 +0200
  • d42dda0372 * Shuffle docs before doing jackknife partition --- otherwise we'll not get the right genre mixes... Matthew Honnibal 2015-05-31 01:25:02 +0200
  • 4d8d490547 * Exclude empty sentences in prepare_treebank Matthew Honnibal 2015-05-31 01:12:46 +0200
  • 87d6551d19 * Allow gold parse to cut non-projective arcs Matthew Honnibal 2015-05-31 01:11:56 +0200
  • d512d20d81 * Allow parser to jackknife POS tags before training. Matthew Honnibal 2015-05-31 01:11:11 +0200
  • c4f0914b4e * Fix POS tag evaluation in scorer.py: do evaluate punctuation tags Matthew Honnibal 2015-05-30 18:24:32 +0200
  • 9e39a206da * Fix efficiency of JSON reading, by using ujson instead of stream Matthew Honnibal 2015-05-30 17:54:52 +0200
  • 6bbdcc5db5 * Fix gold_preproc flag in train.py Matthew Honnibal 2015-05-30 05:23:02 +0200
  • 76300bbb1b * Use updated JSON format, with sentences below paragraphs. Allows use of gold preprocessing flag. Matthew Honnibal 2015-05-30 01:25:46 +0200
  • 2d11739f28 * Change data format of JSON corpus, putting sentences into lists with the paragraph Matthew Honnibal 2015-05-30 01:25:00 +0200
  • 784e577f45 * Check NER length matches conll length in prepare_treebank Matthew Honnibal 2015-05-29 03:54:06 +0200
  • b76bbbd12c * Read json files recursively from a directory, instead of requiring a single .json file Matthew Honnibal 2015-05-29 03:52:55 +0200
  • 8f31d3b864 * Relax constraint on Break transition for non-monotonic parsing. Matthew Honnibal 2015-05-28 23:38:19 +0200
  • ef67ef7a4c * Recomment in training in train.py Matthew Honnibal 2015-05-28 22:40:26 +0200
  • 5eb64eeb11 * Print json treebank by genre, instead of by large file Matthew Honnibal 2015-05-28 22:40:01 +0200
  • 6b2e5c4b8a * Avoid NER scoring for sentences with some missing NER values. Matthew Honnibal 2015-05-28 22:39:08 +0200
  • f42dc1f7d8 * Fix evaluate method in train.py, to use sentences which don't have raw text Matthew Honnibal 2015-05-28 16:30:23 +0200
  • d25d31442d * Hackishly support broken NER annotations. Should fix this. Matthew Honnibal 2015-05-27 19:14:31 +0200
  • a7cee46fe9 * Update train.py, to support paragraphs where there's no raw_text Matthew Honnibal 2015-05-27 19:14:02 +0200
  • 7a2725bca4 * Read input json in a streaming way Matthew Honnibal 2015-05-27 19:13:11 +0200
  • b7fd77779a * Add some tests for reading NER data Matthew Honnibal 2015-05-27 17:37:03 +0200
  • 6a1c91675e * Add file to read ENAMEX ner data Matthew Honnibal 2015-05-27 17:36:23 +0200
  • ef1333cf89 * Have prepare_treebank read train/dev/test IDs. Matthew Honnibal 2015-05-27 17:35:05 +0200
  • e140e03516 * Read in OntoNotes. Doesn't support train/test/dev split yet Matthew Honnibal 2015-05-27 17:04:29 +0200
  • 732fa7709a * Edits to align_raw script, for use in prepare_treebank Matthew Honnibal 2015-05-27 04:23:31 +0200
  • 4010b9b6d9 * Pass parameter for regularization in parser.pyx Matthew Honnibal 2015-05-27 03:18:50 +0200
  • 4c6058baa7 * Fix evaluation of NER in scorer.py Matthew Honnibal 2015-05-27 03:18:16 +0200
  • 6016ee83a6 * Fix reading of NER in gold.pyx Matthew Honnibal 2015-05-27 03:17:50 +0200
  • 04bda8648d * Pass parameter for regularization to model Matthew Honnibal 2015-05-27 03:16:58 +0200
  • 895060e774 * Ensure tagger and NER are trained, even if non-projective problem Matthew Honnibal 2015-05-27 03:16:21 +0200
  • f69fe6a635 * Fix heads problem in read_conll Matthew Honnibal 2015-05-27 01:14:54 +0200
  • 0eec1d12af * Add comment about zipf reweighting Matthew Honnibal 2015-05-27 01:14:07 +0200
  • 4d37b66c55 * Make Zipf regularization a bit more efficient Matthew Honnibal 2015-05-27 01:12:50 +0200
  • 7fc24821bc * Experiment with Zipfian corruptions when calculating prediction Matthew Honnibal 2015-05-26 22:17:15 +0200
  • 32ae2cdabe * In prepare_treebank, move ner into the token descriptions Matthew Honnibal 2015-05-26 19:52:39 +0200
  • 61885aee76 * Work on prepare_treebank script, adding NER to it Matthew Honnibal 2015-05-26 19:28:29 +0200
  • 15bbbf4901 * Remove cruft from train.py Matthew Honnibal 2015-05-25 07:54:10 +0200
  • eba7b34f66 * Add flag to disable loading of word vectors Matthew Honnibal 2015-05-25 01:02:42 +0200
  • 89c3364041 * Update tests, preventing the parser from being loaded if possible Matthew Honnibal 2015-05-25 01:02:03 +0200
  • a9c70c9447 * Add tests for ontonotes sgml extraction Matthew Honnibal 2015-05-24 21:52:12 +0200
  • f460a8d2b6 * Comment out failing test in test_conjuncts Matthew Honnibal 2015-05-24 21:51:41 +0200
  • cc7439a16b * Don't use alignment.pyx file, move functionality to spacy.gold Matthew Honnibal 2015-05-24 21:51:15 +0200
  • 3593babd35 * Add functions for Levenshtein distance alignment Matthew Honnibal 2015-05-24 21:50:48 +0200
  • 744f06abf5 * Add script to read OntoNotes source documents Matthew Honnibal 2015-05-24 21:49:58 +0200
  • 13a8595a4b * Add tests for Levenshtein alignment of training data Matthew Honnibal 2015-05-24 21:45:57 +0200