5103 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Matthew Honnibal	7237c805c7	* Load tag for specials.json token	2015-03-26 16:44:46 +01:00
Matthew Honnibal	13520e6cf0	* Add i.e. to specials.json	2015-03-26 16:44:45 +01:00
Matthew Honnibal	567388e38d	* Use values encoded by StringStore in POS tagging, rather than indices into a list of tags	2015-03-26 16:44:45 +01:00
Matthew Honnibal	3105c7f8ba	* Don't pass label_ids dict to Tokens, since we now use the StringStore to manage string-to-int mapping for labels	2015-03-26 16:44:45 +01:00
Matthew Honnibal	27d9df49e7	* Upd sbd tests	2015-03-26 16:44:45 +01:00
Matthew Honnibal	801bf14f4f	* Clean up handling of dep_strings and ent_strings, using StringStore to encode the label names.	2015-03-26 16:44:45 +01:00
Matthew Honnibal	9061bbaf61	* Move to fixing up ent_strings and dep_strings passing	2015-03-26 16:44:45 +01:00
Matthew Honnibal	31fad99518	* Use StringStore to encode label names, instead of label_ids	2015-03-26 16:44:45 +01:00
Matthew Honnibal	64db61bff1	* Add Span class to Python API	2015-03-26 16:44:45 +01:00
Matthew Honnibal	b9b695fb1b	* Remove debug word list	2015-03-26 16:44:45 +01:00
Matthew Honnibal	8f7eeb1c2d	* Add verbose flag for Scorer, for debugging, and fix ent_strings bug	2015-03-26 16:44:45 +01:00
Matthew Honnibal	f21ab2d7fb	* Fix bug in ugly ent_strings hack on English class	2015-03-26 16:44:45 +01:00
Matthew Honnibal	1c843934be	* Fix oracle bug in NER. Now getting 77% F on ontonotes	2015-03-26 16:44:44 +01:00
Matthew Honnibal	903f196b3f	* Fix verbose printing for scorer	2015-03-26 16:44:44 +01:00
Matthew Honnibal	e181c051d5	* Improve features for NER	2015-03-26 16:44:44 +01:00
Matthew Honnibal	7ecb52c0ed	* Add scorer script	2015-03-26 16:44:44 +01:00
Matthew Honnibal	8057a95f20	* NER seems to be working, scoring 69 F. Need to add decision-history features --- currently only use current word, 2 words context. Need refactoring.	2015-03-26 16:44:44 +01:00
Matthew Honnibal	e99f19dd6c	* Fix clean function	2015-03-26 16:44:44 +01:00
Matthew Honnibal	ae235e07b9	* Refactoring working for parser, but now need to rig up features for NER, and then debug oracle etc.	2015-03-26 16:44:44 +01:00
Matthew Honnibal	4539c70542	* Work on updating train script for named entity recognition	2015-03-26 16:44:44 +01:00
Matthew Honnibal	357dcdcc01	* Fix clean function	2015-03-26 16:44:44 +01:00
Matthew Honnibal	b3eda03c9c	* Tmp	2015-03-26 16:44:44 +01:00
Matthew Honnibal	220ce8bfed	* Prepare English class for NER	2015-03-26 16:44:44 +01:00
Matthew Honnibal	f5830dc1c1	* Remove _transitions.pyx	2015-03-26 16:44:44 +01:00
Matthew Honnibal	7a1a333f04	* Allow gold tokenization training, for debugging	2015-03-26 16:44:43 +01:00
Matthew Honnibal	8da53cbe3c	* Fix setup.py, so that when compiling, only the necessary files are compiled	2015-03-26 16:44:43 +01:00
Matthew Honnibal	6865c2fb4d	* Fix assignment of dep strings in tokens.pyx	2015-03-26 16:44:43 +01:00
Matthew Honnibal	6b6bce9e7a	* Fix label loading for transition system	2015-03-26 16:44:43 +01:00
Matthew Honnibal	5278c7504b	* Hacks to conll.pyx. Should clean these up.	2015-03-26 16:44:43 +01:00
Matthew Honnibal	f321b2b2eb	* Remove TODO comment	2015-03-26 16:44:43 +01:00
Matthew Honnibal	fdabd93bfb	* Ensure high loss for invalid moves, and fix label reading for arc-eager	2015-03-26 16:44:43 +01:00
Matthew Honnibal	f5f15a1ef2	* Tmp commit	2015-03-26 16:44:43 +01:00
Matthew Honnibal	10ed738df2	* Tmp commit	2015-03-26 16:44:43 +01:00
Matthew Honnibal	4f83c9b3d5	* Make costs label-sensitive	2015-03-26 16:44:43 +01:00
Matthew Honnibal	179b7eb0a7	* Specify parser transition system in language	2015-03-26 16:44:43 +01:00
Matthew Honnibal	8c883cef58	* Refactored transition system code now compiling. Still need to hook up label oracle, and test	2015-03-26 16:44:43 +01:00
Matthew Honnibal	6e86790a4e	* Add new syntax modules to setup.py	2015-03-26 16:44:42 +01:00
Matthew Honnibal	34215de61b	* Upd train script, moving lots of functionality to new GoldParse class	2015-03-26 16:44:42 +01:00
Matthew Honnibal	f0159ab4b6	* Add file to hold GoldParse class	2015-03-26 16:44:42 +01:00
Matthew Honnibal	8eadb984cb	* Refactor arc_eager to use new TransitionSystem base class. Need to fix oracle	2015-03-26 16:44:42 +01:00
Matthew Honnibal	b063001596	* Add base TransitionSystem class. Still need to rethink how non-monotonic labelling will work for best_valid	2015-03-26 16:44:42 +01:00
Matthew Honnibal	01bc4d6815	* Add set_parse method, to assign parse to tokens in a less hacky way.	2015-03-26 16:44:42 +01:00
Matthew Honnibal	dc986dbc0b	* Work on refactored parser, where TransitionSystem can be easily subclassed	2015-03-26 16:44:42 +01:00
Matthew Honnibal	1cc6329b18	* Add base class to do transitions	2015-03-26 16:44:42 +01:00
Matthew Honnibal	135756ac3d	* Tmp commit of NER refactoring	2015-03-26 16:44:42 +01:00
Matthew Honnibal	49df1b7002	* Ignore .tgz files	2015-03-26 16:44:42 +01:00
Matthew Honnibal	8715101239	* Merge changes from stash	2015-03-26 16:44:42 +01:00
Matthew Honnibal	23c1f6fc04	* Merge changes from stash	2015-03-26 16:44:41 +01:00
Matthew Honnibal	0ff078876a	* Commit some work on ner.yx done on the plane	2015-03-26 16:44:41 +01:00
Matthew Honnibal	d81b7be6a2	* Merge train.py	2015-03-26 16:44:41 +01:00