5247 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
Matthew Honnibal	b488224c09	* Restoring Lexeme-as-struct	2014-09-10 20:41:37 +02:00
Matthew Honnibal	e80d3b9784	* Compile tokens in setup	2014-09-10 19:41:19 +02:00
Matthew Honnibal	7c09c73a14	* Refactor to use tokens class.	2014-09-10 18:27:44 +02:00
Matthew Honnibal	cf412adba8	* Refactoring to use Tokens object	2014-09-10 18:11:13 +02:00
Matthew Honnibal	b8c4549ffe	* Tweak overview docs	2014-09-07 21:29:41 +02:00
Matthew Honnibal	7dac9b9ccb	* Fix setup script	2014-09-01 23:41:59 +02:00
Matthew Honnibal	5ee4d8c641	* Work on tests for flag features	2014-09-01 23:41:43 +02:00
Matthew Honnibal	8fbe9b6f97	* Bug fixes to flag features	2014-09-01 23:41:31 +02:00
Matthew Honnibal	bf47429368	* Add tests for non_sparse string transform	2014-09-01 23:27:31 +02:00
Matthew Honnibal	c50433163f	* Add tests for flag features	2014-09-01 23:27:09 +02:00
Matthew Honnibal	786a4a86fe	* Add tests for canon_case	2014-09-01 23:26:49 +02:00
Matthew Honnibal	4c7b997df7	* Add tests for word shape features	2014-09-01 23:26:17 +02:00
Matthew Honnibal	c5abb81f4c	* Add incomplete tests of asciify function	2014-09-01 23:25:51 +02:00
Matthew Honnibal	151aa14bba	* Add asciify string transform, and other bits.	2014-09-01 23:25:28 +02:00
Matthew Honnibal	c4ba216642	* Switch canon_case to get value, to avoid keyerror	2014-09-01 17:27:36 +02:00
Matthew Honnibal	a779275a59	* Add canon_case function	2014-08-30 20:57:43 +02:00
Matthew Honnibal	8bbfadfced	* Pass tests. Need to implement more feature functions.	2014-08-30 20:36:06 +02:00
Matthew Honnibal	dcab14ede2	* Begin testing more functionality	2014-08-30 19:01:15 +02:00
Matthew Honnibal	3e3ff99ca0	* Add orth features	2014-08-30 19:01:00 +02:00
Matthew Honnibal	6209d94f83	* Add tests for word shape	2014-08-30 19:00:10 +02:00
Matthew Honnibal	4e5b2d47e2	* More docs	2014-08-29 03:01:40 +02:00
Matthew Honnibal	5233f110c4	* Adding PTB3 tokenizer back in, so can understand how much boilerplate is in the docs for multiple tokenizers	2014-08-29 02:30:27 +02:00
Matthew Honnibal	45a22d6b2c	* Docs coming together	2014-08-29 01:59:23 +02:00
Matthew Honnibal	c282e6d5fb	* Redesign proceeding	2014-08-28 19:45:09 +02:00
Matthew Honnibal	fd4e61e58b	* Fixed contraction tests. Need to correct problem with the way case stats and tag stats are supposed to work.	2014-08-27 20:22:33 +02:00
Matthew Honnibal	fdaf24604a	* Basic punct tests updated and passing	2014-08-27 19:38:57 +02:00
Matthew Honnibal	8d20617dfd	* Whitespace	2014-08-27 17:16:16 +02:00
Matthew Honnibal	e9a62b6eba	* Refactoring with Lexeme as a class now compiles. Basic design seems to work	2014-08-27 17:15:39 +02:00
Matthew Honnibal	68bae2fec6	* More refactoring	2014-08-25 16:42:22 +02:00
Matthew Honnibal	88095666dc	* Remove Lexeme struct, preparing to rename Word to Lexeme.	2014-08-24 19:24:42 +02:00
Matthew Honnibal	ce59526011	* Add Word classes	2014-08-24 18:14:08 +02:00
Matthew Honnibal	3b793cf4f7	* Tests passing for new Word object version	2014-08-24 18:13:53 +02:00
Matthew Honnibal	9815c7649e	* Refactor around Word objects, adapting tests. Tests passing, except for string views.	2014-08-23 19:55:06 +02:00
Matthew Honnibal	4f01df9152	* Moving to Word objects in place of the Lexeme struct.	2014-08-22 17:32:16 +02:00
Matthew Honnibal	782806df08	* Moving to Word objects in place of the Lexeme struct.	2014-08-22 17:28:23 +02:00
Matthew Honnibal	47fbd0475a	* Replace the use of dense_hash_map with Python dict	2014-08-22 17:13:09 +02:00
Matthew Honnibal	6f83dca218	* Fix import for ptb tokenization test	2014-08-22 17:05:44 +02:00
Matthew Honnibal	e289896603	* Fix ptb3 module	2014-08-22 16:36:17 +02:00
Matthew Honnibal	a22101404a	* Move en_ptb data	2014-08-22 04:28:51 +02:00
Matthew Honnibal	89d6faa9c9	* Move en_ptb to ptb3	2014-08-22 04:24:05 +02:00
Matthew Honnibal	4bcdd6d31c	* Further improvements to spacy docs, tweaks to code.	2014-08-22 04:20:24 +02:00
Matthew Honnibal	4eb9c2b30f	* Add overview doc	2014-08-22 03:38:05 +02:00
Matthew Honnibal	07ecf5d2f4	* Fixed group_by, removed idea of general attr_of function.	2014-08-22 00:02:37 +02:00
Matthew Honnibal	811b7a6b91	* Struggling with arbitrary attr access...	2014-08-21 23:49:14 +02:00
Matthew Honnibal	314658b31c	* Improve module docstring	2014-08-21 18:42:47 +02:00
Matthew Honnibal	8bcd07dbae	* More docs work	2014-08-21 17:05:28 +02:00
Matthew Honnibal	d10993f41a	* More docs work	2014-08-21 16:37:13 +02:00
Matthew Honnibal	d5403a6fe3	* More docs work	2014-08-21 16:37:06 +02:00
Matthew Honnibal	248cbb6d07	* Update doc strings	2014-08-21 03:29:15 +02:00
Matthew Honnibal	cbda38e2d9	* Improving docs	2014-08-20 21:09:39 +02:00