Commit Graph

  • ef448649b3 * Add read_freqs function in init_model Matthew Honnibal 2015-07-25 22:16:36 +0200
  • 2e6a60eaec Merge branch 'master' of https://github.com/honnibal/spaCy Matthew Honnibal 2015-07-25 21:14:07 +0200
  • 105305b4aa * Upd get_freqs script Matthew Honnibal 2015-07-25 21:13:41 +0200
  • 616445e027 * Add simple script to collate frequencies from sorted file Matthew Honnibal 2015-07-25 21:12:45 +0200
  • 823ef4a00b * Remove profile declarations Matthew Honnibal 2015-07-25 18:12:25 +0200
  • f4809e562f * Allow json to be used as a fallback if ujson is not available Matthew Honnibal 2015-07-25 18:11:36 +0200
  • 9da06671cf * Remove unused import Matthew Honnibal 2015-07-25 18:11:16 +0200
  • 2060935cdb * Remove explicit bytes type in doc.from_bytes, to accept bytearray Matthew Honnibal 2015-07-24 04:54:13 +0200
  • aa28e2e01d * Release the GIL around parse function Matthew Honnibal 2015-07-24 04:53:27 +0200
  • 287d90e792 * Use thinc 3.3 Matthew Honnibal 2015-07-24 04:52:50 +0200
  • c52179f5fa * Use print function in train.py, for py 2/3 compatibility Matthew Honnibal 2015-07-24 04:52:35 +0200
  • d62eb34b76 * More Py 2/3 compatibility in bit strings Matthew Honnibal 2015-07-24 04:52:06 +0200
  • 6d0cdb1630 * Py 2/3 compatibility of serialize tests Matthew Honnibal 2015-07-24 04:51:53 +0200
  • 0bb839d299 * Fix string coercion for Python 3 Matthew Honnibal 2015-07-24 03:49:30 +0200
  • c4ff410fdb * Fix bytes problems for Python3 Matthew Honnibal 2015-07-24 03:48:23 +0200
  • ce984f471c * Update tests for python3 Matthew Honnibal 2015-07-24 03:47:59 +0200
  • 1ab25e4dad * Fix python3 type error Matthew Honnibal 2015-07-24 02:45:34 +0200
  • 35a9fb437f * Fix travis.yml Matthew Honnibal 2015-07-24 01:43:27 +0200
  • f35ff173b0 * Fix bits.pyx unicode error Matthew Honnibal 2015-07-23 20:37:57 +0200
  • 1406e24327 * Fix unicode error for Python3 Matthew Honnibal 2015-07-23 19:36:21 +0200
  • dbda6c27fa * Fix python3 error Matthew Honnibal 2015-07-23 14:52:30 +0200
  • 99387f9572 * Fix python3 error Matthew Honnibal 2015-07-23 14:30:29 +0200
  • b81ffe9032 * Fix typing on mode string in CFile Matthew Honnibal 2015-07-23 13:24:43 +0200
  • 22028602a9 * Add unicode_literals declaration in vocab.pyx Matthew Honnibal 2015-07-23 13:24:20 +0200
  • 6be3ee311c Py3 compatibility tweak Matthew Honnibal 2015-07-23 13:13:15 +0200
  • d4407d8e2f Py3 compatibility tweak Matthew Honnibal 2015-07-23 09:45:15 +0200
  • 8d7e044e2c * Add clusters file Matthew Honnibal 2015-07-23 09:35:56 +0200
  • 5b41744270 * Check for directory presence before loading annotators Matthew Honnibal 2015-07-23 09:27:37 +0200
  • da4821fc14 * Add cluster words to probs in init_model Matthew Honnibal 2015-07-23 09:27:07 +0200
  • bee2e77983 * Update tests Matthew Honnibal 2015-07-23 09:26:43 +0200
  • 4af2595d99 * Fix structure of wordnet directory for init_model Matthew Honnibal 2015-07-23 06:35:38 +0200
  • 16617142b7 * Fix travis.yml Matthew Honnibal 2015-07-23 06:15:01 +0200
  • 24b98bc1c8 * Fix travis.yml Matthew Honnibal 2015-07-23 06:06:13 +0200
  • 1364d9c74e * Fix travis.yml Matthew Honnibal 2015-07-23 05:58:26 +0200
  • 2c1aae3cfb * Fix travis.yml Matthew Honnibal 2015-07-23 05:30:33 +0200
  • 16d0e1f598 * Fix travis.yml Matthew Honnibal 2015-07-23 04:41:28 +0200
  • 0c3caae085 * Fix travis.yml Matthew Honnibal 2015-07-23 04:13:54 +0200
  • 58e6adfacd * Fix travis.yml Matthew Honnibal 2015-07-23 04:05:51 +0200
  • c4018d6827 * Fix travis.yml Matthew Honnibal 2015-07-23 03:34:23 +0200
  • 2fb4e932a3 * Fix travis.yml Matthew Honnibal 2015-07-23 03:16:49 +0200
  • fc0e964a5c * Fix travis.yml Matthew Honnibal 2015-07-23 03:00:17 +0200
  • af5bab55b3 * Update travis.yml Matthew Honnibal 2015-07-23 02:52:06 +0200
  • 082ae5aaf7 * Fix travis.yml Matthew Honnibal 2015-07-23 02:38:14 +0200
  • 247e592c2c * Update travis.yml Matthew Honnibal 2015-07-23 02:32:47 +0200
  • 83c0f0da22 * Remove lemmatizer from init_model Matthew Honnibal 2015-07-23 02:32:34 +0200
  • df01a88763 Merge branch 'refactor' (and serializaton) Matthew Honnibal 2015-07-23 02:18:35 +0200
  • 4970ebe887 * Add test_io Matthew Honnibal 2015-07-23 01:19:59 +0200
  • 4729200dfc * Whitespace Matthew Honnibal 2015-07-23 01:19:26 +0200
  • 3a5299faec * Upd tests Matthew Honnibal 2015-07-23 01:19:11 +0200
  • 20c2db08b7 * Add flags to pytest to tests requiring models, vectors or slow functions to be toggled. Matthew Honnibal 2015-07-23 01:19:03 +0200
  • a7c4d72e83 * Add serializer property to Vocab, and lazy-load it. Add get_by_orth method. Matthew Honnibal 2015-07-23 01:18:19 +0200
  • 6ab1696b15 * Remove read_encoding_freqs from util.py Matthew Honnibal 2015-07-23 01:17:32 +0200
  • d5255aad77 * Update freqs for missing tags in ner, for serializer Matthew Honnibal 2015-07-23 01:17:11 +0200
  • 12699a1152 * Set initial freqs, to avoid missing values in serializer Matthew Honnibal 2015-07-23 01:16:27 +0200
  • 680bb47b55 * Write serializer freqs to single file, vocab/serializer.json Matthew Honnibal 2015-07-23 01:15:25 +0200
  • a0e36e8efc * Add working to/from bytes API to Doc Matthew Honnibal 2015-07-23 01:14:45 +0200
  • 1f31d96bf9 * Fix Packer API, so that it reads and writes bytes strings, instead of BitArray. Docs are always byte aligned anyway. Matthew Honnibal 2015-07-23 01:12:00 +0200
  • 38ef986b29 * Update spacy/en/attrs.pxd Matthew Honnibal 2015-07-23 01:10:58 +0200
  • 06eac32610 * Add cfile.pyx Matthew Honnibal 2015-07-23 01:10:36 +0200
  • 2b7bd46508 * Update get_freqs script Matthew Honnibal 2015-07-22 15:43:06 +0200
  • 0c507bd80a * Fix tokenizer Matthew Honnibal 2015-07-22 14:10:30 +0200
  • c86dbe4944 * Update English.save_models for new Packer save/load stuff Matthew Honnibal 2015-07-22 13:40:23 +0200
  • bf77bcd6b9 * Add comment explaining hash_string Matthew Honnibal 2015-07-22 13:39:42 +0200
  • 815bda201d * Remove UniStr struct Matthew Honnibal 2015-07-22 13:39:17 +0200
  • 2fc66e3723 * Use Py_UNICODE in tokenizer for now, while sort out Py_UCS4 stuff Matthew Honnibal 2015-07-22 13:38:45 +0200
  • 4d61239eac * Reorganize the serialization functions on Doc Matthew Honnibal 2015-07-22 04:53:01 +0200
  • 109106a949 * Replace UniStr, using unicode objects instead Matthew Honnibal 2015-07-22 04:49:39 +0200
  • 386246db5b * Update init_model, making language resources optional Matthew Honnibal 2015-07-22 00:25:14 +0200
  • 424854028f * Fix decode_int32 Matthew Honnibal 2015-07-21 20:09:59 +0000
  • 304d0e2633 * Use decode_int32 in _orth_decode Matthew Honnibal 2015-07-21 20:40:55 +0200
  • 9cfa59ec33 * Optimistically try orth encoding, with char as a back-off Matthew Honnibal 2015-07-21 20:22:45 +0200
  • c8b89e37a5 * Bug fix to faster huffman decoding Matthew Honnibal 2015-07-21 20:05:53 +0200
  • b166d1d2a2 * Use encode32 and decode32 Matthew Honnibal 2015-07-21 19:59:06 +0200
  • c6cd0ddce8 * Add faster encode_int32 and decode_int32 methods Matthew Honnibal 2015-07-21 19:58:45 +0200
  • dd60594f41 * Fix double encoding error in strings.pyx Matthew Honnibal 2015-07-20 13:52:56 +0200
  • 9cae1b4cad * Restore accidentally clobbered updates to specials.json Matthew Honnibal 2015-07-20 12:19:46 +0200
  • 14e9e6ec6c * Fix ... tokenization, and correct orth inconsistencies in specials.json Matthew Honnibal 2015-07-20 12:09:34 +0200
  • 06639dc497 * Add length cap to word shape feature Matthew Honnibal 2015-07-20 12:06:59 +0200
  • 128b6d9714 * Move Utf8Str struct to strings module, as that's the only place it's relevant Matthew Honnibal 2015-07-20 12:06:41 +0200
  • 01a97b90f3 * Fix header for string store Matthew Honnibal 2015-07-20 12:06:10 +0200
  • 1c9ea7b835 * Add tests for short string optimization Matthew Honnibal 2015-07-20 12:05:45 +0200
  • 52d538ea42 * Fix short string optimization in strings.pyx. StringStore tests now all pass. Matthew Honnibal 2015-07-20 12:05:23 +0200
  • 09a3055630 * Work on short string optimization in Utf8Str Matthew Honnibal 2015-07-20 11:26:46 +0200
  • bb0ba1f0cd * Improve serialization speed Matthew Honnibal 2015-07-20 03:27:59 +0200
  • f13d5dae91 * Update test_packer Matthew Honnibal 2015-07-20 01:38:29 +0200
  • fb7202a173 * Update test_codecs Matthew Honnibal 2015-07-20 01:38:15 +0200
  • 8743a8c084 * Update Doc serialization for new Packer interface Matthew Honnibal 2015-07-20 01:38:04 +0200
  • 1f7170e0e1 * Reinstate the fixed vocabulary --- words are only added to the lexicon in init_model, after that we create LexemeC structs with the Pool given to us. Matthew Honnibal 2015-07-20 01:37:34 +0200
  • 5a7d060d9c * Switch between the orth and char codecs depending on which is shorter for that message. Mostly orth is shorter, except if there are OOV words. Matthew Honnibal 2015-07-20 01:36:22 +0200
  • 5a042ee0d3 * Add function to predict number of bits needed to encode message Matthew Honnibal 2015-07-20 01:35:11 +0200
  • b89b489bb4 * Implement both character and orth encoding in Packer, so that we can decide which to use per-text Matthew Honnibal 2015-07-19 22:39:45 +0200
  • ae78c9e3ce * Implement character-based codec, so that we can do word/char backoff Matthew Honnibal 2015-07-19 22:03:39 +0200
  • cd1d047cb8 * Delete out-dated HuffmanCodec comment Matthew Honnibal 2015-07-19 18:28:14 +0200
  • 879ef9fa3e * Update tests for huffman codec Matthew Honnibal 2015-07-19 17:59:51 +0200
  • b8086067d5 * Build Huffman codec from unsorted inputs Matthew Honnibal 2015-07-19 17:58:44 +0200
  • 317cbbc015 * Serialization round trip now working with decent API, but with rough spots in the organisation and requiring vocabulary to be fixed ahead of time. Matthew Honnibal 2015-07-19 15:18:17 +0200
  • 0973e2f107 * Update serializer tests Matthew Honnibal 2015-07-18 22:46:40 +0200
  • 6b13e7227c * Remove duplicate get_lex_attr method from doc.pyx Matthew Honnibal 2015-07-18 22:46:07 +0200
  • e49c7f1478 * Update oov check in tokenizer Matthew Honnibal 2015-07-18 22:45:28 +0200
  • cfd842769e * Allow infix tokens to be variable length Matthew Honnibal 2015-07-18 22:45:00 +0200