Matthew Honnibal
bb583f7f09
* Hackishly patch long dependencies problem
2015-07-27 23:14:33 +02:00
Matthew Honnibal
aa7a964a4f
* Add a type declaration for doc.from_array
2015-07-27 22:57:22 +02:00
Matthew Honnibal
25a8774f42
* Fix regression in packer
2015-07-27 21:53:38 +02:00
Matthew Honnibal
1601e488ee
* Fix bug in decoding non-ascii characters
2015-07-27 21:43:58 +02:00
Matthew Honnibal
6a95409cd2
* Fix type on bits
2015-07-27 21:16:49 +02:00
Matthew Honnibal
a296d72b54
* Fix en/attrs
2015-07-27 21:16:33 +02:00
Matthew Honnibal
45460f505c
* Fix data type on read32 in BitArray
2015-07-27 21:12:13 +02:00
Matthew Honnibal
3d43f49f69
* Revert prev change
2015-07-27 10:58:15 +02:00
Matthew Honnibal
6b586cdad4
* Change lexemes.bin format. Add a header specifying size of LexemeC and number of lexemes, and don't have the redundant orth information.
2015-07-27 08:31:51 +02:00
Matthew Honnibal
af6ed18f2a
* Ensure we don't use orth_encode on OOV words.
2015-07-27 02:12:01 +02:00
Matthew Honnibal
8535d872e8
* Set is_oov property in get_flags
2015-07-27 01:51:24 +02:00
Matthew Honnibal
8e4c69ee8c
* Add is_oov property, and fix up handling of attributes
2015-07-27 01:50:06 +02:00
Matthew Honnibal
fc268f03eb
* Assert against null pointer exceptions in vocab
2015-07-27 01:00:10 +02:00
Matthew Honnibal
0f093fdb30
* Fix get_by_orth for py3
2015-07-26 19:26:41 +02:00
Matthew Honnibal
ceeda5a739
* Fix get_by_orth for py3
2015-07-26 18:39:27 +02:00
Matthew Honnibal
6bb96c122d
* Host IS_ flags in attrs.pxd, and add properties for them on Token and Lexeme objects
2015-07-26 16:37:16 +02:00
Matthew Honnibal
eeaea25f0c
* Check oov_prob file is present
2015-07-26 16:36:38 +02:00
Matthew Honnibal
7eb2446082
* Return empty lexeme on empty string
2015-07-26 00:18:30 +02:00
Matthew Honnibal
1b5d1da2a7
* Allow an OOV probability to be specified in get_lex_props
2015-07-26 00:03:43 +02:00
Matthew Honnibal
cd6e25132b
* Allow an OOV probability to be specified in get_lex_props
2015-07-26 00:01:46 +02:00
Matthew Honnibal
fd525f0675
* Pass OOV probability around
2015-07-25 23:29:51 +02:00
Matthew Honnibal
3fe14b8ed6
* Fix CFile for Python2
2015-07-25 22:55:53 +02:00
Matthew Honnibal
823ef4a00b
* Remove profile declarations
2015-07-25 18:13:06 +02:00
Matthew Honnibal
f4809e562f
* Allow json to be used as a fallback if ujson is not available
2015-07-25 18:11:36 +02:00
Matthew Honnibal
9da06671cf
* Remove unused import
2015-07-25 18:11:16 +02:00
Matthew Honnibal
2060935cdb
* Remove explicit bytes type in doc.from_bytes, to accept bytearray
2015-07-24 04:54:13 +02:00
Matthew Honnibal
aa28e2e01d
* Release the GIL around parse function
2015-07-24 04:53:27 +02:00
Matthew Honnibal
d62eb34b76
* More Py 2/3 compatibility in bit strings
2015-07-24 04:52:06 +02:00
Matthew Honnibal
0bb839d299
* Fix string coercion for Python 3
2015-07-24 03:49:30 +02:00
Matthew Honnibal
c4ff410fdb
* Fix bytes problems for Python3
2015-07-24 03:48:23 +02:00
Matthew Honnibal
1ab25e4dad
* Fix python3 type error
2015-07-24 02:45:34 +02:00
Matthew Honnibal
f35ff173b0
* Fix bits.pyx unicode error
2015-07-23 20:37:57 +02:00
Matthew Honnibal
1406e24327
* Fix unicode error for Python3
2015-07-23 19:36:21 +02:00
Matthew Honnibal
dbda6c27fa
* Fix python3 error
2015-07-23 14:52:30 +02:00
Matthew Honnibal
99387f9572
* Fix python3 error
2015-07-23 14:30:29 +02:00
Matthew Honnibal
b81ffe9032
* Fix typing on mode string in CFile
2015-07-23 13:24:43 +02:00
Matthew Honnibal
22028602a9
* Add unicode_literals declaration in vocab.pyx
2015-07-23 13:24:20 +02:00
Matthew Honnibal
5b41744270
* Check for directory presence before loading annotators
2015-07-23 09:27:37 +02:00
Matthew Honnibal
df01a88763
Merge branch 'refactor' (and serializaton)
...
Add Huffman-code serialization, and do a lot of
refactoring. Highlights include:
* Much more efficient StringStore
* Vocab maintains a by-orth mapping of Lexemes
* Avoid manually slicing Py_UNICODE buffers,
simplifying tokenizer and vocab C APIs
* Remove various bits of dead code
* Work on removing GIL around parser
* Work on bridge to Theano
Conflicts:
spacy/strings.pxd
spacy/strings.pyx
spacy/structs.pxd
2015-07-23 02:18:35 +02:00
Matthew Honnibal
a7c4d72e83
* Add serializer property to Vocab, and lazy-load it. Add get_by_orth method.
2015-07-23 01:18:19 +02:00
Matthew Honnibal
6ab1696b15
* Remove read_encoding_freqs from util.py
2015-07-23 01:17:32 +02:00
Matthew Honnibal
d5255aad77
* Update freqs for missing tags in ner, for serializer
2015-07-23 01:17:11 +02:00
Matthew Honnibal
12699a1152
* Set initial freqs, to avoid missing values in serializer
2015-07-23 01:16:27 +02:00
Matthew Honnibal
680bb47b55
* Write serializer freqs to single file, vocab/serializer.json
2015-07-23 01:15:25 +02:00
Matthew Honnibal
a0e36e8efc
* Add working to/from bytes API to Doc
2015-07-23 01:14:45 +02:00
Matthew Honnibal
1f31d96bf9
* Fix Packer API, so that it reads and writes bytes strings, instead of BitArray. Docs are always byte aligned anyway.
2015-07-23 01:13:02 +02:00
Matthew Honnibal
38ef986b29
* Update spacy/en/attrs.pxd
2015-07-23 01:10:58 +02:00
Matthew Honnibal
06eac32610
* Add cfile.pyx
2015-07-23 01:10:36 +02:00
Matthew Honnibal
0c507bd80a
* Fix tokenizer
2015-07-22 14:10:30 +02:00
Matthew Honnibal
c86dbe4944
* Update English.save_models for new Packer save/load stuff
2015-07-22 13:40:23 +02:00
