explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-11-11 12:18:04 +03:00
Code Issues Packages Projects Releases Wiki Activity
5,503 Commits 68 Branches 175 Tags 279 MiB
c25f3133ca
Commit Graph

3135 Commits

Author SHA1 Message Date
Matthew Honnibal
7ee1827af0 Disable data caching in parser 2017-05-19 18:17:11 -05:00
Matthew Honnibal
e84de028b5 Remove 'rebatch' op, and remove min-batch cap 2017-05-19 18:16:36 -05:00
Matthew Honnibal
3376d4d6e8 Update the train script, fixing GPU memory leak 2017-05-19 18:15:50 -05:00
Matthew Honnibal
836fe1d880 Update neural net tests 2017-05-19 18:11:29 -05:00
ines
fe5d8819ea Update Matcher docstrings and API docs 2017-05-19 21:47:06 +02:00
Matthew Honnibal
08766240c3 Add incomplete iob converter 2017-05-19 13:27:51 -05:00
Matthew Honnibal
c12ab47a56 Remove state argument in pipeline. Other changes 2017-05-19 13:26:36 -05:00
Matthew Honnibal
66ea9aebe7 Remove the state argument from Language 2017-05-19 13:25:42 -05:00
Matthew Honnibal
09a877886b WIP on iob converter 2017-05-19 13:24:39 -05:00
ines
a804045597 Use is_ancestor instead of deprecated is_ancestor_of 2017-05-19 20:23:40 +02:00
Matthew Honnibal
8d5e6d9f4f Rename no_ner arg to no_entities 2017-05-19 13:23:11 -05:00
ines
e9e62b01b0 Update docstrings and API docs for Token 2017-05-19 18:47:56 +02:00
ines
62ceec4fc6 Update docstrings and API docs for Span 2017-05-19 18:47:46 +02:00
ines
23f9a3ccc8 Update docstrings and API docs for Doc 2017-05-19 18:47:39 +02:00
ines
2c8c9dc0c9 Update docstrings and API docs for Language 2017-05-19 18:47:24 +02:00
ines
0791f0aae6 Update docstrings and API docs for Span class 2017-05-19 00:31:31 +02:00
ines
8455cb1327 Update docstring for Doc.__getitem__ 2017-05-19 00:30:51 +02:00
ines
0fc05e54e4 Document TokenVectorEncoder 2017-05-19 00:00:02 +02:00
ines
b687ad109d Update docstrings and API docs for Doc class 2017-05-18 23:59:44 +02:00
ines
d42bc16868 Update docstrings and API docs for Language class 2017-05-18 23:57:38 +02:00
ines
593361ee3c Update docstrings for Span class 2017-05-18 22:17:41 +02:00
ines
b87066ff10 Update docstrings and API docs for Doc class 2017-05-18 22:17:41 +02:00
Matthew Honnibal
238be0f16a Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-05-18 08:32:22 -05:00
Matthew Honnibal
c214c0decb Improve env_opt reporting 2017-05-18 08:32:03 -05:00
Matthew Honnibal
bbb59e371c Fix GPU evaluation 2017-05-18 08:31:15 -05:00
Matthew Honnibal
c2c825127a Fix use_params and pipe methods 2017-05-18 08:30:59 -05:00
Matthew Honnibal
ca70b08661 Fix GPU training and evaluation 2017-05-18 08:30:33 -05:00
ines
489d2fb4ba Add is_in_jupyter() helper for displaCy (see #1058) 2017-05-18 14:13:14 +02:00
ines
abf0188b0a Move cupy and CudaStream to compat 2017-05-18 14:12:45 +02:00
ines
33decd85b6 Reorganise and explicitly state what's importable 2017-05-18 14:12:31 +02:00
Matthew Honnibal
a438cef8c5 Fix significant bug in feature calculation -- off by 1 2017-05-18 06:21:32 -05:00
Matthew Honnibal
fc8d3a112c Add util.env_opt support: Can set hyper params through environment variables. 2017-05-18 04:36:53 -05:00
Matthew Honnibal
d2626fdb45 Fix name error in nn parser 2017-05-18 04:31:01 -05:00
Matthew Honnibal
b460533827 Bug fixes to pipeline 2017-05-18 04:29:51 -05:00
Matthew Honnibal
8815507f8e Move SpanishDefaults out of Language class, for pickle 2017-05-18 04:28:51 -05:00
Matthew Honnibal
2713041571 Fix GPU usage in Language 2017-05-18 04:25:19 -05:00
Matthew Honnibal
711ad5edc4 Cache features in doc2feats 2017-05-18 04:22:20 -05:00
Matthew Honnibal
39ea38c4b1 Add option to use gpu to spacy train 2017-05-18 04:21:49 -05:00
Matthew Honnibal
a1d8e420b5 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-05-17 08:00:04 -05:00
Matthew Honnibal
edfea3a513 Fix progress bar 2017-05-17 14:59:37 +02:00
Matthew Honnibal
0b7fd67408 Fix style check in displacy 2017-05-17 07:57:24 -05:00
Matthew Honnibal
55dab77de8 Add conversion rule for .conll 2017-05-17 13:13:48 +02:00
Matthew Honnibal
692bd2a186 Bug fix to tagger: wasnt backproping to token vectors 2017-05-17 13:13:14 +02:00
Matthew Honnibal
877f83807f Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-05-17 12:09:29 +02:00
Matthew Honnibal
793430aa7a Get spaCy train command working with neural network 
* Integrate models into pipeline
* Add basic serialization (maybe incorrect)
* Fix pickle on vocab
 2017-05-17 12:04:50 +02:00
Matthew Honnibal
3bf4a28d8d Use tag in CoNLL converter, not POS 2017-05-17 12:04:33 +02:00
ines
1a05078c79 Add language-specific syntax iterators to en and de 2017-05-17 12:04:03 +02:00
Matthew Honnibal
c9a5d5d24b Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-05-16 16:22:05 +02:00
Matthew Honnibal
8cf097ca88 Redesign training to integrate NN components 
* Obsolete .parser, .entity etc names in favour of .pipeline
* Components no longer create models on initialization
* Models created by loading method (from_disk(), from_bytes() etc), or
    .begin_training()
* Add .predict(), .set_annotations() methods in components
* Pass state through pipeline, to allow components to share information
    more flexibly.
 2017-05-16 16:17:30 +02:00
Matthew Honnibal
221b4c1ee8 Fix test for Python 3 2017-05-16 13:06:30 +02:00
Matthew Honnibal
5211645af3 Get data flowing through pipeline. Needs redesign 2017-05-16 11:21:59 +02:00
Matthew Honnibal
1d7c18e58a Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2017-05-15 21:53:47 +02:00
Matthew Honnibal
a9edb3aa1d Improve integration of NN parser, to support unified training API 2017-05-15 21:53:27 +02:00
ines
98354be150 Only get user_data if it exists on doc 2017-05-15 13:39:47 +02:00
ines
c33bdeb564 Use uppercase for entity types 2017-05-15 01:24:57 +02:00
ines
4aaa607b8d Add xmlns:xlink so SVGs are rendered properly as individual files 2017-05-14 19:54:13 +02:00
ines
9dd13cd76a Update docstrings 2017-05-14 19:30:47 +02:00
ines
a04550605a Add Jupyter notebook support (see #1058) 2017-05-14 18:39:01 +02:00
ines
c31792aaec Add displaCy visualisers (see #1058) 2017-05-14 17:50:23 +02:00
ines
b462076d80 Merge load_lang_class and get_lang_class 2017-05-14 01:31:10 +02:00
ines
36bebe7164 Update docstrings 2017-05-14 01:30:29 +02:00
Matthew Honnibal
4b9d69f428 Merge branch 'v2' into develop 
* Move v2 parser into nn_parser.pyx
* New TokenVectorEncoder class in pipeline.pyx
* New spacy/_ml.py module

Currently the two parsers live side-by-side, until we figure out how to
organize them.
 2017-05-14 01:10:23 +02:00
Matthew Honnibal
5cac951a16 Move new parser to nn_parser.pyx, and restore old parser, to make tests pass. 2017-05-14 00:55:01 +02:00
Matthew Honnibal
f8c02b4341 Remove cupy imports from parser, so it can work on CPU 2017-05-14 00:37:53 +02:00
Matthew Honnibal
613ba79e2e Fiddle with sizings for parser 2017-05-13 17:20:23 -05:00
Matthew Honnibal
e6d71e1778 Small fixes to parser 2017-05-13 17:19:04 -05:00
Matthew Honnibal
188c0f6949 Clean up unused import 2017-05-13 17:18:27 -05:00
Matthew Honnibal
f85c8464f7 Draft support of regression loss in parser 2017-05-13 17:17:27 -05:00
ines
1694c24e52 Add docstrings, error messages and fix consistency 2017-05-13 21:22:49 +02:00
ines
ee7dcf65c9 Fix expand_exc to make sure it returns combined dict 2017-05-13 21:22:25 +02:00
ines
824d09bb74 Move resolve_load_name to deprecated 2017-05-13 21:21:47 +02:00
ines
a4a37a783e Remove import from non-existing module 2017-05-13 16:00:09 +02:00
ines
5858857a78 Update languages list in conftest 2017-05-13 15:37:54 +02:00
ines
9d85cda8e4 Fix models error message and use about.__docs_models__ (see #1051) 2017-05-13 13:05:47 +02:00
ines
6b942763f0 Tidy up imports 2017-05-13 13:04:40 +02:00
ines
8c2a0c026d Fix parse_tree test 2017-05-13 12:32:45 +02:00
ines
6129016e15 Replace deepcopy 2017-05-13 12:32:37 +02:00
ines
df68bf45ce Set defaults for light and flat kwargs 2017-05-13 12:32:23 +02:00
ines
b9dea345e5 Remove old import 2017-05-13 12:32:11 +02:00
ines
293ee359c5 Fix formatting 2017-05-13 12:32:06 +02:00
ines
4eefb288e3 Port over PR #1055 2017-05-13 03:25:32 +02:00
Matthew Honnibal
ee1d35bdb0 Fix merge conflict 2017-05-13 03:20:19 +02:00
Matthew Honnibal
b2540d2379 Merge Kengz's tree_print patch 2017-05-13 03:18:49 +02:00
Matthew Honnibal
827b5af697 Update draft of parser neural network model 
Model is good, but code is messy. Currently requires Chainer, which may cause the build to fail on machines without a GPU.

Outline of the model:

We first predict context-sensitive vectors for each word in the input:

(embed_lower | embed_prefix | embed_suffix | embed_shape)
>> Maxout(token_width)
>> convolution ** 4

This convolutional layer is shared between the tagger and the parser. This prevents the parser from needing tag features.
To boost the representation, we make a "super tag" with POS, morphology and dependency label. The tagger predicts this
by adding a softmax layer onto the convolutional layer --- so, we're teaching the convolutional layer to give us a
representation that's one affine transform from this informative lexical information. This is obviously good for the
parser (which backprops to the convolutions too).

The parser model makes a state vector by concatenating the vector representations for its context tokens. Current
results suggest few context tokens works well. Maybe this is a bug.

The current context tokens:

* S0, S1, S2: Top three words on the stack
* B0, B1: First two words of the buffer
* S0L1, S0L2: Leftmost and second leftmost children of S0
* S0R1, S0R2: Rightmost and second rightmost children of S0
* S1L1, S1L2, S1R2, S1R, B0L1, B0L2: Likewise for S1 and B0

This makes the state vector quite long: 13*T, where T is the token vector width (128 is working well). Fortunately,
there's a way to structure the computation to save some expense (and make it more GPU friendly).

The parser typically visits 2*N states for a sentence of length N (although it may visit more, if it back-tracks
with a non-monotonic transition). A naive implementation would require 2*N (B, 13*T) @ (13*T, H) matrix multiplications
for a batch of size B. We can instead perform one (B*N, T) @ (T, 13*H) multiplication, to pre-compute the hidden
weights for each positional feature wrt the words in the batch. (Note that our token vectors come from the CNN
-- so we can't play this trick over the vocabulary. That's how Stanford's NN parser works --- and why its model
is so big.)

This pre-computation strategy allows a nice compromise between GPU-friendliness and implementation simplicity.
The CNN and the wide lower layer are computed on the GPU, and then the precomputed hidden weights are moved
to the CPU, before we start the transition-based parsing process. This makes a lot of things much easier.
We don't have to worry about variable-length batch sizes, and we don't have to implement the dynamic oracle
in CUDA to train.

Currently the parser's loss function is multilabel log loss, as the dynamic oracle allows multiple states to
be 0 cost. This is defined as:

(exp(score) / Z) - (exp(score) / gZ)

Where gZ is the sum of the scores assigned to gold classes. I'm very interested in regressing on the cost directly,
but so far this isn't working well.

Machinery is in place for beam-search, which has been working well for the linear model. Beam search should benefit
greatly from the pre-computation trick.
 2017-05-12 16:09:15 -05:00
ines
c4857bc7db Remove unused argument 2017-05-12 15:37:54 +02:00
ines
c13b3fa052 Add LEX_ATTRS 2017-05-12 15:37:45 +02:00
ines
bca2ea9c72 Update Portuguese lexical attributes 2017-05-12 15:37:39 +02:00
ines
2f870123bf Fix formatting 2017-05-12 15:37:20 +02:00
ines
ca65993d59 Add basic Polish Language class 2017-05-12 09:25:37 +02:00
ines
48177c4f92 Add missing tokenizer exceptions 2017-05-12 09:25:24 +02:00
ines
bb8be3d194 Add Danish language data 2017-05-10 21:15:12 +02:00
Matthew Honnibal
4efb391994 Fix serializer 2017-05-09 18:45:18 +02:00
Matthew Honnibal
b16ae75824 Remove serializer hacks from pipeline classes 2017-05-09 18:16:40 +02:00
Matthew Honnibal
7253b4e649 Remove old serialization tests 2017-05-09 18:12:58 +02:00
Matthew Honnibal
f9327343ce Start updating serializer test 2017-05-09 18:12:03 +02:00
Matthew Honnibal
1166b0c491 Implement Doc.to_bytes and Doc.from_bytes methods 2017-05-09 18:11:34 +02:00
Matthew Honnibal
9e167b7bb6 Strip serializer from code 2017-05-09 17:28:50 +02:00
Matthew Honnibal
b53f7dfdc3 Remove spacy.serialize 2017-05-09 17:22:06 +02:00
Matthew Honnibal
62ecdea9f2 Add binder class for document serialization 2017-05-09 17:21:00 +02:00
ines
a0b00624bb Make sure like_email returns bool 2017-05-09 11:37:29 +02:00
ines
ea60932e1b Fix formatting 2017-05-09 11:08:14 +02:00
ines
2c3bdd09b1 Add English test for like_num 2017-05-09 11:06:34 +02:00
ines
22375eafb0 Fix and merge attrs and lex_attrs tests 2017-05-09 11:06:25 +02:00
ines
02d0ac5cab Remove redundant function and fix formatting 2017-05-09 11:06:04 +02:00
ines
b5ca50607e Reorganise entity rules 2017-05-09 01:37:10 +02:00
ines
564939391a Remove spacy.orth 2017-05-09 01:21:47 +02:00
ines
12c3d5fbba Fix formatting 2017-05-09 01:15:28 +02:00
ines
2829a024ef Re-add basic like_num check to global lex_attrs 2017-05-09 01:15:23 +02:00
ines
88adeee548 Add English lex_attrs overrides 2017-05-09 01:09:52 +02:00
ines
8f3fbbb147 Fix typos 2017-05-09 01:09:37 +02:00
ines
ea5fa46475 Import LEX_ATTRS from lang.lex_attrs 2017-05-09 00:58:10 +02:00
ines
2216e5f326 Reorganise lex_attrs and add dict 2017-05-09 00:57:54 +02:00
ines
e666f14d20 Add global lex_attrs 2017-05-09 00:41:53 +02:00
ines
41972c43fe Use consistent regex imports 2017-05-09 00:34:31 +02:00
ines
7b83977020 Remove unused munge package 2017-05-09 00:16:16 +02:00
ines
c714841cc8 Move language-specific tests to tests/lang 2017-05-09 00:02:37 +02:00
ines
bd57b611cc Update conftest to lazy load languages 2017-05-09 00:02:21 +02:00
ines
9f0fd5963f Reorganise Hungarian punctuation rules 2017-05-09 00:01:59 +02:00
ines
fc0d793360 Reorganise Bengali punctuation rules 2017-05-09 00:01:52 +02:00
ines
e895d1afd7 Reorganise French punctuation rules 2017-05-09 00:00:54 +02:00
ines
014bda0ae3 Reorganise global punctuation rules 2017-05-09 00:00:46 +02:00
ines
a91278cb32 Rename _URL_PATTERN to URL_PATTERN 2017-05-09 00:00:00 +02:00
ines
604f299cf6 Add char classes to global language data 2017-05-08 23:59:33 +02:00
ines
f6f5d78cb9 Fix formatting 2017-05-08 23:59:17 +02:00
ines
6eb6306843 Fix language data imports 2017-05-08 23:58:31 +02:00
ines
3c0f85de8e Remove imports in /lang/__init__.py 2017-05-08 23:58:07 +02:00
ines
86d9c29f30 Reorder util functions 2017-05-08 23:51:15 +02:00
ines
9a0d2fdef1 Add load_lang_class() util function 2017-05-08 23:50:45 +02:00
ines
614aa09582 Tidy up Bengali tokenizer exceptions 2017-05-08 22:29:49 +02:00
ines
73b577cb01 Fix relative imports 2017-05-08 22:29:04 +02:00
ines
ae99990f63 Fix formatting 2017-05-08 22:23:48 +02:00
ines
f46ffe3e89 Move language data to /lang module 2017-05-08 20:00:40 +02:00
ines
41a322c733 Fix LEMMA in exceptions and morph rules 2017-05-08 19:57:36 +02:00
ines
2edc0aee12 Update warning message 2017-05-08 19:53:36 +02:00
ines
6025cdb992 Fix string interpolation in times 2017-05-08 16:38:16 +02:00
ines
b9ba58ba5c Add function to resolve load name 
Warn if old 'path' keyword argument is used.
 2017-05-08 16:33:37 +02:00
ines
e6f1a5d0a1 Add unicode declaration 2017-05-08 16:22:17 +02:00
ines
be5541bd16 Fix import and tokenizer exceptions 2017-05-08 16:20:14 +02:00
ines
2324788970 Remove bad tests 2017-05-08 16:15:27 +02:00
ines
b88c4193e7 Add missing symbol 2017-05-08 16:15:20 +02:00
ines
9a5b2bdd4c Don't set morph rules without tag map 2017-05-08 16:15:12 +02:00
ines
4930f0fa8f Explicitly import TOKEN_MATCH 2017-05-08 16:11:54 +02:00
ines
50b7ec03ca Fix typo 2017-05-08 16:11:45 +02:00
ines
3ca611fe48 Fix wildcard imports 2017-05-08 15:56:29 +02:00
ines
c2469b8135 Remove __all__ export 2017-05-08 15:56:22 +02:00
ines
14a9c3ee7a Fix wildcard import 2017-05-08 15:56:13 +02:00
ines
deed623864 Remove comment 2017-05-08 15:56:05 +02:00
ines
e7f95c37ee Merge base tokenizer exceptions 2017-05-08 15:55:52 +02:00
ines
24606d364c Remove redundant language_data.py files in languages 
Originally intended to collect all components of a language, but just
made things messy. Now each component is in charge of exporting itself
properly.
 2017-05-08 15:55:29 +02:00
ines
a627d3e3b0 Reorganise Chinese language data 2017-05-08 15:54:36 +02:00