explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-01-13 18:56:36 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,299 Commits 71 Branches 177 Tags 279 MiB
9f6ad06452
Commit Graph

7900 Commits

Author SHA1 Message Date
Ines Montani
58dde293ce
Merge pull request #6089 from adrianeboyd/feature/doc-ents-v3-2 2020-09-24 14:44:42 +02:00
Ines Montani
74e1f192b4
Merge pull request #6134 from explosion/feature/training_before_to_disk 2020-09-24 14:44:11 +02:00
Ines Montani
24e7ac3f2b Fix download CLI [ci skip] 2020-09-24 14:43:56 +02:00
Ines Montani
88e54caa12 accuracy -> performance 2020-09-24 14:32:35 +02:00
Ines Montani
92f8b6959a Fix typo 2020-09-24 13:48:41 +02:00
Adriane Boyd
5c13e0cf1b Remove unused error 2020-09-24 13:41:55 +02:00
Ines Montani
be56c0994b Add [training.before_to_disk] callback 2020-09-24 12:40:25 +02:00
Adriane Boyd
8eaacaae97 Refactor Doc.ents setter to use Doc.set_ents 
Additional changes:

* Entity spans with missing labels are ignored
* Fix ent_kb_id setting in `Doc.set_ents`
 2020-09-24 12:36:51 +02:00
Ines Montani
c6c67b606e
Merge pull request #6133 from explosion/fix/score_weights 2020-09-24 12:00:57 +02:00
Ines Montani
f69fea8b25 Improve error handling around non-number scores 2020-09-24 11:29:07 +02:00
Ines Montani
4eb39b5c43 Fix logging 2020-09-24 11:04:35 +02:00
Ines Montani
4bbe41f017 Fix combined scores and update test 2020-09-24 10:42:47 +02:00
Sofie Van Landeghem
c645c4e7ce
fix micro PRF for textcat (#6130) 
* fix micro PRF for textcat

* small fix
 2020-09-24 10:31:17 +02:00
Matthew Honnibal
17a6b0a173
Make project pull order insensitive (#6131) 2020-09-24 10:30:42 +02:00
Ines Montani
ae51f580c1 Fix handling of score_weights 2020-09-24 10:27:33 +02:00
Ines Montani
f25f05c503 Adjust sort order [ci skip] 2020-09-23 20:03:04 +02:00
Ines Montani
3f77eb749c Increment version [ci skip] 2020-09-23 19:50:15 +02:00
svlandeg
b816ace4bb format 2020-09-23 17:33:13 +02:00
svlandeg
5a9fdbc8ad state_type as Literal 2020-09-23 17:32:14 +02:00
svlandeg
35dbc63578 Merge remote-tracking branch 'upstream/develop' into fix/nr_features 
# Conflicts:
#	spacy/ml/models/parser.py
#	spacy/tests/serialize/test_serialize_config.py
#	website/docs/api/architectures.md
 2020-09-23 17:01:13 +02:00
svlandeg
25b34bba94 throw custom error when state_type is invalid 2020-09-23 16:57:14 +02:00
Ines Montani
916050bf2f
Merge pull request #6127 from explosion/feature/literal-nr_feature_tokens 2020-09-23 16:56:08 +02:00
Ines Montani
3c3863654e Increment version [ci skip] 2020-09-23 16:54:43 +02:00
svlandeg
dd2292793f 'parser' instead of 'deps' for state_type 2020-09-23 16:53:49 +02:00
Ines Montani
50a4425cda Adjust docs 2020-09-23 16:03:32 +02:00
Ines Montani
76bbed3466 Use Literal type for nr_feature_tokens 2020-09-23 16:00:03 +02:00
svlandeg
6c85fab316 state_type and extra_state_tokens instead of nr_feature_tokens 2020-09-23 13:35:09 +02:00
Ines Montani
7745d77a38 Fix whitespace in template [ci skip] 2020-09-23 13:21:42 +02:00
svlandeg
6435458d51 simplify expression 2020-09-23 12:12:38 +02:00
svlandeg
20b0ec5dcf avoid logging performance of frozen components 2020-09-23 10:37:12 +02:00
Ines Montani
ae5dacf75f Tidy up and add types 2020-09-23 10:14:34 +02:00
Ines Montani
6ca06cb62c Update docs and formatting [ci skip] 2020-09-23 10:14:27 +02:00
Ines Montani
888f936a73
Merge pull request #6106 from svlandeg/feature/textcat-quickstart 2020-09-23 10:11:45 +02:00
Ines Montani
60a317520a
Merge pull request #6109 from svlandeg/feature/2rename 2020-09-23 09:47:12 +02:00
Ines Montani
f976bab710 Remove empty file [ci skip] 2020-09-23 09:30:09 +02:00
svlandeg
556f3e4652 add pooling to NEL's TransformerListener 2020-09-23 09:24:28 +02:00
svlandeg
4a56ea72b5 fallbacks for old names 2020-09-23 09:15:07 +02:00
Sofie Van Landeghem
86a08f819d
tok2vec.update instead of predict (#6113) 2020-09-22 21:54:52 +02:00
Sofie Van Landeghem
e0e793be4d
fix KB IO (#6118) 2020-09-22 21:53:06 +02:00
Adriane Boyd
b1a7d6c528 Refactor seen token detection 2020-09-22 14:42:51 +02:00
Sofie Van Landeghem
d53c84b6d6
avoid None callback (#6100) 2020-09-22 13:54:44 +02:00
Adriane Boyd
535842e483
Merge branch 'develop' into feature/doc-ents-v3-2 2020-09-22 13:45:50 +02:00
Ines Montani
5e3b796b12 Validate section refs in debug config 2020-09-22 12:24:39 +02:00
svlandeg
085a1c8e2b add no_output_layer to TextCatBOW config 2020-09-22 12:06:40 +02:00
svlandeg
e1b8090b9b few more fixes 2020-09-22 12:01:06 +02:00
svlandeg
b556a10808 rename converts in_to_out 2020-09-22 11:50:19 +02:00
svlandeg
e931f4d757 add textcat score 2020-09-22 10:56:43 +02:00
svlandeg
396b33257f add entity_linker to jinja template 2020-09-22 10:40:05 +02:00
Ines Montani
db7126ead9 Increment version 2020-09-22 10:31:26 +02:00
svlandeg
135de82a2d add textcat to quickstart 2020-09-22 10:22:06 +02:00
Ines Montani
6316d5f398 Improve messages in project CLI [ci skip] 2020-09-22 09:45:34 +02:00
Ines Montani
49e80dbcac
Merge pull request #6103 from explosion/chore/tidy-up-tests-docs-get-doc 2020-09-22 09:45:04 +02:00
Ines Montani
81606b29bd
Merge pull request #6104 from svlandeg/fix/debug_model [ci skip] 2020-09-22 09:31:23 +02:00
Ines Montani
beb766d0a0 Add test 2020-09-22 09:15:57 +02:00
Ines Montani
285fa934d8 Merge branch 'chore/tidy-up-tests-docs-get-doc' of https://github.com/explosion/spaCy into chore/tidy-up-tests-docs-get-doc 2020-09-22 09:10:14 +02:00
Ines Montani
69f7e52c26 Update README.md 2020-09-22 09:10:06 +02:00
svlandeg
45b29c4a5b cleanup 2020-09-21 23:17:23 +02:00
svlandeg
fa5c416db6 initialize through nlp object and with train_corpus 2020-09-21 23:09:22 +02:00
Matthew Honnibal
3abc4a5adb Slightly tidy doc.ents.__set__ 2020-09-21 22:58:03 +02:00
Ines Montani
67fbcb3da5 Tidy up tests and docs 2020-09-21 20:43:54 +02:00
Ines Montani
a5f6ab4943
Merge pull request #6098 from adrianeboyd/feature/doc-init 2020-09-21 18:35:20 +02:00
Adriane Boyd
f212303729 Add sent_starts to Doc.__init__ 
Add sent_starts to `Doc.__init__`. Officially specify `is_sent_start`
values but also convert to and accept `sent_start` internally.
 2020-09-21 17:59:09 +02:00
svlandeg
447b3e5787 Merge remote-tracking branch 'upstream/develop' into fix/debug_model 
# Conflicts:
#	spacy/cli/debug_model.py
 2020-09-21 16:58:40 +02:00
Ines Montani
b3327c1e45 Increment version [ci skip] 2020-09-21 16:04:30 +02:00
Ines Montani
e8bcaa44f1 Don't auto-decompress archives with smart_open [ci skip] 2020-09-21 16:01:46 +02:00
Adriane Boyd
6aa91c7ca0 Make user_data keyword-only 2020-09-21 16:00:06 +02:00
Adriane Boyd
177df15d89 Implement Doc.set_ents 2020-09-21 15:54:05 +02:00
Adriane Boyd
13fbf6556a Merge remote-tracking branch 'upstream/develop' into feature/doc-ents-v3-2 2020-09-21 14:42:04 +02:00
svlandeg
eb9b447960 Merge remote-tracking branch 'upstream/develop' into fix/debug_model 
# Conflicts:
#	spacy/cli/debug_model.py
 2020-09-21 14:05:16 +02:00
Adriane Boyd
ce455f30ca Fix formatting 2020-09-21 13:53:29 +02:00
Adriane Boyd
bc02e86494 Extend Doc.__init__ with additional annotation 
Mostly copying from `spacy.tests.util.get_doc`, add additional kwargs to
`Doc.__init__` to initialize the most common doc/token values.
 2020-09-21 13:36:24 +02:00
Ines Montani
758ead8a47 Sync overrides with CLI overrides 2020-09-21 12:50:13 +02:00
Ines Montani
5497acf49a Support config overrides via environment variables 2020-09-21 11:25:10 +02:00
Ines Montani
1114219ae3 Tidy up and auto-format 2020-09-21 10:59:07 +02:00
Ines Montani
b2302c0a1c Improve error for missing dependency 2020-09-20 17:44:51 +02:00
Matthew Honnibal
8fb59d958c Format 2020-09-20 16:31:48 +02:00
Matthew Honnibal
dc22771f87 Fix sparse checkout 2020-09-20 16:30:05 +02:00
Matthew Honnibal
a0fb5e50db Use simple git clone call if not sparse 2020-09-20 16:22:04 +02:00
Matthew Honnibal
2c24d633d0 Use updated run_command 2020-09-20 16:21:43 +02:00
Matthew Honnibal
889128e5c5 Improve error handling in run_command 2020-09-20 16:20:57 +02:00
Ines Montani
554c9a2497 Update docs [ci skip] 2020-09-20 12:30:53 +02:00
svlandeg
6db1d5dc0d trying some stuff 2020-09-19 19:11:30 +02:00
Ines Montani
e863b3dc14
Merge pull request #6092 from adrianeboyd/bugfix/load-vocab-lookups-2 2020-09-19 12:33:38 +02:00
Sofie Van Landeghem
39872de1f6
Introducing the gpu_allocator (#6091) 
* rename 'use_pytorch_for_gpu_memory' to 'gpu_allocator'

* --code instead of --code-path

* update documentation

* avoid querying the "system" section directly

* add explanation of gpu_allocator to TF/PyTorch section in docs

* fix typo

* fix typo 2

* use set_gpu_allocator from thinc 8.0.0a34

* default null instead of empty string
 2020-09-19 01:17:02 +02:00
Adriane Boyd
47080fba98 Minor renaming / refactoring 
* Rename loader to `spacy.LookupsDataLoader.v1`, add debugging message
* Make `Vocab.lookups` a property
 2020-09-18 19:43:19 +02:00
svlandeg
73ff52b9ec hack for tok2vec listener 2020-09-18 16:43:15 +02:00
Adriane Boyd
eed4b785f5 Load vocab lookups tables at beginning of training 
Similar to how vectors are handled, move the vocab lookups to be loaded
at the start of training rather than when the vocab is initialized,
since the vocab doesn't have access to the full config when it's
created.

The option moves from `nlp.load_vocab_data` to `training.lookups`.

Typically these tables will come from `spacy-lookups-data`, but any
`Lookups` object can be provided.

The loading from `spacy-lookups-data` is now strict, so configs for each
language should specify the exact tables required. This also makes it
easier to control whether the larger clusters and probs tables are
included.

To load `lexeme_norm` from `spacy-lookups-data`:

```
[training.lookups]
@misc = "spacy.LoadLookupsData.v1"
lang = ${nlp.lang}
tables = ["lexeme_norm"]
```
 2020-09-18 15:59:16 +02:00
Ines Montani
a127fa475e
Merge pull request #6078 from svlandeg/fix/corpus 2020-09-18 14:44:21 +02:00
Matthew Honnibal
bbdb5f62b7
Temporary work-around for scoring a subset of components (#6090) 
* Try hacking the scorer to work around sentence boundaries

* Upd scorer

* Set dev version

* Upd scorer hack

* Fix version

* Improve comment on hack
 2020-09-18 14:26:42 +02:00
Adriane Boyd
a88106e852
Remove W106: HEAD and SENT_START in doc.from_array (#6086) 
* Remove W106: HEAD and SENT_START in doc.from_array

This warning was hacky and being triggered too often.

* Fix test
 2020-09-18 03:01:29 +02:00
svlandeg
e4fc7e0222 fixing output sample to proper 2D array 2020-09-17 22:34:36 +02:00
Adriane Boyd
8b650f3a78 Modify setting missing and blocked entity tokens 
In order to make it easier to construct `Doc` objects as training data,
modify how missing and blocked entity tokens are set to prioritize
setting `O` and missing entity tokens for training purposes over setting
blocked entity tokens.

* `Doc.ents` setter sets tokens outside entity spans to `O` regardless
of the current state of each token

* For `Doc.ents`, setting a span with a missing label sets the `ent_iob`
to missing instead of blocked

* `Doc.block_ents(spans)` marks spans as hard `O` for use with the
`EntityRecognizer`
 2020-09-17 21:27:42 +02:00
Ines Montani
3865214343 Use consistent shortcut 2020-09-17 16:57:02 +02:00
svlandeg
35a3931064 fix typo 2020-09-17 16:36:27 +02:00
svlandeg
ddfc1fc146 add pretraining option to init config 2020-09-17 16:05:40 +02:00
svlandeg
427dbecdd6 cleanup and formatting 2020-09-17 11:48:04 +02:00
svlandeg
0c35885751 generalize corpora, dot notation for dev and train corpus 2020-09-17 11:38:59 +02:00
svlandeg
781fae678b Merge remote-tracking branch 'upstream/develop' into fix/corpus 2020-09-17 09:24:36 +02:00
Matthew Honnibal
8303d101a5 Set version to v3.0.0a19 2020-09-17 00:18:49 +02:00
Adriane Boyd
7e4cd7575c
Refactor Docs.is_ flags (#6044) 
* Refactor Docs.is_ flags

* Add derived `Doc.has_annotation` method

  * `Doc.has_annotation(attr)` returns `True` for partial annotation

  * `Doc.has_annotation(attr, require_complete=True)` returns `True` for
    complete annotation

* Add deprecation warnings to `is_tagged`, `is_parsed`, `is_sentenced`
and `is_nered`

* Add `Doc._get_array_attrs()`, which returns a full list of `Doc` attrs
for use with `Doc.to_array`, `Doc.to_bytes` and `Doc.from_docs`. The
list is the `DocBin` attributes list plus `SPACY` and `LENGTH`.

Notes on `Doc.has_annotation`:

* `HEAD` is converted to `DEP` because heads don't have an unset state

* Accept `IS_SENT_START` as a synonym of `SENT_START`

Additional changes:

* Add `NORM`, `ENT_ID` and `SENT_START` to default attributes for
`DocBin`

* In `Doc.from_array()` the presence of `DEP` causes `HEAD` to override
`SENT_START`

* In `Doc.from_array()` using `attrs` other than
`Doc._get_array_attrs()` (i.e., a user's custom list rather than our
default internal list) with both `HEAD` and `SENT_START` shows a warning
that `HEAD` will override `SENT_START`

* `set_children_from_heads` does not require dependency labels to set
sentence boundaries and sets `sent_start` for all non-sentence starts to
`-1`

* Fix call to set_children_form_heads

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-09-17 00:14:01 +02:00