explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-11-11 04:08:09 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,454 Commits 68 Branches 175 Tags 279 MiB
b0463fbf75
Commit Graph

1247 Commits

Author SHA1 Message Date
svlandeg
0c35885751 generalize corpora, dot notation for dev and train corpus 2020-09-17 11:38:59 +02:00
svlandeg
8cedb2f380 Merge branch 'fix/corpus' of https://github.com/svlandeg/spaCy into fix/corpus 2020-09-17 09:27:55 +02:00
svlandeg
781fae678b Merge remote-tracking branch 'upstream/develop' into fix/corpus 2020-09-17 09:24:36 +02:00
Sofie Van Landeghem
21dcf92964
Update website/docs/api/data-formats.md 
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-09-17 09:21:36 +02:00
Adriane Boyd
7e4cd7575c
Refactor Docs.is_ flags (#6044) 
* Refactor Docs.is_ flags

* Add derived `Doc.has_annotation` method

  * `Doc.has_annotation(attr)` returns `True` for partial annotation

  * `Doc.has_annotation(attr, require_complete=True)` returns `True` for
    complete annotation

* Add deprecation warnings to `is_tagged`, `is_parsed`, `is_sentenced`
and `is_nered`

* Add `Doc._get_array_attrs()`, which returns a full list of `Doc` attrs
for use with `Doc.to_array`, `Doc.to_bytes` and `Doc.from_docs`. The
list is the `DocBin` attributes list plus `SPACY` and `LENGTH`.

Notes on `Doc.has_annotation`:

* `HEAD` is converted to `DEP` because heads don't have an unset state

* Accept `IS_SENT_START` as a synonym of `SENT_START`

Additional changes:

* Add `NORM`, `ENT_ID` and `SENT_START` to default attributes for
`DocBin`

* In `Doc.from_array()` the presence of `DEP` causes `HEAD` to override
`SENT_START`

* In `Doc.from_array()` using `attrs` other than
`Doc._get_array_attrs()` (i.e., a user's custom list rather than our
default internal list) with both `HEAD` and `SENT_START` shows a warning
that `HEAD` will override `SENT_START`

* `set_children_from_heads` does not require dependency labels to set
sentence boundaries and sets `sent_start` for all non-sentence starts to
`-1`

* Fix call to set_children_form_heads

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-09-17 00:14:01 +02:00
svlandeg
55f8d5478e fix example output 2020-09-15 22:09:30 +02:00
svlandeg
51fa929f47 rewrite train_corpus to corpus.train in config 2020-09-15 21:58:04 +02:00
Ines Montani
2214d1bb7b
Merge pull request #6067 from explosion/feature/spacy-blank-from-config 2020-09-15 14:18:33 +02:00
Ines Montani
b7faa38960 Update docs [ci skip] 2020-09-15 12:44:03 +02:00
Ines Montani
0edd695bf6 Update docs 2020-09-15 11:41:49 +02:00
Ines Montani
99549a5ace Fix consistency and update docs 2020-09-15 11:37:37 +02:00
Ines Montani
154752f9c2 Update docs and consistency [ci skip] 2020-09-15 00:32:49 +02:00
Sofie Van Landeghem
3216a33149
positive_label config for textcat (#6062) 
* hook up positive_label in textcat

* unit tests

* documentation

* formatting

* tests

* fix typo

* move verify_config to after begin_training

* revert accidential commit
 2020-09-14 17:08:00 +02:00
Ines Montani
b854e0bef9 Update styleguide [ci skip] 2020-09-14 11:25:57 +02:00
Ines Montani
9afb1d9965
Merge pull request #6063 from svlandeg/feature/doc_cleanup [ci skip] 2020-09-14 10:35:43 +02:00
Ines Montani
85e5910102 Update docs [ci skip] 2020-09-13 23:09:19 +02:00
Ines Montani
5ebb2a2ac8 Update docs [ci skip] 2020-09-13 22:36:20 +02:00
Ines Montani
47acb45850 Update docs [ci skip] 2020-09-13 22:30:33 +02:00
Ines Montani
2e3d067a7b Update docs [ci skip] 2020-09-13 19:29:06 +02:00
Ines Montani
99b26fe492 Update docs [ci skip] 2020-09-13 17:59:38 +02:00
Sofie Van Landeghem
744df9814a
define threshold for scoring textcat in TextCat config (#6055) 
* define threshold for scoring textcat in TextCat config

* fix unit test and documentation
 2020-09-13 14:15:52 +02:00
Ines Montani
1316071086 Update docs [ci skip] 2020-09-13 11:31:50 +02:00
Ines Montani
368ecf705a Update docs [ci skip] 2020-09-12 17:40:50 +02:00
svlandeg
c4f324d5f1 doc fixes 2020-09-12 17:38:54 +02:00
Ines Montani
8b0dabe987 Update docs [ci skip] 2020-09-12 17:05:10 +02:00
Ines Montani
0b2e07215d Support overwriting name on spacy package 2020-09-11 11:38:28 +02:00
Ines Montani
4fec8c39a3 Update project teaser [ci skip] 2020-09-10 13:23:03 +02:00
Ines Montani
9f08ea80b4
Merge pull request #6047 from svlandeg/feature/doc-fixes 
Fix branch for spacy clone + UX
 2020-09-10 13:05:41 +02:00
Ines Montani
763e302dcc Update project widgets and examples [ci skip] 2020-09-10 13:04:16 +02:00
svlandeg
97d99f7efa Merge remote-tracking branch 'upstream/develop' into feature/doc-fixes 2020-09-10 11:51:34 +02:00
Ines Montani
908f3a4494 Update default projects repo [ci skip] 2020-09-10 11:42:14 +02:00
Ines Montani
15bc3a37b4 Add --branch to project clone 2020-09-10 11:08:15 +02:00
Ines Montani
b7afd09d27 Update formatting [ci skip] 2020-09-10 11:07:09 +02:00
svlandeg
9073d99fc9 fix link to shape inference section 2020-09-10 10:22:59 +02:00
Ines Montani
1955aaaa20
Merge pull request #6045 from svlandeg/feature/more-layers-docs [ci skip] 2020-09-09 21:46:40 +02:00
Ines Montani
2e567a47c2 Update docs and formatting 2020-09-09 21:26:10 +02:00
svlandeg
aa27e3f1f2 PyTorch spelling 2020-09-09 16:27:21 +02:00
svlandeg
c89e07927e document individual component API pages 2020-09-09 16:18:38 +02:00
Sofie Van Landeghem
cb66ea7400
Remove simple_ner code (#6041) 
* remove simple_ner code

* remove unused _biluo and _iob files
 2020-09-09 16:11:27 +02:00
svlandeg
a8aa9a8068 document Pipe API details, crossreferences etc 2020-09-09 15:56:27 +02:00
svlandeg
9a7c6cc61a references to usage page on layers and architectures 2020-09-09 14:47:32 +02:00
svlandeg
e80898092b Merge branch 'feature/more-layers-docs' of https://github.com/svlandeg/spaCy into feature/more-layers-docs 2020-09-09 14:44:28 +02:00
svlandeg
4c080b3a98 details on Thinc shape inference 2020-09-09 13:57:05 +02:00
svlandeg
39aa740777 Merge remote-tracking branch 'upstream/develop' into feature/more-layers-docs 2020-09-09 11:59:34 +02:00
svlandeg
e39242c4e6 formatting 2020-09-09 11:25:35 +02:00
Ines Montani
24053d83ec Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-09-09 11:20:14 +02:00
Ines Montani
406aed78ee Update docs [ci skip] 2020-09-09 11:20:07 +02:00
Sofie Van Landeghem
8e7557656f
Renaming gold & annotation_setter (#6042) 
* version bump to 3.0.0a16

* rename "gold" folder to "training"

* rename 'annotation_setter' to 'set_extra_annotations'

* formatting
 2020-09-09 10:31:03 +02:00
Marek Grzenkowicz
a26f864ed3
Clarify how to choose pretrained weights files (closes #6027) [ci skip] (#6039) 2020-09-08 21:13:50 +02:00
svlandeg
a16afb79e3 add section on Thinc implementation details 2020-09-08 20:43:09 +02:00
svlandeg
1c476b4b41 how to register and use custom function 2020-09-08 20:22:20 +02:00
svlandeg
b35a26ea5d example wrapped Torch model and chaining with Thinc 2020-09-08 18:32:58 +02:00
svlandeg
bd8f9b188b small fixes 2020-09-08 17:24:36 +02:00
Ines Montani
d98ae9d918 Update docs [ci skip] 2020-09-08 10:33:48 +02:00
Ines Montani
c443c82722 Update docs [ci skip] 2020-09-05 13:41:10 +02:00
Ines Montani
b3e338d65e Update docs [ci skip] 2020-09-04 20:58:36 +02:00
Ines Montani
157caf4dfa WIP: update docs [ci skip] 2020-09-04 16:30:31 +02:00
Ines Montani
f174c7b1f3 Merge branch 'develop' into pr/6018 2020-09-04 15:54:49 +02:00
Ines Montani
864a697e63 Merge branch 'develop' into master-tmp 2020-09-04 13:15:36 +02:00
Adriane Boyd
b927893309
Merge branch 'develop' into feature/dependency-matcher-v3 2020-09-04 13:03:30 +02:00
Ines Montani
2189046869
Merge pull request #6024 from explosion/chore/registry-renaming 2020-09-04 10:54:10 +02:00
Ines Montani
4daf138136 Fix alphabetic ordering [ci skip] 2020-09-03 23:01:50 +02:00
Ines Montani
b1eb98b15c Remove todos [ci skip] 2020-09-03 17:43:58 +02:00
Ines Montani
23b7d9cfa3 Prefix span getters 2020-09-03 17:37:06 +02:00
Ines Montani
5afe6447cd registry.assets -> registry.misc 2020-09-03 17:31:14 +02:00
Ines Montani
c063e55eb7 Add prefix to batchers 2020-09-03 17:30:41 +02:00
Ines Montani
804f120361 Don't use registered function version in title 2020-09-03 17:29:47 +02:00
Ines Montani
c53b1433b9 Adjust more arguments [ci skip] 2020-09-03 17:12:24 +02:00
Ines Montani
121809dd1e Fix anchor [ci skip] 2020-09-03 16:49:56 +02:00
Ines Montani
25a595dc10 Fix typos and wording [ci skip] 2020-09-03 16:37:45 +02:00
Ines Montani
b5a0657fd6 "model" terminology consistency in docs 2020-09-03 13:13:03 +02:00
Ines Montani
b02ad8045b Update docs [ci skip] 2020-09-03 10:10:13 +02:00
Ines Montani
1815c613c9 Update docs [ci skip] 2020-09-03 10:07:45 +02:00
Adriane Boyd
960d9cfadc Officially support DependencyMatcher 
Add official support for the `DependencyMatcher`. Redesign the pattern
specification. Fix and extend operator implementations. Update API docs
and add usage docs.

Patterns
--------

Refactor pattern structure to:

```
{
  "LEFT_ID": str,
  "REL_OP": str,
  "RIGHT_ID": str,
  "RIGHT_ATTRS": dict,
}
```

The first node contains only `RIGHT_ID` and `RIGHT_ATTRS` and all
subsequent nodes contain all four keys.

New operators
-------------

Because of the way patterns are constructed from left to right, it's
helpful to have `follows` operators along with `precedes` operators. Add
operators for simple precedes / follows alongside immediate precedes /
follows.

* `.*`: precedes
* `;`: immediately follows
* `;*`: follows

Operator fixes
--------------

* `<` and `<<` do not include the node itself
* Fix reversed order for all operators involving linear precedence (`.`,
  all sibling operators)
* Linear precedence operators do not match nodes outside the same parse

Additional fixes
----------------

* Use v3 Matcher API
* Support `get` and `remove`
* Support pickling
 2020-09-02 17:45:29 +02:00
svlandeg
19298de352 small fix 2020-09-02 17:43:11 +02:00
svlandeg
bbaea530f6 sublayers paragraph 2020-09-02 17:36:22 +02:00
svlandeg
1be7ff02a6 swapping section 2020-09-02 15:26:07 +02:00
svlandeg
57e432ba2a editor tip as Accordion instead of Infobox 2020-09-02 14:26:57 +02:00
svlandeg
d19ec6c67b small rewrites in types paragraph 2020-09-02 14:25:18 +02:00
svlandeg
821b2d4e63 update examples 2020-09-02 14:15:50 +02:00
svlandeg
e29a33449d rewrite intro, simpel Model example 2020-09-02 13:41:18 +02:00
svlandeg
422df9c2e2 Merge remote-tracking branch 'upstream/develop' into feature/docs-layers 
# Conflicts:
#	website/docs/usage/layers-architectures.md
 2020-09-02 13:17:11 +02:00
Ines Montani
70238543c8 Update layers/arch docs structure [ci skip] 2020-09-02 13:04:35 +02:00
svlandeg
6fd7f140ec custom-architectures section 2020-09-02 11:14:06 +02:00
svlandeg
3d9ae9286f small fixes 2020-09-02 10:46:38 +02:00
Ines Montani
690bd77669 Add todos [ci skip] 2020-09-01 14:04:36 +02:00
Ines Montani
70b226f69d Support ignore marker in project document [ci skip] 2020-09-01 12:49:04 +02:00
Ines Montani
9af82f3f11
Merge pull request #6003 from explosion/feature/matcher-as-spans 2020-08-31 17:50:56 +02:00
Sofie Van Landeghem
3ac620f09d
fix config example [ci skip] 2020-08-31 17:40:04 +02:00
Ines Montani
3929431af1 Update docs [ci skip] 2020-08-31 17:06:33 +02:00
Ines Montani
add9de5487 Deprecate (Phrase)Matcher.pipe 2020-08-31 17:01:24 +02:00
svlandeg
2c3b64a567 console logging example 2020-08-31 16:56:13 +02:00
Ines Montani
bca6bf8dda Update docs [ci skip] 2020-08-31 16:39:53 +02:00
Ines Montani
db9f8896f5 Add docs [ci skip] 2020-08-31 16:10:41 +02:00
svlandeg
fe6c08218e fixes 2020-08-31 14:51:49 +02:00
svlandeg
0e0abb0378 fix 2020-08-31 14:50:29 +02:00
svlandeg
56ba691ecd small fixes 2020-08-31 14:46:00 +02:00
svlandeg
e47ea88aeb revert annotations refactor 2020-08-31 14:40:55 +02:00
svlandeg
13ee742fb4 example of custom logger 2020-08-31 14:24:41 +02:00
svlandeg
2c90a06fee some more information about the loggers 2020-08-31 13:43:17 +02:00
svlandeg
c18eb63483 Merge remote-tracking branch 'upstream/develop' into feature/vectors-docs 
# Conflicts:
#	website/docs/usage/embeddings-transformers.md
 2020-08-31 13:21:36 +02:00
Juan Gutiérrez
9002bea29f
Update suffixes example (#5989) 
* Update suffixes example

The current example will throw `TypeError: can only concatenate list (not "tuple") to list`

* Signing Contributor Agreement
 2020-08-31 12:44:56 +02:00
Sofie Van Landeghem
ec14744ee4
Rename Transformer listener (#6001) 
* rename to spacy-transformers.TransformerListener

* add some more tok2vec tests

* use select_pipes

* fix docs - annotation setter was not changed in the end
 2020-08-31 12:41:39 +02:00
Adriane Boyd
216efaf5f5 Restrict tokenizer exceptions to ORTH and NORM 2020-08-31 09:55:01 +02:00
Ines Montani
9b86312bab Update docs [ci skip] 2020-08-29 18:43:19 +02:00
Adriane Boyd
870774f475
Merge branch 'develop' into docs/morph-usage-v3 2020-08-29 16:00:50 +02:00
Ines Montani
45f46a5c85
Merge pull request #5993 from explosion/feature/disabled-components 2020-08-29 15:58:41 +02:00
Adriane Boyd
f9ed31a757 Update usage docs for lemmatization and morphology 2020-08-29 15:56:50 +02:00
Ines Montani
bc0730be3f Update docs [ci skip] 2020-08-29 12:53:14 +02:00
Ines Montani
450bf806b0
Merge pull request #5991 from adrianeboyd/docs/sent-usage-v3 
Update sentence segmentation usage docs
 2020-08-29 12:40:06 +02:00
Ines Montani
66d76f5126 Update docs 2020-08-29 12:36:05 +02:00
svlandeg
9f00a20ce4 proofreading and custom examples 2020-08-28 21:50:42 +02:00
svlandeg
5230529de2 add loggers registry & logger docs sections 2020-08-28 21:44:04 +02:00
Adriane Boyd
48df50533d Update sentence segmentation usage docs 
Update sentence segmentation usage docs to incorporate `senter`.
 2020-08-28 10:58:16 +02:00
svlandeg
72a87095d9 add loggers registry 2020-08-27 20:26:28 +02:00
svlandeg
aa9e0c9c39 small fix 2020-08-27 19:56:52 +02:00
svlandeg
8cde6ccb7d Merge remote-tracking branch 'upstream/develop' into feature/vectors-docs 2020-08-27 19:56:09 +02:00
svlandeg
556e975a30 various fixes 2020-08-27 19:24:44 +02:00
Ines Montani
ff4175e839 Add more info to debug config 2020-08-27 18:17:58 +02:00
svlandeg
329e490560 small import fixes 2020-08-27 14:50:43 +02:00
svlandeg
28e4ba7270 fix references to TransformerListener 2020-08-27 14:33:28 +02:00
svlandeg
4d37ac3f33 configure_custom_sent_spans example 2020-08-27 14:14:16 +02:00
svlandeg
c68169f83f fix link 2020-08-27 10:19:43 +02:00
svlandeg
acc794c975 example of writing to other custom attribute 2020-08-27 10:10:10 +02:00
svlandeg
559b65f2e0 adjust references to null_annotation_setter to trfdata_setter 2020-08-27 09:43:32 +02:00
Ines Montani
696f167478 Add diff example to docs [ci skip] 2020-08-26 15:57:54 +02:00
Adriane Boyd
90d88729e0
Add AttributeRuler.score (#5963) 
* Add AttributeRuler.score

Add scoring for TAG / POS / MORPH / LEMMA if these are present in the
assigned token attributes.

Add default score weights (that don't really make a lot of sense) so
that the scores are in the default config in some form.

* Update docs
 2020-08-26 15:39:30 +02:00
svlandeg
ec069627fe rename to TransformerListener 2020-08-26 13:31:01 +02:00
Ines Montani
627617a079 Tidy up and add docs [ci skip] 2020-08-26 13:24:55 +02:00
svlandeg
15902c5aa2 fix link 2020-08-26 11:51:57 +02:00
svlandeg
feb86d5206 clarify default 2020-08-26 11:21:30 +02:00
Ines Montani
f31c4462ca Update docs [ci skip] 2020-08-25 13:27:59 +02:00
Ines Montani
8ac5ef1284 Update docs 2020-08-25 11:54:37 +02:00
Matthew Honnibal
8038b87f04
Various small tweaks to project CLI (#5965) 
* Fix up/download of http and local paths

* Support git_sparse_checkout for assets

* Fix scorer

* Handle already-present directories for git assets

* Improve convert command

* Fix support for existant files in git assets

* Support branches in git sparse checkout

* Format

* Fix git assets

* Document git block in assets

* Fix test

* Fix test

* Revert "Fix test"

This reverts commit cf3097260f.

* Revert "Fix test"

This reverts commit 964d636e27.

* Dont multiply p/r/f by 100

* Display scores * 100 during training
 2020-08-25 00:30:52 +02:00
Ines Montani
26405710e0 Add icon credit [ci skip] 2020-08-24 10:28:15 +02:00
Matthew Honnibal
e559867605
Allow spacy project to push and pull to/from remote storage (#5949) 
* Add utils for working with remote storage

* WIP add remote_cache for project

* WIP add push and pull commands

* Use pathy in remote_cache

* Updarte util

* Update remote_cache

* Update util

* Update project assets

* Update pull script

* Update push script

* Fix type annotation in util

* Work on remote storage

* Remove site and env hash

* Fix imports

* Fix type annotation

* Require pathy

* Require pathy

* Fix import

* Add a util to handle project variable substitution

* Import push and pull commands

* Fix pull command

* Fix push command

* Fix tarfile in remote_storage

* Improve printing

* Fiddle with status messages

* Set version to v3.0.0a9

* Draft docs for spacy project remote storages

* Update docs [ci skip]

* Use Thinc config to simplify and unify template variables

* Auto-format

* Don't import Pathy globally for now

Causes slow and annoying Google Cloud warning

* Tidy up test

* Tidy up and update tests

* Update to latest Thinc

* Update docs

* variables -> vars

* Update docs [ci skip]

* Update docs [ci skip]

Co-authored-by: Ines Montani <ines@ines.io>
 2020-08-23 18:32:09 +02:00
Ines Montani
f27aecac14 Update formatting [ci skip] 2020-08-23 11:57:56 +02:00
Ines Montani
98a9e063b6 Update docs [ci skip] 2020-08-22 17:15:05 +02:00
Matthew Honnibal
8dfc4cbfe7 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-22 17:12:09 +02:00
Matthew Honnibal
048de64d4c Suggest edits 2020-08-22 17:11:28 +02:00
Ines Montani
adcf790b96 Update docs[ci skip] 2020-08-22 17:04:16 +02:00
Ines Montani
37ebff6997 Update docs [ci skip] 2020-08-22 16:47:03 +02:00
Matthew Honnibal
8685229891 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-22 16:06:59 +02:00
Matthew Honnibal
d97695d09d Update embeddings-transformers.md 2020-08-22 15:41:35 +02:00
Ines Montani
c7c9b0451f Update docs [ci skip] 2020-08-22 13:52:52 +02:00
Ines Montani
71aeae89c5
Merge pull request #5948 from svlandeg/feature/docs-docs-docs [ci skip] 2020-08-22 12:18:47 +02:00
Ines Montani
27f81109d6 Update docs [ci skip] 2020-08-21 20:02:18 +02:00
Ines Montani
f102164a1f Update docs [ci skip] 2020-08-21 19:34:06 +02:00
svlandeg
1b7cfa7347 Merge remote-tracking branch 'upstream/develop' into feature/docs-docs-docs 2020-08-21 18:36:18 +02:00
svlandeg
942adf0f4d comma 2020-08-21 18:36:02 +02:00