Commit Graph

1140 Commits

Author SHA1 Message Date
Ines Montani
2cc4640385 Update docs [ci skip] 2020-08-21 16:21:55 +02:00
Ines Montani
74cb6d39d0 Update docs [ci skip] 2020-08-21 16:11:38 +02:00
Matthew Honnibal
f5bcc10268 Update architectures 2020-08-21 15:34:54 +02:00
Matthew Honnibal
7ed8f4504b Update API docs for architectures 2020-08-21 15:22:19 +02:00
Ines Montani
aa6a7cd6e7 Update docs and consistency [ci skip] 2020-08-21 13:49:18 +02:00
Ines Montani
52bd3a8b48 Update docs [ci skip] 2020-08-21 13:22:59 +02:00
Ines Montani
e60442d83a Adjust label casing in displaCy NER visualizer (resolves #4866)
- Accept any case for label names in ents and colors option, even if actual predicted label uses different casing
- Don't text-transform: uppercase visually, if it's important to users that the label is represented as-is in the UI
2020-08-21 11:51:31 +02:00
Ines Montani
04e4d59235 Update docs [ci skip] 2020-08-20 16:17:25 +02:00
Ines Montani
7f2e4244df
Merge pull request #5941 from svlandeg/feature/update-more-docs 2020-08-20 11:21:24 +02:00
Ines Montani
6ad59d59fe Merge branch 'develop' of https://github.com/explosion/spaCy into develop [ci skip] 2020-08-20 11:20:58 +02:00
Ines Montani
fb51b55eb9 Add comment [ci skip] 2020-08-20 11:20:43 +02:00
Sofie Van Landeghem
410b54e10e
Update website/docs/api/data-formats.md
Co-authored-by: Ines Montani <ines@ines.io>
2020-08-20 11:15:34 +02:00
svlandeg
ae719b354f fix typos 2020-08-20 10:20:40 +02:00
svlandeg
f728c00cbb Merge remote-tracking branch 'upstream/develop' into feature/update-more-docs
# Conflicts:
#	website/docs/api/data-formats.md
2020-08-20 10:02:13 +02:00
svlandeg
229033831a add explanation of raw_text 2020-08-20 10:00:45 +02:00
Ines Montani
2253d26b82 Update vectors and similarity docs [ci skip] 2020-08-19 21:18:26 +02:00
Ines Montani
ea6640ea72
Merge pull request #5939 from explosion/feature/thinc-v8.0.0a28
Update Thinc and config variables
2020-08-19 21:14:36 +02:00
Ines Montani
15e6feed01 Update docs [ci skip] 2020-08-19 20:37:54 +02:00
svlandeg
09f3cfc985 add version 2020-08-19 19:58:45 +02:00
svlandeg
7d9f00bdbf waltzing schedule 2020-08-19 19:53:00 +02:00
Ines Montani
3dd390b1a1 Update Thinc and config variables 2020-08-19 19:46:12 +02:00
svlandeg
85b39639e1 small fix 2020-08-19 19:17:36 +02:00
svlandeg
d8f6abdc23 add linking TODO back in 2020-08-19 18:00:35 +02:00
svlandeg
169b5bcda0 Merge remote-tracking branch 'upstream/develop' into feature/update-docs
# Conflicts:
#	website/docs/usage/training.md
2020-08-19 17:58:25 +02:00
svlandeg
7119295a8a badgers intro 2020-08-19 17:53:22 +02:00
svlandeg
4906a2ae6c custom functions intro 2020-08-19 17:32:35 +02:00
svlandeg
7a2e6a96f5 fix typo 2020-08-19 16:54:16 +02:00
svlandeg
648499157a rename "custom models" to "custom functions" 2020-08-19 16:53:51 +02:00
Ines Montani
63921161c8 Update docs [ci skip] 2020-08-19 16:04:21 +02:00
svlandeg
d3a8321172 fix typos 2020-08-19 15:12:12 +02:00
svlandeg
60fedb8518 fix 2 more API lines 2020-08-19 14:55:32 +02:00
svlandeg
2dfd919585 add kb_loader and get_candidates back to EL API 2020-08-19 14:52:49 +02:00
Ines Montani
225f8866a1 Fix consistency 2020-08-19 12:47:57 +02:00
Ines Montani
9c25656ccc Update docs [ci skip] 2020-08-19 12:14:41 +02:00
Ines Montani
2285e59765
Merge pull request #5933 from svlandeg/feature/more-v3-docs [ci skip] 2020-08-19 11:29:02 +02:00
Ines Montani
13291e97ba Update docs [ci skip] 2020-08-19 00:28:37 +02:00
svlandeg
6ed67d495a format 2020-08-18 19:43:20 +02:00
svlandeg
f9fe5eb323 clean up example 2020-08-18 19:35:23 +02:00
svlandeg
a8acedd4ba example of custom reader and batcher 2020-08-18 19:15:16 +02:00
svlandeg
0d55b6ebb4 formatting 2020-08-18 18:55:56 +02:00
svlandeg
abba639565 Merge remote-tracking branch 'upstream/develop' into feature/more-v3-docs 2020-08-18 18:55:12 +02:00
Sofie Van Landeghem
358cbb21e3
Define candidate generator in EL config (#5876)
* candidate generator as separate part of EL config

* update comment

* ent instead of str as input for candidate generation

* Span instead of str: correct type indication

* fix types

* unit test to create new candidate generator

* fix replace_pipe argument passing

* move error message, general cleanup

* add vocab back to KB constructor

* provide KB as callable from Vocab arg

* rename to kb_loader, fix KB serialization as part of the EL pipe

* fix typo

* reformatting

* cleanup

* fix comment

* fix wrongly duplicated code from merge conflict

* rename dump to to_disk

* from_disk instead of load_bulk

* update test after recent removal of set_morphology in tagger

* remove old doc
2020-08-18 16:10:36 +02:00
Ines Montani
82f0e20318 Update docs and consistency [ci skip] 2020-08-18 14:39:40 +02:00
Matthew Honnibal
b72bd1767f Remove todo 2020-08-18 13:52:22 +02:00
Matthew Honnibal
574fd53289 Add precision/recall description 2020-08-18 13:51:08 +02:00
Matthew Honnibal
96a9c65f97 Add model architectures intro 2020-08-18 13:50:55 +02:00
svlandeg
705e1cb06c typo in link 2020-08-18 12:04:05 +02:00
svlandeg
f7b76d2d83 Merge remote-tracking branch 'upstream/develop' into feature/more-v3-docs 2020-08-18 11:57:52 +02:00
svlandeg
8dcda351ec typo's and quick note on default values 2020-08-18 10:23:27 +02:00
Ines Montani
ef6cf3b276 Update docs [ci skip] 2020-08-18 01:29:34 +02:00
Ines Montani
1c3bcfb488 Update docs and util consistency 2020-08-18 01:22:59 +02:00
Ines Montani
728fec0194 Update docs [ci skip] 2020-08-18 00:49:19 +02:00
Ines Montani
9299166c75
Merge pull request #5925 from explosion/docs/vectors [ci skip]
Update the 'vectors' docs page
2020-08-17 21:45:09 +02:00
Ines Montani
990c6b4c32 Update docs and CLI [ci skip] 2020-08-17 21:38:20 +02:00
svlandeg
4fe4bab1c9 typo fixes 2020-08-17 17:10:15 +02:00
svlandeg
da80c18660 merge develop into branch 2020-08-17 16:57:18 +02:00
Ines Montani
3ae5e02f4f Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
Matthew Honnibal
052d82aa4e Suggest vectors changes 2020-08-17 15:32:30 +02:00
svlandeg
961e818be6 p/r definitions 2020-08-17 15:02:39 +02:00
svlandeg
319692aa53 fix typos 2020-08-17 14:05:48 +02:00
Matthew Honnibal
61dfdd9fbd Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-16 20:30:01 +02:00
Matthew Honnibal
be07567ac6 Update transformers page 2020-08-16 20:29:50 +02:00
Matthew Honnibal
8e5f99ee25 Update transformer docs intro. Also write system requirements 2020-08-16 20:13:24 +02:00
Ines Montani
2ac4b0ef3e Finish Transformer docs [ci skip] 2020-08-16 15:56:32 +02:00
Ines Montani
6ae83bde0c Fix CLI consistency [ci skip] 2020-08-16 15:46:29 +02:00
Ines Montani
a570c304df Update quickstart, template and docs 2020-08-15 14:50:29 +02:00
Ines Montani
950832f087
Tidy up pipes (#5906)
* Tidy up pipes

* Fix init, defaults and raise custom errors

* Update docs

* Update docs [ci skip]

* Apply suggestions from code review

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>

* Tidy up error handling and validation, fix consistency

* Simplify get_examples check

* Remove unused import [ci skip]

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-08-11 23:29:31 +02:00
Ines Montani
b7ec06e331 Update docs [ci skip] 2020-08-11 20:57:23 +02:00
Ines Montani
10f42e3a39 Update docs [ci skip] 2020-08-11 00:09:49 +02:00
Ines Montani
2778d04377 Update docs [ci skip] 2020-08-10 23:41:09 +02:00
Ines Montani
adf2b1c8a9 Update graphic [ci skip] 2020-08-10 17:20:04 +02:00
Ines Montani
023ba7ae26 Update docs 2020-08-10 17:13:11 +02:00
Ines Montani
c099f6eece Add Token.lex 2020-08-10 16:43:52 +02:00
Ines Montani
64f2f84098 Update docstrings and docs [ci skip] 2020-08-10 13:45:22 +02:00
Ines Montani
12052bd8f6 Update docs [ci skip] 2020-08-10 01:20:10 +02:00
Ines Montani
0832cdd443 Fix formatting [ci skip] 2020-08-10 00:46:32 +02:00
Ines Montani
d611cbef43 Update docs [ci skip] 2020-08-10 00:42:26 +02:00
Ines Montani
c044460823 Update docs [ci skip] 2020-08-10 00:01:38 +02:00
Ines Montani
05dcab10aa Fix typo 2020-08-09 22:34:03 +02:00
Ines Montani
d5c78c7a34 Update docs and fix consistency 2020-08-09 22:31:52 +02:00
Ines Montani
a15c5fb191 Update docstrings and docs 2020-08-09 16:10:48 +02:00
Ines Montani
8d2baa153d Update tokenizer docs and add test 2020-08-09 15:24:01 +02:00
Ines Montani
46bc513a4e Update docs [ci skip] 2020-08-07 20:14:31 +02:00
Ines Montani
fe29ceec9e Merge branch 'develop' into docs/model-docstrings 2020-08-07 18:42:01 +02:00
Ines Montani
470b6f8073 Update docs 2020-08-07 18:41:15 +02:00
Ines Montani
3901b088ff Update graphics and 101 [ci skip] 2020-08-07 17:14:13 +02:00
Ines Montani
5e1421e5a6 Update docs [ci skip] 2020-08-07 16:23:12 +02:00
Ines Montani
b7e34c1451 Update docs [ci skip] 2020-08-07 16:13:13 +02:00
Ines Montani
6f3649923c
Merge pull request #5893 from explosion/feature/validate-arg 2020-08-07 15:47:20 +02:00
Ines Montani
e829d3bf14 Update docs [ci skip] 2020-08-07 15:46:20 +02:00
Adriane Boyd
e962784531
Add Lemmatizer and simplify related components (#5848)
* Add Lemmatizer and simplify related components

* Add `Lemmatizer` pipe with `lookup` and `rule` modes using the
`Lookups` tables.
* Reduce `Tagger` to a simple tagger that sets `Token.tag` (no pos or lemma)
* Reduce `Morphology` to only keep track of morph tags (no tag map, lemmatizer,
or morph rules)
* Remove lemmatizer from `Vocab`
* Adjust many many tests

Differences:

* No default lookup lemmas
* No special treatment of TAG in `from_array` and similar required
* Easier to modify labels in a `Tagger`
* No extra strings added from morphology / tag map

* Fix test

* Initial fix for Lemmatizer config/serialization

* Adjust init test to be more generic

* Adjust init test to force empty Lookups

* Add simple cache to rule-based lemmatizer

* Convert language-specific lemmatizers

Convert language-specific lemmatizers to component lemmatizers. Remove
previous lemmatizer class.

* Fix French and Polish lemmatizers

* Remove outdated UPOS conversions

* Update Russian lemmatizer init in tests

* Add minimal init/run tests for custom lemmatizers

* Add option to overwrite existing lemmas

* Update mode setting, lookup loading, and caching

* Make `mode` an immutable property
* Only enforce strict `load_lookups` for known supported modes
* Move caching into individual `_lemmatize` methods

* Implement strict when lang is not found in lookups

* Fix tables/lookups in make_lemmatizer

* Reallow provided lookups and allow for stricter checks

* Add lookups asset to all Lemmatizer pipe tests

* Rename lookups in lemmatizer init test

* Clean up merge

* Refactor lookup table loading

* Add helper from `load_lemmatizer_lookups` that loads required and
optional lookups tables based on settings provided by a config.

Additional slight refactor of lookups:

* Add `Lookups.set_table` to set a table from a provided `Table`
* Reorder class definitions to be able to specify type as `Table`

* Move registry assets into test methods

* Refactor lookups tables config

Use class methods within `Lemmatizer` to provide the config for
particular modes and to load the lookups from a config.

* Add pipe and score to lemmatizer

* Simplify Tagger.score

* Add missing import

* Clean up imports and auto-format

* Remove unused kwarg

* Tidy up and auto-format

* Update docstrings for Lemmatizer

Update docstrings for Lemmatizer.

Additionally modify `is_base_form` API to take `Token` instead of
individual features.

* Update docstrings

* Remove tag map values from Tagger.add_label

* Update API docs

* Fix relative link in Lemmatizer API docs
2020-08-07 15:27:13 +02:00
Adriane Boyd
4aecccf153 Update API docs for AttributeRuler.__init__ 2020-08-07 15:17:25 +02:00
Ines Montani
a8404c3517 validation -> validate 2020-08-07 14:43:47 +02:00
Ines Montani
1d01d89b79 Update CLI docs and evaluate command [ci skip] 2020-08-07 14:40:58 +02:00
Ines Montani
ef2c67cca5
Add DocBin to/from_disk methods and update docs (#5892)
* Add DocBin to/from_disk methods and update docs

* Use DocBin.from_disk in Corpus
2020-08-07 14:30:59 +02:00
Ines Montani
4ca08c6d5d
Merge pull request #5891 from adrianeboyd/docs/attribute-ruler-api
Add AttributeRuler API docs
2020-08-07 13:55:12 +02:00
Adriane Boyd
b8d0c23857 Add AttributeRuler API docs
With additional minor updates to AttributeRuler docstrings.
2020-08-07 12:43:23 +02:00
svlandeg
824f4b2107 casing consistent 2020-08-06 23:20:13 +02:00
svlandeg
b17db0e994 Merge remote-tracking branch 'upstream/develop' into feature/el-docs
# Conflicts:
#	website/docs/usage/training.md
2020-08-06 19:48:52 +02:00
svlandeg
49ddeb99ea add textcat architectures documentation 2020-08-06 19:44:47 +02:00
Ines Montani
e5995904d6 Update docs 2020-08-06 19:30:43 +02:00
svlandeg
e8fd0c1f1e EL architectures documentation 2020-08-06 17:41:26 +02:00
svlandeg
f396f091dc update EL API 2020-08-06 16:40:48 +02:00
svlandeg
81d0b1c390 update EL pipe arguments 2020-08-06 16:22:50 +02:00
svlandeg
0b4d1e1bc4 'debug data' instead of 'debug-data' 2020-08-06 15:47:31 +02:00
svlandeg
881e3f8fd0 add docbin explanation and example 2020-08-06 15:29:44 +02:00
Ines Montani
5d417d3b19 WIP: Update docs [ci skip] 2020-08-06 13:10:15 +02:00
Ines Montani
06e80d95cd
Sync develop with nightly docs state (#5883)
Co-authored-by: svlandeg <sofie.vanlandeghem@gmail.com>
2020-08-06 00:28:14 +02:00
Ines Montani
5cc0d89fad
Simplify config overrides in CLI and deserialization (#5880) 2020-08-05 23:35:09 +02:00
Ines Montani
50311a4d37 Update docs [ci skip] 2020-08-05 20:29:53 +02:00
Ines Montani
2a4d56e730 Update docs 2020-08-05 15:01:00 +02:00
Ines Montani
cdec46493f Update docs 2020-08-05 15:00:54 +02:00
Adriane Boyd
c62fd878a3
Allow Doc.char_span to snap to token boundaries (#5849)
* Allow Doc.char_span to snap to token boundaries

Add a `mode` option to allow `Doc.char_span` to snap to token
boundaries. The `mode` options:

* `strict`: character offsets must match token boundaries (default, same as
before)
* `inside`: all tokens completely within the character span
* `outside`: all tokens at least partially covered by the character span

Add a new helper function `token_by_char` that returns the token
corresponding to a character position in the text. Update
`token_by_start` and `token_by_end` to use `token_by_char` for more
efficient searching.

* Remove unused import

* Rename mode to alignment_mode

Rename `mode` to `alignment_mode` with the options
`strict`/`contract`/`expand`. Any unrecognized modes are silently
converted to `strict`.
2020-08-04 13:36:32 +02:00
Ines Montani
4c055f0aa7
Add init CLI and init config (#5854)
* Add init CLI and init config draft

* Improve config validation

* Auto-format

* Don't export anything in debug config

* Update docs
2020-08-02 15:18:30 +02:00
Ines Montani
b40f44419b Simplify pipe analysis
- remove unused code
- don't print by default
- integrate attrs info into analysis output
2020-08-01 13:40:06 +02:00
Ines Montani
98c6a85c8b Update docs [ci skip] 2020-07-31 18:55:38 +02:00
Ines Montani
e9e8fa2466 Update docs and types 2020-07-31 17:02:54 +02:00
Ines Montani
6365837ca9
Merge pull request #5833 from explosion/feature/scorer-adjustments 2020-07-31 14:00:39 +02:00
Ines Montani
5a221f79c2 Revert "Remove keyword-only from Scorer API docs" [ci skip]
This reverts commit 7a6ac47dc1.
2020-07-31 14:00:21 +02:00
Ines Montani
160f1a5f94 Update docs [ci skip] 2020-07-31 13:26:39 +02:00
Adriane Boyd
9b509aa87f Move Language.evaluate scorer config to new arg
Move `Language.evaluate` scorer config from `component_cfg` to separate
argument `scorer_cfg`.
2020-07-31 11:05:16 +02:00
Adriane Boyd
9d79916792 Merge branch 'develop' into feature/scorer-adjustments 2020-07-31 10:48:14 +02:00
Ines Montani
3449c45fd9 Update docs [ci skip] 2020-07-29 19:48:26 +02:00
Ines Montani
9c80cb673d Update docs [ci skip] 2020-07-29 19:41:34 +02:00
Ines Montani
9f69afdd1e Update docs [ci skip] 2020-07-29 19:09:44 +02:00
Ines Montani
7a21775cd0
Merge pull request #5834 from explosion/feature/vectors 2020-07-29 18:49:26 +02:00
Ines Montani
6a5c853edb Fix docs [ci skip] 2020-07-29 18:45:12 +02:00
Ines Montani
158d8c1e48 Update docs [ci skip] 2020-07-29 18:44:10 +02:00
Matthew Honnibal
f7adc9d3b7 Start rewriting vectors docs 2020-07-29 17:10:06 +02:00
Ines Montani
b0f57a0cac Update docs and consistency 2020-07-29 15:14:07 +02:00
Ines Montani
e0ffe36e79 Update docstrings, docs and types 2020-07-29 11:36:42 +02:00
Adriane Boyd
7a6ac47dc1 Remove keyword-only from Scorer API docs 2020-07-29 10:40:30 +02:00
Ines Montani
ac24adec73 Small adjustments to Scorer and docs 2020-07-28 21:39:42 +02:00
Ines Montani
256b24b720 Update arch docs WIP [ci skip] 2020-07-28 20:33:52 +02:00
Ines Montani
ae4d8a6ffd Update docstrings, docs and pipe consistency 2020-07-28 13:37:31 +02:00
Ines Montani
0094cb0d04 Remove scores list from config and document 2020-07-28 11:22:24 +02:00
Ines Montani
894e20c466 Merge branch 'develop' into feature/component-scores 2020-07-27 18:14:39 +02:00
Ines Montani
d8b519c23c API docs, docstrings and argument consistency 2020-07-27 18:11:45 +02:00
Ines Montani
10b84e1e27 Add flag to toggle sdist creation on package [ci skip] 2020-07-27 16:52:23 +02:00
Adriane Boyd
fdf09cb231 Update Scorer API docs for score_cats 2020-07-27 15:34:42 +02:00
Ines Montani
7dd53d0964 Fix typo [ci skip] 2020-07-27 00:34:00 +02:00
Ines Montani
7adbaf9a5b Update docs [ci skip] 2020-07-27 00:29:45 +02:00
Matthew Honnibal
fb5dbe30b5 Trim training 101 2020-07-26 13:43:22 +02:00
Matthew Honnibal
e6a7deb7cc Edits to the training 101 section 2020-07-26 13:42:08 +02:00
Ines Montani
c288dba8e7 Update docs [ci skip] 2020-07-25 18:51:12 +02:00
Ines Montani
eb9acae34d
Merge pull request #5791 from adrianeboyd/docs/morphology 2020-07-25 15:10:21 +02:00
Li Zhe
a69eb445dc
fix the wrong hash url in adding-languages.md file (#5810)
* fix the wrong hash url in adding-languages.md file

change the #101 url hash path to #language-data

* filled in the spaCy Contributor Agreement 

filled in the spaCy Contributor Agreement
2020-07-25 13:13:38 +02:00
Adriane Boyd
2bcceb80c4
Refactor the Scorer to improve flexibility (#5731)
* Refactor the Scorer to improve flexibility

Refactor the `Scorer` to improve flexibility for arbitrary pipeline
components.

* Individual pipeline components provide their own `evaluate` methods
that score a list of `Example`s and return a dictionary of scores
* `Scorer` is initialized either:
  * with a provided pipeline containing components to be scored
  * with a default pipeline containing the built-in statistical
    components (senter, tagger, morphologizer, parser, ner)
* `Scorer.score` evaluates a list of `Example`s and returns a dictionary
of scores referring to the scores provided by the components in the
pipeline

Significant differences:

* `tags_acc` is renamed to `tag_acc` to be consistent with `token_acc`
and the new `morph_acc`, `pos_acc`, and `lemma_acc`
* Scoring is no longer cumulative: `Scorer.score` scores a list of
examples rather than a single example and does not retain any state
about previously scored examples
* PRF values in the returned scores are no longer multiplied by 100

* Add kwargs to Morphologizer.evaluate

* Create generalized scoring methods in Scorer

* Generalized static scoring methods are added to `Scorer`
  * Methods require an attribute (either on Token or Doc) that is
used to key the returned scores

Naming differences:

* `uas`, `las`, and `las_per_type` in the scores dict are renamed to
`dep_uas`, `dep_las`, and `dep_las_per_type`

Scoring differences:

* `Doc.sents` is now scored as spans rather than on sentence-initial
token positions so that `Doc.sents` and `Doc.ents` can be scored with
the same method (this lowers scores since a single incorrect sentence
start results in two incorrect spans)

* Simplify / extend hasattr check for eval method

* Add hasattr check to tokenizer scoring
* Simplify to hasattr check for component scoring

* Reset Example alignment if docs are set

Reset the Example alignment if either doc is set in case the
tokenization has changed.

* Add PRF tokenization scoring for tokens as spans

Add PRF scores for tokens as character spans. The scores are:

* token_acc: # correct tokens / # gold tokens
* token_p/r/f: PRF for (token.idx, token.idx + len(token))

* Add docstring to Scorer.score_tokenization

* Rename component.evaluate() to component.score()

* Update Scorer API docs

* Update scoring for positive_label in textcat

* Fix TextCategorizer.score kwargs

* Update Language.evaluate docs

* Update score names in default config
2020-07-25 12:53:02 +02:00
Adriane Boyd
8f44584bef Update MorphAnalysis.get and related examples 2020-07-23 08:51:31 +02:00
Adriane Boyd
941b9e33f7 Add Token.morph_ 2020-07-22 17:59:45 +02:00
Ines Montani
be476e495e
Merge pull request #5787 from adrianeboyd/docs/morphologizer
Initial draft of Morphologizer API docs
2020-07-22 17:16:57 +02:00
Adriane Boyd
d3385f4be2 Add Morphology and MorphAnalysis to overview 2020-07-21 13:06:22 +02:00
Adriane Boyd
fcd3a4abe3 Add morph to Token API docs 2020-07-21 13:05:58 +02:00
Adriane Boyd
14df00ae98 Add Morphology and MorphAnalsysis API docs
Add initial draft of `Morphology` and `MorphAnalysis` API docs.
2020-07-21 10:33:46 +02:00
Ines Montani
644074b954 Merge branch 'develop' into master-tmp 2020-07-20 14:58:04 +02:00
Adriane Boyd
986f7e4d69 Initial draft of Morphologizer API docs 2020-07-20 12:53:02 +02:00
Adriane Boyd
39ebcd9ec9
Refactor Chinese tokenizer configuration (#5736)
* Refactor Chinese tokenizer configuration

Refactor `ChineseTokenizer` configuration so that it uses a single
`segmenter` setting to choose between character segmentation, jieba, and
pkuseg.

* replace `use_jieba`, `use_pkuseg`, `require_pkuseg` with the setting
`segmenter` with the supported values: `char`, `jieba`, `pkuseg`
* make the default segmenter plain character segmentation `char` (no
additional libraries required)

* Fix Chinese serialization test to use char default

* Warn if attempting to customize other segmenter

Add a warning if `Chinese.pkuseg_update_user_dict` is called when
another segmenter is selected.
2020-07-19 13:34:37 +02:00
Adriane Boyd
cd5af72c9a
Update pkuseg version (#5774)
* Update pkuseg version in Chinese tokenizer warnings
* Update pkuseg version in `Makefile`
* Remove warning about python3.8 wheels in docs
2020-07-19 11:09:49 +02:00
Ines Montani
872938ec76
Merge pull request #5747 from explosion/feature/refactor-config-args 2020-07-14 00:00:22 +02:00
Ines Montani
5f6f4ff594 Remove object subclassing 2020-07-12 14:03:23 +02:00
Ines Montani
c96535e338 Update command docstrings and docs 2020-07-12 13:53:49 +02:00
Ines Montani
3f948b9c74 Update docs 2020-07-12 12:32:28 +02:00
Ines Montani
11bbc82c24 Update cli.md [ci skip] 2020-07-10 23:37:52 +02:00
Ines Montani
9455b060d2 Update cli.md 2020-07-10 22:57:22 +02:00
Ines Montani
7b5717cac3 Merge branch 'develop' into feature/refactor-config-args 2020-07-10 22:50:07 +02:00
Ines Montani
e6a6587a9a Update projects.md [ci skip] 2020-07-10 22:41:27 +02:00
Ines Montani
f2cd982e7b Update training.md 2020-07-10 22:34:27 +02:00
Ines Montani
52e9b5b472 Fix formatting 2020-07-09 23:25:58 +02:00
Ines Montani
28cdae898a Update projects.md 2020-07-09 22:35:54 +02:00
Ines Montani
7bcf9f7cfb Document new features 2020-07-09 21:10:36 +02:00
Ines Montani
ea01831f6a Update projects docs etc. 2020-07-09 19:43:25 +02:00
Ines Montani
175d34d8f9 Update sidebar menu 2020-07-09 11:44:09 +02:00
Ines Montani
9ee5b71412 Update cli.md 2020-07-09 11:44:00 +02:00
Ines Montani
9ae4040183 Update API docs 2020-07-08 13:34:35 +02:00
svlandeg
c94279ac1b remove tensors, fix predict, get_loss and set_annotations 2020-07-08 13:11:54 +02:00
svlandeg
90b100c39f remove component.Model, update constructor, losses is return value of update 2020-07-08 12:14:30 +02:00
Ines Montani
2298e129e6 Update example and training docs 2020-07-07 20:30:12 +02:00
svlandeg
2b60e894cb fix component constructors, update, begin_training, reference to GoldParse 2020-07-07 19:17:19 +02:00
svlandeg
14a796e3f9 add Example API with examples of Example usage 2020-07-07 14:46:41 +02:00
Ines Montani
bb3ee38cf9 Update WIP 2020-07-06 22:22:37 +02:00
Ines Montani
44da24ddd0 Update doc.md 2020-07-06 18:17:00 +02:00
Ines Montani
44790c1c32 Update docs and add keyword-only tag 2020-07-06 18:14:57 +02:00
Ines Montani
a35236e5f0 Update v3 docs WIP [ci skip] 2020-07-06 15:57:44 +02:00
Ines Montani
63247cbe87 Update v3 docs [ci skip] 2020-07-05 16:11:16 +02:00
Matthew Honnibal
3e78e82a83
Experimental character-based pretraining (#5700)
* Use cosine loss in Cloze multitask

* Fix char_embed for gpu

* Call resume_training for base model in train CLI

* Fix bilstm_depth default in pretrain command

* Implement character-based pretraining objective

* Use chars loss in ClozeMultitask

* Add method to decode predicted characters

* Fix number characters

* Rescale gradients for mlm

* Fix char embed+vectors in ml

* Fix pipes

* Fix pretrain args

* Move get_characters_loss

* Fix import

* Fix import

* Mention characters loss option in pretrain

* Remove broken 'self attention' option in pretrain

* Revert "Remove broken 'self attention' option in pretrain"

This reverts commit 56b820f6af.

* Document 'characters' objective of pretrain
2020-07-05 15:48:39 +02:00
Ines Montani
dc8c9d912f Update docs [ci skip] 2020-07-04 16:47:24 +02:00
Ines Montani
4498dfe99d Update docs 2020-07-04 16:25:30 +02:00
Ines Montani
1e0d54edd1 Update docs 2020-07-04 14:23:10 +02:00
Ines Montani
fe224dc2dd Merge branch 'develop' into nightly.spacy.io 2020-07-03 16:48:27 +02:00
Ines Montani
06f1ecb308 Update v3 docs 2020-07-03 16:48:21 +02:00
Ines Montani
cdf9ee1716 Add stub for Example API docs [ci skip] 2020-07-03 15:46:10 +02:00
Ines Montani
fa8e097c04 Update convert docs [ci skip] 2020-07-03 15:42:04 +02:00
Jan Jessewitsch
e4dcac4a4b
Merging multiple docs into one (#5032)
* Add static method to Doc to allow merging of multiple docs.

* Add error description for the error that occurs if docs with different
vocabs (from different languages) are merged in Doc.from_docs().

* Add test for Doc.from_docs() implementation.

* Fix using numpy's concatenate in Doc.from_docs.

* Replace typing's type annotations in from_docs.

* Simply remove type annotations in from_docs.

* Add documentation for Doc.from_docs to api.

* Simplify from_docs, its test and the api doc for codebase consistency.

* Fix merging of Doc objects that end with whitespaces (Achieved by simply not setting the SPACY attribute on whitespace tokens). Remove two unnecessary imports of attributes.

* Add merging of user data from Doc objects in from_docs. Add user data test case to corresponding test. Add applicable warning messages.

* Fix incorrect setting of tokens idx by using concatenated spaces (again). Add test case to corresponding test.

* Add MORPH to attrs

* Update warnings calls

* Remove out-dated error from merge

* Rename space_delimiter to ensure_whitespace

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2020-07-03 11:32:42 +02:00
Adriane Boyd
a723fa02a1
DocBin: add version number, missing attributes and strings (#5685)
* Add version number to DocBin

Add a version number to DocBin for future use.

* Add POS to all attributes in DocBin

* Add morph string to strings in DocBin

* Update DocBin API

* Add string for ENT_KB_ID in DocBin
2020-07-02 17:41:50 +02:00
Ines Montani
b5268955d7 Update matcher usage examples [ci skip] 2020-07-02 15:39:45 +02:00
Ines Montani
a4cfe9fc33 Remove inline notes on v2 changes [ci skip] 2020-07-01 22:29:22 +02:00
Ines Montani
fe4cfd0632 Start updating website for v3 [ci skip] 2020-07-01 21:26:39 +02:00
Ines Montani
26df4efa94 Add new in v3.0 2020-07-01 13:02:17 +02:00
Ines Montani
18a900abc2 Fix markup 2020-07-01 13:02:07 +02:00
Ines Montani
414dc7ace1 Merge branch 'spacy.io' into spacy.io-develop 2020-07-01 11:47:47 +02:00