explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-12-25 17:36:30 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,186 Commits 70 Branches 177 Tags 279 MiB
6ca06cb62c
Commit Graph

7814 Commits

Author SHA1 Message Date
Matthew Honnibal
465785a672 Fix project pull and push 2020-09-04 21:15:55 +02:00
Ines Montani
f174c7b1f3 Merge branch 'develop' into pr/6018 2020-09-04 15:54:49 +02:00
Ines Montani
f06eed800e
Merge pull request #6029 from explosion/master-tmp 2020-09-04 15:11:55 +02:00
Ines Montani
f9550b4493 Fix components in meta.json and website [ci skip] 2020-09-04 14:42:12 +02:00
Ines Montani
d7cc2ee72d Fix tests 2020-09-04 14:05:55 +02:00
Ines Montani
90043a6f9b Tidy up and auto-format 2020-09-04 13:42:33 +02:00
Ines Montani
df0b68f60e Remove unicode declarations and update language data 2020-09-04 13:19:16 +02:00
Ines Montani
ba600f91c5 Tidy up imports 2020-09-04 13:15:44 +02:00
Ines Montani
864a697e63 Merge branch 'develop' into master-tmp 2020-09-04 13:15:36 +02:00
Adriane Boyd
b927893309
Merge branch 'develop' into feature/dependency-matcher-v3 2020-09-04 13:03:30 +02:00
Ines Montani
ab1bb421ed Update docs links in codebase 2020-09-04 12:58:50 +02:00
holubvl3
0a27fca557
Create examples.py (#5985) 
* Create examples.py

* Create tag_map.py

* Delete tag_map.py

* Update examples.py

formatting: add empty line

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
 2020-09-04 11:00:14 +02:00
Ines Montani
2189046869
Merge pull request #6024 from explosion/chore/registry-renaming 2020-09-04 10:54:10 +02:00
svlandeg
c32fcdf4c9 fix typo 2020-09-04 09:10:21 +02:00
Ines Montani
595f9dc2e4 Make displacy color registry consistent with others 
This was the only registry that expected the registered objects to be dictionaries instead of functions that return something. We can still support plain dicts but we should also support functions for consistency
 2020-09-03 23:05:41 +02:00
Matthew Honnibal
1c07820681 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-09-03 18:54:21 +02:00
Matthew Honnibal
7be8a0516a Fix project pull 2020-09-03 18:54:03 +02:00
Ines Montani
23b7d9cfa3 Prefix span getters 2020-09-03 17:37:06 +02:00
Ines Montani
5afe6447cd registry.assets -> registry.misc 2020-09-03 17:31:14 +02:00
Ines Montani
c063e55eb7 Add prefix to batchers 2020-09-03 17:30:41 +02:00
Ines Montani
896caf45e3
Merge pull request #6023 from explosion/ux/model-terminology-consistency [ci skip] 2020-09-03 17:13:44 +02:00
Ines Montani
c53b1433b9 Adjust more arguments [ci skip] 2020-09-03 17:12:24 +02:00
Ines Montani
b5a0657fd6 "model" terminology consistency in docs 2020-09-03 13:13:03 +02:00
Matthew Honnibal
f038841798 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-09-03 12:52:39 +02:00
Matthew Honnibal
ef0d0630a4 Let Langugae.use_params work with falsey inputs 
The Language.use_params method was failing if you passed in None, which
meant we had to use awkward conditionals for the parameter averaging.
This solves the problem.
 2020-09-03 12:51:04 +02:00
Yohei Tamura
5af432e0f2
fix for empty string (#5936) 2020-09-03 10:09:03 +02:00
Adriane Boyd
77ac4a38aa
Simplify specials and cache checks (#6012) 2020-09-03 09:42:49 +02:00
Adriane Boyd
8b5594df86 Remove near-duplicate test 2020-09-02 20:32:01 +02:00
Matthew Honnibal
122cb02001 Fix averages 2020-09-02 19:37:43 +02:00
Adriane Boyd
960d9cfadc Officially support DependencyMatcher 
Add official support for the `DependencyMatcher`. Redesign the pattern
specification. Fix and extend operator implementations. Update API docs
and add usage docs.

Patterns
--------

Refactor pattern structure to:

```
{
  "LEFT_ID": str,
  "REL_OP": str,
  "RIGHT_ID": str,
  "RIGHT_ATTRS": dict,
}
```

The first node contains only `RIGHT_ID` and `RIGHT_ATTRS` and all
subsequent nodes contain all four keys.

New operators
-------------

Because of the way patterns are constructed from left to right, it's
helpful to have `follows` operators along with `precedes` operators. Add
operators for simple precedes / follows alongside immediate precedes /
follows.

* `.*`: precedes
* `;`: immediately follows
* `;*`: follows

Operator fixes
--------------

* `<` and `<<` do not include the node itself
* Fix reversed order for all operators involving linear precedence (`.`,
  all sibling operators)
* Linear precedence operators do not match nodes outside the same parse

Additional fixes
----------------

* Use v3 Matcher API
* Support `get` and `remove`
* Support pickling
 2020-09-02 17:45:29 +02:00
Marek Grzenkowicz
92d7832a86
Fix off-by-one error for best iteration calculation (closes #6014) (#6016) 2020-09-02 15:15:45 +02:00
Matthew Honnibal
737a1408d9 Improve implementation of fix #6010 
Follow-ups to the parser efficiency fix.

* Avoid introducing new counter for number of pushes
* Base cut on number of transitions, keeping it more even
* Reintroduce the randomization we had in v2.
 2020-09-02 14:42:32 +02:00
Sofie Van Landeghem
eb56377799
Fix overfitting test (#6011) 
* remove unused MORPH_RULES

* fix textcat architecture in overfitting test
 2020-09-02 13:07:41 +02:00
Adriane Boyd
b97d98783a
Fix Hungarian % tokenization (#6013) 2020-09-02 13:06:16 +02:00
Matthew Honnibal
c1bf3a5602
Fix significant performance bug in parser training (#6010) 
The parser training makes use of a trick for long documents, where we
use the oracle to cut up the document into sections, so that we can have
batch items in the middle of a document. For instance, if we have one
document of 600 words, we might make 6 states, starting at words 0, 100,
200, 300, 400 and 500.

The problem is for v3, I screwed this up and didn't stop parsing! So
instead of a batch of [100, 100, 100, 100, 100, 100], we'd have a batch
of [600, 500, 400, 300, 200, 100]. Oops.

The implementation here could probably be improved, it's annoying to
have this extra variable in the state. But this'll do.

This makes the v3 parser training 5-10 times faster, depending on document
lengths. This problem wasn't in v2.
 2020-09-02 12:57:13 +02:00
Sofie Van Landeghem
f7a25d69f7
Bugfix in merge_entities (#6005) 
* failing test

* bugfix
 2020-09-01 21:57:52 +02:00
Sofie Van Landeghem
6bfb1b3a29
Fix sparse checkout for 'spacy project' (#6008) 
* exit if cloning fails

* UX

* rewrite http link to git protocol, don't use stdin

* fixes to sparse checkout

* formatting
 2020-09-01 19:49:01 +02:00
Matthew Honnibal
4cce32f090 Fix tagger initialization 2020-09-01 16:38:34 +02:00
Matthew Honnibal
046c38bd26
Remove 'cleanup' of strings (#6007) 
A long time ago we went to some trouble to try to clean up "unused"
strings, to avoid the `StringStore` growing in long-running processes.

This never really worked reliably, and I think it was a really wrong
approach. It's much better to let the user reload the `nlp` object as
necessary, now that the string encoding is stable (in v1, the string IDs
were sequential integers, making reloading the NLP object really
annoying.)

The extra book-keeping does make some performance difference, and the
feature is unsed, so it's past time we killed it.
 2020-09-01 16:12:15 +02:00
Ines Montani
70b226f69d Support ignore marker in project document [ci skip] 2020-09-01 12:49:04 +02:00
Ines Montani
a4c51f0f18 Add v3 info to project docs [ci skip] 2020-09-01 12:36:21 +02:00
Ines Montani
ef9005273b Update fill-config command and add silent mode [ci skip] 2020-09-01 12:07:04 +02:00
Matthew Honnibal
ec660e3131 Fix use_pytorch_for_gpu_memory 2020-09-01 00:41:38 +02:00
Adriane Boyd
9130094199
Prevent Tagger model init with 0 labels (#5984) 
* Prevent Tagger model init with 0 labels

Raise an error before trying to initialize a tagger model with 0 labels.

* Add dummy tagger label for test

* Remove tagless tagger model initializiation

* Fix error number after merge

* Add dummy tagger label to test

* Fix formatting

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-08-31 21:24:33 +02:00
Matthw Honnibal
c38298b8fa Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-31 19:55:55 +02:00
Matthw Honnibal
fe298fa50a Shuffle on first epoch of train 2020-08-31 19:55:22 +02:00
Ines Montani
9af82f3f11
Merge pull request #6003 from explosion/feature/matcher-as-spans 2020-08-31 17:50:56 +02:00
Ines Montani
add9de5487 Deprecate (Phrase)Matcher.pipe 2020-08-31 17:01:24 +02:00
Ines Montani
83aff38c59
Make argument keyword-only 
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-08-31 15:39:03 +02:00
Ines Montani
6340d1c63d Add as_spans to Matcher/PhraseMatcher 2020-08-31 14:53:22 +02:00
svlandeg
13ee742fb4 example of custom logger 2020-08-31 14:24:41 +02:00
svlandeg
c18eb63483 Merge remote-tracking branch 'upstream/develop' into feature/vectors-docs 
# Conflicts:
#	website/docs/usage/embeddings-transformers.md
 2020-08-31 13:21:36 +02:00
Sofie Van Landeghem
ec14744ee4
Rename Transformer listener (#6001) 
* rename to spacy-transformers.TransformerListener

* add some more tok2vec tests

* use select_pipes

* fix docs - annotation setter was not changed in the end
 2020-08-31 12:41:39 +02:00
Adriane Boyd
216efaf5f5 Restrict tokenizer exceptions to ORTH and NORM 2020-08-31 09:55:01 +02:00
Matthew Honnibal
9341cbc013 Set version to v3.0.0a13 2020-08-30 23:10:43 +02:00
Ines Montani
45f46a5c85
Merge pull request #5993 from explosion/feature/disabled-components 2020-08-29 15:58:41 +02:00
Ines Montani
34146750d4 Use frozen list with custom errors 
We don't want to break backwards compatibility too much but we also want to provide the best possible UX
 2020-08-29 15:20:11 +02:00
Ines Montani
744f432420
Merge pull request #5994 from explosion/feature/idempotent-component-decorator 2020-08-29 13:17:13 +02:00
Ines Montani
5de3f8604d
Update spacy/util.py 
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-08-29 13:17:06 +02:00
Ines Montani
091a9b522a Remove unused variable [ci skip] 2020-08-29 13:11:26 +02:00
Ines Montani
2bc31e15c9 Tidy up and auto-format [ci skip] 2020-08-29 13:01:10 +02:00
Ines Montani
6520d1a1df Work around set order in Language.disabled 2020-08-29 12:58:22 +02:00
Ines Montani
f45095a666
Merge pull request #5995 from adrianeboyd/bugfix/attribute-ruler-bugfixes 2020-08-29 12:38:30 +02:00
Ines Montani
e0b4984aa4 Make deprecated disable_pipes call into select_pipes 2020-08-29 12:08:46 +02:00
Ines Montani
15d73f4dc3 Make user-facing Language.disabled return list 
More consistent with all the other properties
 2020-08-29 12:08:33 +02:00
Matthew Honnibal
58f19421b1 Return empty batch from tok2vec listener if no doc.tensor 2020-08-29 03:46:50 +02:00
svlandeg
5230529de2 add loggers registry & logger docs sections 2020-08-28 21:44:04 +02:00
Ines Montani
0687d7148e Rename user-facing API 2020-08-28 21:04:02 +02:00
Adriane Boyd
0104bd1600 Sort the AttributeRuler matches by rule order 
Sort the returned matches by rule order (the `match_id`) so that the
rules are applied in the order they were added. This is necessary, for
instance, if the `AttributeRuler` is used for the tag map and later
rules require POS tags.
 2020-08-28 21:01:06 +02:00
Ines Montani
6a999c9303 Remove outdated component attr check 2020-08-28 20:59:19 +02:00
Adriane Boyd
8674b17651 Serialize AttributeRuler.patterns 
Serialize `AttributeRuler.patterns` instead of the individual lists to
simplify the serialized and so that patterns are reloaded exactly as
they were originally provided (preserving `_attrs_unnormed`).
 2020-08-28 20:44:45 +02:00
Ines Montani
10da74382f Raise if disabled components are removed before DisabledPipes.restore 2020-08-28 20:35:26 +02:00
Ines Montani
1e0363290e Remove todos and update docstrings 2020-08-28 20:34:46 +02:00
Ines Montani
cad988da7f Allow component decorators to re-run with same function 2020-08-28 16:27:22 +02:00
Ines Montani
3ce5be4b76 Allow loaded but disabled components 2020-08-28 15:20:14 +02:00
Ines Montani
89f692bc8a
Merge pull request #5992 from svlandeg/feature/wandb-restrict-config 2020-08-28 15:05:29 +02:00
Ines Montani
9c4049b57f
Merge pull request #5986 from explosion/fix/language-config-interpolate-disk-bytes 2020-08-28 15:03:52 +02:00
Ines Montani
adc050cdc5 Fix code style in test [ci skip] 2020-08-28 15:03:21 +02:00
svlandeg
05a1bafa15 fix type 2020-08-28 14:08:33 +02:00
svlandeg
33883aa764 rename field 2020-08-28 14:06:23 +02:00
svlandeg
1d8c4070aa add disable_fields to wandb_logger 2020-08-28 13:55:32 +02:00
Ines Montani
a51b4f3a19 Merge branch 'develop' into fix/language-config-interpolate-disk-bytes 2020-08-28 13:21:17 +02:00
Ines Montani
03dde511b4
Merge pull request #5987 from explosion/feature/debug-config [ci skip] 2020-08-28 11:30:18 +02:00
Ines Montani
62e9967228 Merge branch 'develop' into fix/language-config-interpolate-disk-bytes 2020-08-28 11:19:36 +02:00
Ines Montani
4ca2698f85 Merge branch 'develop' into feature/debug-config 2020-08-28 11:19:17 +02:00
svlandeg
9a8255ffd5 two tests because of different exit type 2020-08-28 10:50:26 +02:00
svlandeg
73baaf330a update error type 2020-08-28 10:46:21 +02:00
Matthew Honnibal
c558ca4485 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-27 19:47:26 +02:00
Matthew Honnibal
d3ffe4ca63 Fix error when tagger was initialized with no labels 2020-08-27 18:56:58 +02:00
Ines Montani
d1780db6a4 Tidy up and use different error [ci skip] 2020-08-27 18:56:55 +02:00
Ines Montani
ff4175e839 Add more info to debug config 2020-08-27 18:17:58 +02:00
Ines Montani
daac8ebacd Don't interpolate config on Language deserialization 2020-08-27 16:44:36 +02:00
Matthew Honnibal
e1e1760fd6 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-27 03:22:11 +02:00
Matthew Honnibal
95adb58f15 Force tagger to pass batch of docs into model in begin_training 2020-08-27 03:21:03 +02:00
Ines Montani
cdc114e212
Merge pull request #5977 from explosion/refactor/vector-names 2020-08-26 19:03:16 +02:00
Ines Montani
8692d176f6
Merge pull request #5978 from explosion/feature/update-wasabi 
Update wasabi: new diff_strings and MarkdownRenderer
 2020-08-26 19:02:52 +02:00
Matthew Honnibal
9b22714a4e Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-26 15:48:45 +02:00
Matthew Honnibal
172af24f95 Fix upload and download 2020-08-26 15:48:23 +02:00
Ines Montani
a5fff1df51 Remove outdated non-empty output dir warning [ci skip] 2020-08-26 15:45:51 +02:00
Matthew Honnibal
2d520d3b45 Remove unused error 2020-08-26 15:41:14 +02:00
Adriane Boyd
90d88729e0
Add AttributeRuler.score (#5963) 
* Add AttributeRuler.score

Add scoring for TAG / POS / MORPH / LEMMA if these are present in the
assigned token attributes.

Add default score weights (that don't really make a lot of sense) so
that the scores are in the default config in some form.

* Update docs
 2020-08-26 15:39:30 +02:00
Ines Montani
3aec98ca38 Update wasabi: new diff_strings and MarkdownRenderer 2020-08-26 15:33:11 +02:00
Sofie Van Landeghem
79d460e3a2
Weights & Biases logger for train CLI (#5971) 
* quick test as part of train script

* train_logger in config, default ConsoleLogger in loggers catalogue

* entitiy typo

* add wandb_logger

* cleanup

* Update spacy/cli/train_logger.py

Co-authored-by: Ines Montani <ines@ines.io>

* move loggers to gold.loggers

Co-authored-by: Ines Montani <ines@ines.io>
 2020-08-26 15:24:33 +02:00
Ines Montani
0997c30b9e
Merge pull request #5974 from explosion/feature/project-document 2020-08-26 15:14:13 +02:00
Matthew Honnibal
191fb4144f Merge branch 'develop' into refactor/vector-names 2020-08-26 14:26:45 +02:00
Ines Montani
627617a079 Tidy up and add docs [ci skip] 2020-08-26 13:24:55 +02:00
Adriane Boyd
43c61da209 Set macro AUC score in Scorer.score_cats 2020-08-26 10:49:30 +02:00
Ines Montani
aeebc6678d Small cleanup and adjustments 2020-08-26 10:26:57 +02:00
Ines Montani
31567d1e42 Link project.yml 2020-08-26 10:26:32 +02:00
Ines Montani
6c2a5ff53b Auto-link local sources 2020-08-26 10:26:06 +02:00
Matthew Honnibal
77852d2428 Fix run_command for python 3.6 2020-08-26 05:02:43 +02:00
Matthew Honnibal
884cac5fb5 Make run_command backwards compatible 2020-08-26 04:33:42 +02:00
Matthew Honnibal
6547472347 Set version to v3.0.0a12 2020-08-26 04:02:34 +02:00
Adriane Boyd
7d7b65ffd4
Fix raw strings in URL pattern (#5972) 
Add missing raw string specifiers.
 2020-08-26 04:00:49 +02:00
Matthew Honnibal
2771e4f2b3
Fix the git "sparse checkout" functionality (#5973) 
* Fix the git sparse checkout functionality

* Format
 2020-08-26 04:00:14 +02:00
Ines Montani
1c958a76c1 Add comment markers to only replace auto-generated docs 2020-08-26 00:03:06 +02:00
Ines Montani
f10989e8c4 Add "project document" and more project.yml meta fields 2020-08-25 17:14:27 +02:00
Ines Montani
fdcaf86c54 Adjust docstring 
End sentence earlier so it's shown as a full sentence in --help
 2020-08-25 17:13:50 +02:00
Ines Montani
b89f6fa011 Fix meta defaults and error in package command 2020-08-25 17:13:33 +02:00
Ines Montani
94705c21c8 Allow reuse on validators to prevent reload error 
Otherwise this will cause an error if spaCy is live reloaded, e.g. in Streamlit
 2020-08-25 17:13:11 +02:00
Matthew Honnibal
4f82a02b70 Remove 'fix_pretrained_vectors_name' hack 2020-08-25 14:37:45 +02:00
Adriane Boyd
0bab7c8b91
Remove PRON_LEMMA symbol (#5968) 2020-08-25 14:21:29 +02:00
Hiroshi Matsuda
332803eda9
fix ja leading spaces (#5969) 
* change condition for space after

* add NAUGHTY_STRINGS test example
 2020-08-25 14:16:24 +02:00
Ines Montani
dd84577a98 Update CLI utils, project.yml schema and add test 2020-08-25 11:54:53 +02:00
Shashank
450720aca2
Added support for Sanskrit language (#5956) 
* Added support for Sanskrit language

* Added tests for lexical attribute like_num
 2020-08-25 10:56:29 +02:00
Matthew Honnibal
ef43152af4 Update scorer 2020-08-25 02:42:47 +02:00
Matthew Honnibal
8d6e1ce306 Update v3.0.0a11 2020-08-25 00:32:08 +02:00
Matthew Honnibal
8038b87f04
Various small tweaks to project CLI (#5965) 
* Fix up/download of http and local paths

* Support git_sparse_checkout for assets

* Fix scorer

* Handle already-present directories for git assets

* Improve convert command

* Fix support for existant files in git assets

* Support branches in git sparse checkout

* Format

* Fix git assets

* Document git block in assets

* Fix test

* Fix test

* Revert "Fix test"

This reverts commit cf3097260f.

* Revert "Fix test"

This reverts commit 964d636e27.

* Dont multiply p/r/f by 100

* Display scores * 100 during training
 2020-08-25 00:30:52 +02:00
Adriane Boyd
abd3f2b65a
Rename Polish lemmatizer method (#5960) 
Rename Polish lemmatizer method to `pos_lookup` to distinguish it from
pure token-based lookup methods.
 2020-08-25 00:22:27 +02:00
Ines Montani
e12b03358b
Support removing extra values in fill-config (#5966) 
* Support removing extra values in fill-config

* Fix test
 2020-08-24 22:53:47 +02:00
Matthew Honnibal
f232d8db96 Report p/r/f out of 100 2020-08-24 17:17:23 +02:00
Ines Montani
0e7f99da58
Fix handling of optional [pretraining] block (#5954) 
* Fix handling of optional [pretraining] block

* Remote pretraining from default config

* Fix test

* Add schema option for empty pretrain block
 2020-08-24 15:56:03 +02:00
idoshr
b10c7bc56e
Hebrew like num (#5952) 
* Update stop_words.py

Hebrew STOP WORDS

* Update stop_words.py

* contributor

* contributor

* add some common domain extentions
support human number 1K/1M....

* support human number 1K/1M....

* hebrew number tokenize
1K/1M implement in EN

* test human tokenize fix

* test

* heb like num
revert human number change

* heb like num
 2020-08-24 14:30:05 +02:00
Matthew Honnibal
64df37643f Update lockfile after project pull 2020-08-24 03:27:09 +02:00
Matthew Honnibal
588c28fe45 Fix project pull when deps missing 2020-08-24 01:23:36 +02:00
Matthew Honnibal
001546c19e Set version to v3.0.0a10 2020-08-23 21:15:38 +02:00
Matthew Honnibal
160a855246 Format 2020-08-23 21:15:12 +02:00
Matthew Honnibal
89f5b8abb3 Fix project push 2020-08-23 21:14:44 +02:00
Matthew Honnibal
3828bc3ed0 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-08-23 18:32:24 +02:00
Matthew Honnibal
e559867605
Allow spacy project to push and pull to/from remote storage (#5949) 
* Add utils for working with remote storage

* WIP add remote_cache for project

* WIP add push and pull commands

* Use pathy in remote_cache

* Updarte util

* Update remote_cache

* Update util

* Update project assets

* Update pull script

* Update push script

* Fix type annotation in util

* Work on remote storage

* Remove site and env hash

* Fix imports

* Fix type annotation

* Require pathy

* Require pathy

* Fix import

* Add a util to handle project variable substitution

* Import push and pull commands

* Fix pull command

* Fix push command

* Fix tarfile in remote_storage

* Improve printing

* Fiddle with status messages

* Set version to v3.0.0a9

* Draft docs for spacy project remote storages

* Update docs [ci skip]

* Use Thinc config to simplify and unify template variables

* Auto-format

* Don't import Pathy globally for now

Causes slow and annoying Google Cloud warning

* Tidy up test

* Tidy up and update tests

* Update to latest Thinc

* Update docs

* variables -> vars

* Update docs [ci skip]

* Update docs [ci skip]

Co-authored-by: Ines Montani <ines@ines.io>
 2020-08-23 18:32:09 +02:00
Matthew Honnibal
fe1cf7e124 Allow score_weights to list extra scores 2020-08-23 18:31:30 +02:00
Ines Montani
9bdc9e81f5 Fix error message [ci skip] 2020-08-23 12:14:02 +02:00
Sofie Van Landeghem
56eabcb2f2
Adding num_like test for Czech (#5946) 
* Create lex_attrs.py

Hello,

I am missing a CZECH language in SpaCy. So I would like to help to push it a little. This file is base on others lex_attrs.py files just with translation to Czech.

* Update __init__.py

Updated for use with new Czech Lex_attrs file

* Update stop_words.py

* Create test_text.py

* add like_num testing for czech

Co-authored-by: holubvl3 <47881982+holubvl3@users.noreply.github.com>
Co-authored-by: holubvl3 <vilemrousi@gmail.com>
Co-authored-by: Vladimír Holubec <vholubec@arcdata.cz>
 2020-08-21 17:06:33 +02:00
holubvl3
a341b4ef09
Adding support for Czech language (#5826) 
* Create lex_attrs.py

Hello,

I am missing a CZECH language in SpaCy. So I would like to help to push it a little. This file is base on others lex_attrs.py files just with translation to Czech.

* Update __init__.py

Updated for use with new Czech Lex_attrs file

* Update stop_words.py

* Create test_text.py

Co-authored-by: Vladimír Holubec <vholubec@arcdata.cz>
 2020-08-21 16:17:53 +02:00
svlandeg
af36d77d01 fix typo in docstring 2020-08-21 15:56:03 +02:00
svlandeg
3060e4ae65 Merge remote-tracking branch 'upstream/develop' into feature/docs-docs-docs 
# Conflicts:
#	website/src/widgets/quickstart-training-generator.js
 2020-08-21 15:16:30 +02:00
svlandeg
cc926267f8 small fixes 2020-08-21 15:05:40 +02:00
Ines Montani
aa6a7cd6e7 Update docs and consistency [ci skip] 2020-08-21 13:49:18 +02:00
Ines Montani
3826cfb8fe
Merge pull request #5930 from svlandeg/feature/init-config-fix 
UX for init config
 2020-08-21 12:06:33 +02:00
Ines Montani
79af7dcd6d Small wording adjustments [ci skip] 2020-08-21 12:06:19 +02:00
Ines Montani
e60442d83a Adjust label casing in displaCy NER visualizer (resolves #4866) 
- Accept any case for label names in ents and colors option, even if actual predicted label uses different casing
- Don't text-transform: uppercase visually, if it's important to users that the label is represented as-is in the UI
 2020-08-21 11:51:31 +02:00
Matthew Honnibal
c356e62908 Minor adjustments to quickstart template 2020-08-21 00:10:21 +02:00
Ines Montani
6ad59d59fe Merge branch 'develop' of https://github.com/explosion/spaCy into develop [ci skip] 2020-08-20 11:20:58 +02:00
Sofie Van Landeghem
071c09ff35
add coding (#5942) 2020-08-20 11:08:38 +02:00
Ines Montani
ea6640ea72
Merge pull request #5939 from explosion/feature/thinc-v8.0.0a28 
Update Thinc and config variables
 2020-08-19 21:14:36 +02:00
Ines Montani
3dd390b1a1 Update Thinc and config variables 2020-08-19 19:46:12 +02:00
svlandeg
b96cd9fa5e fix typo 2020-08-19 18:46:08 +02:00
Ines Montani
e2f2ef3a5a Update init config and recommendations 
- As much as I dislike YAML, it seemed like a better format here because it allows us to add comments if we want to explain the different recommendations
- Don't include the generated JS in the repo by default and build it on the fly when running or deploying the site. This ensures it's always up to date.
- Simplify jinja_to_js script and use fewer dependencies
 2020-08-19 13:33:15 +02:00
Ines Montani
2285e59765
Merge pull request #5933 from svlandeg/feature/more-v3-docs [ci skip] 2020-08-19 11:29:02 +02:00
Matthew Honnibal
c0f6e77a41 Set version to v3.0.0a8 2020-08-18 23:29:00 +02:00
svlandeg
a8acedd4ba example of custom reader and batcher 2020-08-18 19:15:16 +02:00
Sofie Van Landeghem
358cbb21e3
Define candidate generator in EL config (#5876) 
* candidate generator as separate part of EL config

* update comment

* ent instead of str as input for candidate generation

* Span instead of str: correct type indication

* fix types

* unit test to create new candidate generator

* fix replace_pipe argument passing

* move error message, general cleanup

* add vocab back to KB constructor

* provide KB as callable from Vocab arg

* rename to kb_loader, fix KB serialization as part of the EL pipe

* fix typo

* reformatting

* cleanup

* fix comment

* fix wrongly duplicated code from merge conflict

* rename dump to to_disk

* from_disk instead of load_bulk

* update test after recent removal of set_morphology in tagger

* remove old doc
 2020-08-18 16:10:36 +02:00
Sofie Van Landeghem
688e77562b
Train CLI script fixes (#5931) 
* fix dash replacement in overrides arguments

* perform interpolation on training config

* make sure only .spacy files are read
 2020-08-18 16:06:37 +02:00
Ines Montani
82f0e20318 Update docs and consistency [ci skip] 2020-08-18 14:39:40 +02:00
svlandeg
10e67b400c output_file required, spacy-transformers prefered instead of required 2020-08-18 13:38:43 +02:00
Ines Montani
1c3bcfb488 Update docs and util consistency 2020-08-18 01:22:59 +02:00
Ines Montani
990c6b4c32 Update docs and CLI [ci skip] 2020-08-17 21:38:20 +02:00
Ines Montani
3ae5e02f4f Update docs, types and API consistency 2020-08-17 16:45:24 +02:00
Matthew Honnibal
a95a36ce2a Set version to v3.0.0a7 2020-08-16 15:51:05 +02:00
Ines Montani
6ae83bde0c Fix CLI consistency [ci skip] 2020-08-16 15:46:29 +02:00
Ines Montani
45f13cbf64
Merge pull request #5916 from explosion/feature/new-thinc-config 2020-08-16 15:24:12 +02:00
Ines Montani
34bda91695 Show warnings if there's nothing to auto-fill 2020-08-16 14:19:43 +02:00
Ines Montani
dd5804d499 Update type hints 2020-08-16 14:19:33 +02:00
Ines Montani
a570c304df Update quickstart, template and docs 2020-08-15 14:50:29 +02:00
Ines Montani
3272a63430
Merge pull request #5920 from explosion/fix/logging-warning-various 2020-08-15 14:41:15 +02:00
Ines Montani
fdcde9b0bf Add init fill-config 2020-08-14 16:49:26 +02:00
Matthew Honnibal
9ebf39fb5f Relax test 2020-08-14 16:31:09 +02:00
Ines Montani
8128e5eb35 Replace lexeme_norm warning with logging 2020-08-14 15:00:52 +02:00
Ines Montani
37814b608d Remove env_opt and simplfy default Optimizer 2020-08-14 14:59:54 +02:00
Ines Montani
ab1d165bba Pass optimizer defined in config to resume/begin_training 
Otherwise, this would create a default optimizer, which isn't what we want?
 2020-08-14 14:59:22 +02:00
Ines Montani
e4d0990857 Only receive from listener if listener exists 2020-08-14 14:58:48 +02:00
Ines Montani
cef97e4b63 Fix path check 2020-08-14 14:58:18 +02:00
Ines Montani
db2dbc8e59 Remove unused warning 2020-08-14 14:58:03 +02:00
Ines Montani
67cc39af7f Update Thinc and include section order 2020-08-14 14:06:22 +02:00
Ines Montani
88b0a96801 Update for new Thinc and adjust config 2020-08-13 17:38:30 +02:00
Adam Bittlingmayer
7b33b2854f
Add Armenian sentence-final verchaket, Greek question mark and Arabic question mark to default punct (#5910) 
* Add Armenian sentence-final verchaket

* Add Greek and Arabic question marks, and contributor agreement

* Check box
 2020-08-12 15:36:14 +02:00
graue70
49e690bde1
Fix typos in comments (#5904) 
* Fix typo in comment

* Fix typo

* Add spaCy Contributor Agreement
 2020-08-12 15:35:25 +02:00
graue70
ba84371ab0
Use init parameter (#5909) 2020-08-11 23:41:58 +02:00
Ines Montani
950832f087
Tidy up pipes (#5906) 
* Tidy up pipes

* Fix init, defaults and raise custom errors

* Update docs

* Update docs [ci skip]

* Apply suggestions from code review

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>

* Tidy up error handling and validation, fix consistency

* Simplify get_examples check

* Remove unused import [ci skip]

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-08-11 23:29:31 +02:00
Ines Montani
f79e4c094d Remove generic type 
Seems to cause error on Python 3.8 with Cython?
 2020-08-10 17:24:30 +02:00
Ines Montani
c099f6eece Add Token.lex 2020-08-10 16:43:52 +02:00
Ines Montani
933a7cf8d1 Fix Lexeme.from_ptr 2020-08-10 16:43:37 +02:00
Ines Montani
64f2f84098 Update docstrings and docs [ci skip] 2020-08-10 13:45:22 +02:00
Ines Montani
a4b448eec4 Remove unused compiler flag 2020-08-10 13:13:18 +02:00
Ines Montani
3eaeb73342 Tidy up and auto-format 2020-08-09 22:36:23 +02:00
Ines Montani
d5c78c7a34 Update docs and fix consistency 2020-08-09 22:31:52 +02:00
Ines Montani
7c6854d8d4 Fix missing imports 2020-08-09 22:28:29 +02:00
Matthew Honnibal
0fc13b2f14 Set version to v3.0.0a6 2020-08-09 21:53:32 +02:00
Ines Montani
a15c5fb191 Update docstrings and docs 2020-08-09 16:10:48 +02:00
Ines Montani
8d2baa153d Update tokenizer docs and add test 2020-08-09 15:24:01 +02:00