explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-09-22 11:59:14 +03:00
Code Issues Packages Projects Releases Wiki Activity
14,482 Commits 66 Branches 174 Tags 277 MiB
1eaf6e5ccb
Commit Graph

8582 Commits

Author SHA1 Message Date
Ines Montani
592678fb7d
Merge pull request #7073 from adrianeboyd/feature/logger-level-in-formatter 
Add time and level to default logging formatter
 2021-02-24 22:40:46 +11:00
Sofie Van Landeghem
0563cd73d6
Fix SpanGroup import (#7182) 
* import SpanGroup from tokens module

* revert edits from different PR

* add to __all__
 2021-02-24 21:06:16 +11:00
Sofie Van Landeghem
b92f81d5da
fix NEL config and IO, and n_sents functionality (#7100) 
* fix NEL config and IO, and n_sents functionality

* add docs

* fix test
 2021-02-22 14:49:52 +11:00
Sofie Van Landeghem
113e8d082b
only evaluate named entities for NEL if there is a corresponding gold span (#7074) 2021-02-22 11:06:50 +11:00
Adriane Boyd
264862c67a
Fix Ukrainian lemmatizer init (#7127) 
Fix class variable and init for `UkrainianLemmatizer` so that it loads
the `uk` dictionaries rather than having the parent `RussianLemmatizer`
override with the `ru` settings.
 2021-02-22 11:05:08 +11:00
Sofie Van Landeghem
ba5a50f62b
NEL docs & UX (#7129) 
* EL set_kb docs fix

* custom warning for set_kb mistake
 2021-02-22 11:04:22 +11:00
Boian Tzonev
cca8651fc8
Bulgarian tokenizer exceptions (#7114) 
* [Bulgarian] Add tokenizer exceptions and like_num for Bulgarian

* [Bulgarian] Add tokenizer exceptions and like_num for Bulgarian
 2021-02-19 19:19:19 +01:00
Sofie Van Landeghem
709c9e75af
span.ent only returns first sentence (#7084) 
* return first sentence when span contains sentence boundary

* docs fix

* small fixes

* cleanup
 2021-02-19 23:02:38 +11:00
Adriane Boyd
30e1a89aeb
Fix displacy output in evaluate CLI (#7122) 
Now that `nlp.evaluate()` does not modify the examples, rerun the
pipeline on the (limited) texts in order to provide the predicted
annotation in the displacy output option.
 2021-02-19 23:01:20 +11:00
Adriane Boyd
4188beda87
Fix conll converter option (#7071) 
Map `conll` to the NER converter, not the `CoNLL-U` converter.
 2021-02-18 10:22:41 +01:00
Adriane Boyd
a3293efc48 Add time and level to default logging formatter 2021-02-15 14:19:20 +01:00
Ines Montani
1e3a326e53 Change Dutch transformer recommendation [ci skip] 
https://github.com/explosion/spaCy/discussions/6529#discussioncomment-366620
 2021-02-14 15:30:16 +11:00
Ines Montani
f4f46b617f
Preserve sourced components in fill-config (fixes #7055) (#7058) 2021-02-14 14:02:14 +11:00
Matthew Honnibal
0fb8d437c0
Fix sentence fragments bug (#7056, #7035) (#7057) 
* Add test for #7035

* Update test for issue 7056

* Fix test

* Fix transitions method used in testing

* Fix state eol detection when rebuffer

* Clean up redundant fix
 2021-02-14 13:38:13 +11:00
Ines Montani
660642902a Increment version [ci skip] 2021-02-14 13:36:13 +11:00
Matthew Honnibal
b31471b5b8 Set version to v3.0.2 2021-02-13 23:50:00 +11:00
Ines Montani
9ba715ed16 Tidy up and auto-format 2021-02-13 12:55:56 +11:00
Ines Montani
34ee0fbd70
Merge pull request #7011 from Shumie82/master 2021-02-13 12:30:42 +11:00
Ines Montani
e583050547
Merge pull request #7039 from svlandeg/debug 2021-02-13 11:53:41 +11:00
Ines Montani
6c450decfc Fix punctuation settings and add to initialize tests 2021-02-13 11:51:21 +11:00
Ines Montani
f4712a634e
Merge pull request #7046 from adrianeboyd/bugfix/vocab-pickle-noun-chunks-6891 
Include noun chunks method when pickling Vocab
 2021-02-13 11:43:03 +11:00
Adriane Boyd
0ee2ae86bf Update trf quickstart recommendations 
Add/update trf recommendations for Bengali, Hindi, Sinhala, and Tamil
based on #7044.
 2021-02-12 15:55:17 +01:00
svlandeg
03b4ec7d7f fix typo 2021-02-12 14:30:16 +01:00
Adriane Boyd
5e47a54d29 Include noun chunks method when pickling Vocab 2021-02-12 13:27:46 +01:00
svlandeg
aa3ad8825d loop instead of any 2021-02-12 13:14:30 +01:00
svlandeg
278e9eaa14 remove ner 2021-02-11 21:08:04 +01:00
svlandeg
ebeedfc70b regression test for 7029 2021-02-11 20:56:48 +01:00
svlandeg
a52d466bfc any instead of all 2021-02-11 20:50:55 +01:00
Shumi
4e514f1ea8
Update stop_words.py 
I have deleted line 1 to 5 and the statement print(STOP_WORDS)
 2021-02-11 21:30:34 +02:00
Shumi
0d57e84b7b
Update lex_attrs.py 
I have removed line 1 to 4
 2021-02-11 21:28:23 +02:00
Shumi
37ec67f868
Update examples.py 
I have removed two lines:
# coding: utf8
from __future__ import unicode_literals

And updated: >>> from spacy.lang.tn.examples import sentences
 2021-02-11 21:25:58 +02:00
Shumi
39eeba6760
Update __init__.py 
Added infixes = TOKENIZER_INFIXES
 2021-02-11 21:20:46 +02:00
Ines Montani
26bf642afd
Fix issue #7019: Handle None scores in evaluate printer (#7026) 2021-02-11 16:45:23 +11:00
Ines Montani
6b9026a219
Merge pull request #7000 from explosion/feature/project-yml-overrides 
Support env vars and CLI overrides for project.yml
 2021-02-11 12:31:45 +11:00
Ines Montani
ad9ce3c8f6 Fix issue #6950: allow pickling Tok2Vec with listeners 2021-02-11 11:37:39 +11:00
Shumi
ed3397727e
Delete tag_map.py 
Tag map file is deleted. I will add it later because it was failing validations
 2021-02-10 20:41:18 +02:00
Shumi
7c8721b1bd
Update tag_map.py 
Updated tag_map
 2021-02-10 20:21:22 +02:00
Shumi
f6be28cfb2
Added files to Setswana Language 
Add South African Setswana Language
 2021-02-10 20:15:13 +02:00
Shumi
24046fef17
South African Setswana language 
Please accept the additional of Setswana language
 2021-02-10 20:12:33 +02:00
Peter Baumann
61b04a70d5
Run PhraseMatcher on Spans (#6918) 
* Add regression test

* Run PhraseMatcher on Spans

* Add test for PhraseMatcher on Spans and Docs

* Add SCA

* Add test with 3 matches in Doc, 1 match in Span

* Update docs

* Use doc.length for find_matches in tokenizer

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2021-02-10 23:43:32 +11:00
Ines Montani
21176c69b0 Update and add test 2021-02-10 14:12:00 +11:00
Ines Montani
c08b3f294c Support env vars and CLI overrides for project.yml 2021-02-10 13:45:27 +11:00
Koichi Yasuoka
8ed788660b
Several callable objects do not have __qualname__ 2021-02-09 14:43:02 +09:00
Adriane Boyd
6108dabdc8 Rephrase error related to sample data initialization 
Now that the initialize step is fully implemented, the source of E923 is
typically missing or improperly converted/formatted data rather than a
bug in spaCy, so rephrase the error and message and remove the prompt to
open an issue.
 2021-02-08 09:21:36 +01:00
Sofie Van Landeghem
6ed423c16c
reduce memory load when reading all vectors from file (#6945) 
* reduce memory load when reading all vectors from file

* one more small typo fix
 2021-02-07 08:05:43 +08:00
Sofie Van Landeghem
a323ef90df
ensure the loss value is cast as float (#6928) 2021-02-07 07:51:56 +08:00
melonwater211
a7977b5143
The test spacy/tests/vocab_vectors/test_lexeme.py::test_vocab_lexeme_add_flag_auto_id seems to fail occasionally when the test suite is run in a random order. (#6956) 
```python
    def test_vocab_lexeme_add_flag_auto_id(en_vocab):
        is_len4 = en_vocab.add_flag(lambda string: len(string) == 4)
        assert en_vocab["1999"].check_flag(is_len4) is True
        assert en_vocab["1999"].check_flag(IS_DIGIT) is True
        assert en_vocab["199"].check_flag(is_len4) is False
>       assert en_vocab["199"].check_flag(IS_DIGIT) is True
E       assert False is True
E        +  where False = <built-in method check_flag of spacy.lexeme.Lexeme object at 0x7fa155c36840>(3)
E        +    where <built-in method check_flag of spacy.lexeme.Lexeme object at 0x7fa155c36840> = <spacy.lexeme.Lexeme object at 0x7fa155c36840>.check_flag

spacy/tests/vocab_vectors/test_lexeme.py:49: AssertionError
```

>  `pytest==6.1.1`
>
>  `numpy==1.19.2`
>
> `Python version: 3.8.3`

To reproduce the error, run `pytest --random-order-bucket=global --random-order-seed=170158 -v spacy/tests`

If `test_vocab_lexeme_add_flag_auto_id` is run after `test_vocab_lexeme_add_flag_provided_id`, it fails.
It seems like `test_vocab_lexeme_add_flag_provided_id` uses the `IS_DIGIT` bit for testing purposes but does not reset the bit.

This solution seems to work but, if anyone has a better fix, please let me know and I will integrate it.
 2021-02-07 07:51:34 +08:00
René Octavio Queiroz Dias
59271e887a
fix: TransformerListener with TextCatEnsemble (#6951) 
* bug: Regression test
Issue #6946

* fix: Fix issue #6946

* chore: Remove regression test
 2021-02-06 13:44:51 +01:00
René Octavio Queiroz Dias
999ff03b19
fix: Fix textcat labels to expect a Optional[Iterable[str]] instead of Optional[Dict] (#6911) 
* docs: Add agreement

* bug: Regression test

Issue #6908

* fix: Changed from Dict to Iterable[str]

Fix #6908

* Update test to use make_tempdir

* fix: Fix WindowsPath error

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2021-02-04 23:37:13 +01:00
Adriane Boyd
b903de3fcb
Pass on vocab arg in spacy.blank() (#6924) 2021-02-04 15:09:01 +01:00
svlandeg
f852af2acf add capture arg 2021-02-02 19:47:12 +01:00
Matthew Honnibal
b6a198481b Set version to v3.0.0 2021-02-02 20:26:17 +11:00
Sofie Van Landeghem
f319d2765f
Add capture argument to project_run (#6878) 
* add capture argument to project_run and run_commands

* git bump to 3.0.1

* Set version to 3.0.1.dev0

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2021-02-02 10:11:15 +08:00
Sofie Van Landeghem
f638306598
remove link_components flag again (#6883) 2021-02-02 10:08:40 +08:00
Ines Montani
a59f3fcf5d Make wheel the default format and update docs [ci skip] 2021-02-01 23:18:43 +11:00
Ines Montani
b9573e9e22 Fix pip args 2021-02-01 23:15:00 +11:00
Ines Montani
b46073234a Fix default clone branch and error handling [ci skip] 2021-02-01 22:29:04 +11:00
Sofie Van Landeghem
acabb284dd
Fix linking resumed components (#6859) 
* link components across enabled, resumed and frozen

* revert renaming

* revert renaming, the sequel
 2021-02-01 22:19:58 +11:00
Adriane Boyd
35a863cd27 Remove nlp.tokenizer from quickstart template 
Remove `nlp.tokenizer` from quickstart template so that the default
language-specific tokenizer settings are filled instead.
 2021-02-01 11:20:12 +01:00
svlandeg
91e72c031e reformatting 2021-01-30 17:29:33 +01:00
svlandeg
a8d84188f0 add stop words 
Co-authored-by: tewodrosm <tedmaam2006@gmail.com>
 2021-01-30 17:26:49 +01:00
Ines Montani
f058cbd751 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2021-01-30 21:03:25 +11:00
Ines Montani
14f631f52c Update parent package and version [ci skip] 2021-01-30 20:12:42 +11:00
Ines Montani
3435b894df Remove nightly reference from auto docs [ci skip] 2021-01-30 20:12:08 +11:00
Ines Montani
d0c3775712 Replace links to nightly docs [ci skip] 2021-01-30 20:09:38 +11:00
Ines Montani
b26a3daa9a
Merge pull request #6860 from explosion/feature/package-wheel 2021-01-30 14:17:01 +11:00
Ines Montani
2332c4280b Update and use unified --build option 2021-01-30 13:11:36 +11:00
Ines Montani
e6accb3a9e Tidy up and auto-format 2021-01-30 12:52:33 +11:00
Ines Montani
817b0db521 Fix escape sequence 2021-01-30 12:39:58 +11:00
Ines Montani
526b416118 Tidy up comments 2021-01-30 12:34:09 +11:00
Ines Montani
30765674d0 Merge branch 'master' into develop 2021-01-30 12:20:28 +11:00
Ines Montani
2609ba4e89 Support building wheel in spacy package 2021-01-30 11:54:02 +11:00
Pamphile ROY
41ee75ac6d
Remove --no-cache-dir when downloading models 
When `--no-cache-dir` is present, it prevents caching to properly function.

If the user still wants to do this, there is the possibility to pass options with `user_pip_args`.
But you should not enforce options like these. In my case this is preventing some docker build (using buildkit caching) to have proper caching of models.
 2021-01-29 15:37:44 +01:00
Ines Montani
bbf080dfe5
Merge pull request #6645 from bittlingmayer/patch-3 2021-01-30 01:26:28 +11:00
Adriane Boyd
bced6309e5
Add full exceptions with spaces 2021-01-29 14:27:22 +01:00
Ines Montani
7886d59c56 Add check for remove_listener method 2021-01-29 23:47:30 +11:00
Ines Montani
7694f76dd1 Update warning and mention replace_listeners 2021-01-29 23:46:01 +11:00
Ines Montani
94232aea08 Improve E889 2021-01-29 23:39:23 +11:00
Ines Montani
924396c20c Merge branch 'feature/replace-listeners' of https://github.com/explosion/spaCy into feature/replace-listeners 2021-01-29 21:43:10 +11:00
Ines Montani
2102082478 Make Tok2Vec.remove_listener return bool 
Whether listener was removed
 2021-01-29 21:41:38 +11:00
Ines Montani
e766e8c56d
Apply suggestions from code review 
Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
 2021-01-29 21:41:17 +11:00
Ines Montani
bc089b693c Update tests 2021-01-29 19:38:09 +11:00
Ines Montani
325f47500d Move replacement logic to Language.from_config 2021-01-29 19:37:04 +11:00
Ines Montani
0f3e3eedc2 Add Tok2vec.remove_listener 2021-01-29 19:36:38 +11:00
Ines Montani
99842387cb Remove default value 2021-01-29 18:45:37 +11:00
Ines Montani
44b5542d14 Change method order 2021-01-29 18:42:41 +11:00
Ines Montani
8c15d1daec Update and validate config first and exit early if paths don't exist 2021-01-29 18:24:47 +11:00
Ines Montani
bbb94b37c6 Update error handling and docstring 2021-01-29 16:27:49 +11:00
Ines Montani
01ecfbcc45 Merge branch 'develop' into feature/replace-listeners 2021-01-29 15:57:32 +11:00
Ines Montani
911dfcccfc Add option to replace listeners for sourced components 2021-01-29 15:57:04 +11:00
Adriane Boyd
fcce3600ed
Forbid OP matching 2+ tokens in DependencyMatcher (#6824) 
Instead of silently using only the first token in each matched span:

* Forbid `OP: ?/*/+` through `DependencyMatcher` validation
* As a fail-safe, add warning if a token match that's not exactly one
token long is found by a token pattern.
 2021-01-29 08:52:01 +08:00
Sofie Van Landeghem
24a697abb8
avoid empty aliases and improve UX and docs (#6840) 2021-01-29 08:51:40 +08:00
Sofie Van Landeghem
837a4f53c2
Error handling in nlp.pipe (#6817) 
* add error handler for pipe methods

* add unit tests

* remove pipe method that are the same as their base class

* have Language keep track of a default error handler

* cleanup

* formatting

* small refactor

* add documentation
 2021-01-29 08:51:21 +08:00
Ines Montani
cc18f3f23c
Improve Example error handling for NER data (#6835) 
* Improve Example error handling for NER data

* Fix conditional
 2021-01-28 13:11:20 +11:00
Ines Montani
78d6ff4dd4 Update quickstart recommendations 2021-01-28 11:14:49 +11:00
Ines Montani
ec5f55aa5b
Update config generation defaults and transformers (#6832) 2021-01-27 23:56:33 +11:00
Adriane Boyd
4096a79de7
Add alignment mode error and fix Doc.char_span docs (#6820) 
* Raise an error on an unrecognized alignment mode rather than
defaulting to `strict`
* Fix the `Doc.char_span` API doc alignment mode details
 2021-01-27 23:40:42 +11:00
Sofie Van Landeghem
6b68ad027b
Fix beam NER resizing (#6834) 
* move label check to sub methods

* add tests
 2021-01-27 23:39:14 +11:00
Ines Montani
5ed51c9dd2
Merge pull request #6828 from explosion/master-tmp 2021-01-27 23:05:46 +11:00
Adriane Boyd
d17afb4826
Add Spanish rule-based lemmatizer (#6833) 
* Initial Spanish lemmatizer

* Handle merged verb+pron(s) multi-word tokens

* Use VERB for AUX rule lookup

* Add morph to lemma cache key

* Fix aux lookups, minor refactoring

* Improve verb+pron handling

* Move verb+pron handling into its own method
* Check for exceptions (primarily for se)
* Collect pronouns in the same (not reversed) order

* Only add modified possible lemmas
 2021-01-27 19:21:35 +08:00