1
1
mirror of https://github.com/explosion/spaCy.git synced 2025-02-24 15:47:33 +03:00
Commit Graph

9141 Commits

Author SHA1 Message Date
Peter Baumgartner
a944e55291 partial annotation support 2023-01-12 09:10:39 -05:00
Peter Baumgartner
28c31048c9 use 0 instead of none for no annotation 2023-01-11 14:47:10 -05:00
Peter Baumgartner
bfb9fa44a0 additional fixes
- set approach to identifying unique trees
- adjust line length on messages
- add logic for detecting docs without annotations
2023-01-11 14:36:28 -05:00
Peter Baumgartner
c53d57a54e fix reference dataset for dev data 2022-12-01 14:01:10 -05:00
Peter Baumgartner
73bedaa9ec rm component function 2022-09-08 13:53:16 -04:00
Peter Baumgartner
5c3337b81b cleanup + reword 2022-09-08 11:57:15 -04:00
Peter Baumgartner
c09d99a069 cleanup 2022-09-07 10:30:58 -04:00
Peter Baumgartner
64b57204e2 WIP 2022-09-01 13:00:18 -04:00
Peter Baumgartner
19145c6c92 rm total 2022-08-31 14:22:17 -04:00
Peter Baumgartner
0bd7c095e3 rm ipython embeds 2022-08-31 14:06:13 -04:00
Peter Baumgartner
6007404682 WIP 2022-08-31 14:00:43 -04:00
Edward
6723d76f24
Add ConsoleLogger.v2 ()
* Init

* Change logger to ConsoleLogger.v2

* adjust naming

* More naming adjustments

* Fix output_file reference error

* ignore type

* Add basic test for logger

* Hopefully fix mypy issue

* mypy ignore line

* Update mypy line

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Update test method name

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Change file saving logic

* Fix finalize method

* increase spacy-legacy version in requirements

* Update docs

* small adjustments

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-08-29 10:23:05 +02:00
Tobius Saul
c09d2fa25b
luganda language extension ()
* luganda language extension

* __init__.py changes

* New enhancements

* Lexical attribute changed

* punctuaction and sentence additions

* Remove comment header

* Fix typos, reformat

* reformated version

* Add tokenizer test

* Remove contractions from stop words

* Format

* Add Luganda to website

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-08-23 13:09:36 +02:00
Edward
5afa98aabf
Support custom attributes for tokens and spans in json conversion ()
* Add token and span custom attributes to to_json()

* Change logic for to_json

* Add functionality to from_json

* Small adjustments

* Move token/span attributes to new dict key

* Fix test

* Fix the same test but much better

* Add backwards compatibility tests and adjust logic

* Add test to check if attributes not set in underscore are not saved in the json

* Add tests for json compatibility

* Adjust test names

* Fix tests and clean up code

* Fix assert json tests

* small adjustment

* adjust naming and code readability

* Adjust naming, added more tests and changed logic

* Fix typo

* Adjust errors, naming, and small test optimization

* Fix byte tests

* Fix bytes tests

* Change naming and json structure

* update schema

* Update spacy/schemas.py

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Update spacy/tokens/doc.pyx

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Update spacy/tokens/doc.pyx

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Update spacy/schemas.py

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Update schema for underscore attributes

* Adjust underscore schema

* adjust schema tests

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-08-23 10:05:02 +02:00
Adriane Boyd
f55bb7470d
Clean up warnings in the test suite () 2022-08-22 12:04:30 +02:00
Paul O'Leary McCann
0f07defe2c
Remove reference to voting on issue ()
Not clear which issue this refers to, we don't suggest this for any
other issues, and we don't use votes in general.
2022-08-22 11:29:05 +02:00
Adriane Boyd
3e4cf1bbe1
Check for . in factory names () 2022-08-19 09:52:12 +02:00
Sofie Van Landeghem
cab263791f
include span_ruler for default warning filter () 2022-08-17 19:55:54 +02:00
Peter Baumgartner
db7b9938a4
Docs: displaCy documentation - data types, parse_{deps,ents,spans}, spans example ()
* add in spans example and parse references

* rm autoformatter

* rm extra ents copy

* TypedDict draft

* type fixes

* restore non-documentation files

* docs update

* fix spans example

* fix hyperlinks

* add parse example

* example fix + argument fix

* fix api arg in docs

* fix bad variable replacement

* fix spacing in style

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* fix spacing on table

* fix spacing on table

* rm temp files

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-08-16 11:23:34 -04:00
Adriane Boyd
ed4ad309e6
Fix Dutch noun chunks to skip overlapping spans ()
* Add test for overlapping noun chunks

* Skip overlapping noun chunks

* Update spacy/tests/lang/nl/test_noun_chunks.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-08-10 09:49:08 +02:00
Adriane Boyd
fc4246558b
Fix regex invalid escape sequences () 2022-08-09 10:59:36 +02:00
stefawolf
23749cfc91
adding spans to doc_annotation in Example.to_dict ()
* adding spans to doc_annotation in Example.to_dict

* to_dict compatible with from_dict: tuples instead of spans

* use strings for label and kb_id

* Simplify test

* Update data formats docs

Co-authored-by: Stefanie Wolf <stefanie.wolf@vitecsoftware.com>
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-08-05 12:26:38 +02:00
Adriane Boyd
b07708d5d0
Support full prerelease versions in the compat table ()
* Support full prerelease versions in the compat table

* Fix types
2022-08-04 15:14:19 +02:00
ninjalu
95a1b8aca6
add additional REL_OP ()
* add additional  REL_OP

* change to condition and new rel_op symbols

* add operators to docs

* add the anchor while we're in here

* add tests

Co-authored-by: Peter Baumgartner <5107405+pmbaumgartner@users.noreply.github.com>
2022-07-27 13:16:44 +02:00
Edward
360a702ecd
Add parent argument () 2022-07-26 14:35:18 +02:00
Adriane Boyd
5c2a00cef0
Set version to v3.4.1 () 2022-07-26 12:52:38 +02:00
Daniël de Kok
4ee8a06149
Fix compatibility with CuPy 9.x ()
After the precomputable affine table of shape [nB, nF, nO, nP] is
computed, padding with shape [1, nF, nO, nP] is assigned to the first
row of the precomputed affine table. However, when we are indexing the
precomputed table, we get a row of shape [nF, nO, nP]. CuPy versions
before 10.0 cannot paper over this shape difference.

This change fixes compatibility with CuPy < 10.0 by squeezing the first
dimension of the padding before assignment.
2022-07-26 10:52:01 +02:00
Adriane Boyd
e5990db713 Revert "Temporarily skip tests that require models/compat"
This reverts commit d9320db7db.
2022-07-25 18:12:18 +02:00
Nicolai Bjerre Pedersen
2fa983aa2e
Fix span typings ()
Add id, id_ to span.pyi.
2022-07-12 13:47:35 +02:00
Peter Baumgartner
36cb2029a9
displaCy Spans Vertical Alignment Fix 2 ()
* add in span render slot fix

* fix spacing off by one

* rm demo

* adjust comments

* fix whitespace and overlap issue
2022-07-08 19:20:13 +02:00
github-actions[bot]
e7fd06bdbe
Auto-format code with black ()
Co-authored-by: explosion-bot <explosion-bot@users.noreply.github.com>
2022-07-08 18:43:25 +09:00
Daniël de Kok
a06cbae70d
precompute_hiddens/Parser: do not look up CPU ops (3.4) ()
* precompute_hiddens/Parser: do not look up CPU ops

`get_ops("cpu")` is quite expensive. To avoid this, we want to cache the
result as in . However, for 3.x we do not want to change the ABI.
So we avoid the expensive lookup by using NumpyOps. This should have a
minimal impact, since `get_ops("cpu")` was only used when the model ops
were `CupyOps`. If the ops are `AppleOps`, we are still passing through
the correct BLAS implementation.

* _NUMPY_OPS -> NUMPY_OPS
2022-07-05 10:53:42 +02:00
Madeesh Kannan
d36d66b7ca
Increase test deadline to 30 minutes to prevent spurious test failures ()
* Increase test deadline to 30 minutes to prevent spurious test failures

* Reduce deadline to 2 minutes
2022-07-04 18:37:09 +02:00
kadarakos
5240baccfe
dont use get_array_module () 2022-07-04 17:15:33 +02:00
Raphael Mitsch
e9eb59699f
NEL confidence threshold ()
* Add base for NEL abstention threshold mechanism.

* Add abstention threshold to entity linker. Add test.

* Fix entity linking tests.

* Changed abstention default threshold from 0 to None.

* Fix default values for abstention thresholds.

* Fix mypy errors.

* Replace assertion with raise of proper error code.

* Simplify threshold check. Remove thresholding from EntityLinker_v1.

* Rename test.

* Update spacy/pipeline/entity_linker.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/pipeline/entity_linker.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Make E1043 configurable.

* Update docs.

* Rephrase description in docs. Adjusting error code message.

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-07-04 17:05:21 +02:00
Madeesh Kannan
59c763eec1
StringStore-related optimizations ()
* `strings`: More roubust type checking of keys/IDs, coerce `int`-like types to `hash_t`

* Preserve existing public API behaviour

* Fix return type

* Replace `bool` with `bint`, rename to `_try_coerce_to_hash`, replace `id` with `hash`

* Avoid unnecessary re-encoding and re-calculation of strings and hashs respectively

* Rename variables named `hash`
Add comment on early return
2022-07-04 15:04:03 +02:00
explosion-bot
7e55a51314 Auto-format code with black 2022-07-01 08:04:32 +00:00
Madeesh Kannan
eaf66e7431
Add NVTX ranges to TrainablePipe components ()
* `TrainablePipe`: Add NVTX range decorator

* Annotate `TrainablePipe` subclasses with NVTX ranges

* Export function signature to allow introspection of args in tests

* Revert "Annotate `TrainablePipe` subclasses with NVTX ranges"

This reverts commit d8684f7372.

* Revert "Export function signature to allow introspection of args in tests"

This reverts commit f4405ca3ad.

* Revert "`TrainablePipe`: Add NVTX range decorator"

This reverts commit 26536eb6b8.

* Add `spacy.pipes_with_nvtx_range` pipeline callback

* Show warnings for all missing user-defined pipe functions that need to be annotated
Fix imports, typos

* Rename `DEFAULT_ANNOTATABLE_PIPE_METHODS` to `DEFAULT_NVTX_ANNOTATABLE_PIPE_METHODS`
Reorder import

* Walk model nodes directly whilst applying NVTX ranges
Ignore pipe method wrapper when applying range
2022-06-30 11:28:12 +02:00
Adriane Boyd
3fe9f47de4
Revert "disable failing test because Stanford servers are down ()" ()
This reverts commit f8116078ce.
2022-06-30 11:24:54 +02:00
Shen Qin
be00db6645
Addition of min_max quantifier in matcher {n,m} ()
* Min_max_operators
1. Modified API and Usage for spaCy website to include min_max operator
2. Modified matcher.pyx to include min_max function {n,m} and its variants
3. Modified schemas.py to include min_max validation error
4. Added test cases to test_matcher_api.py, test_matcher_logic.py and test_pattern_validation.py

* attempt to fix mypy/pydantic compat issue

* formatting

* Update spacy/tests/matcher/test_pattern_validation.py

Co-authored-by: Source-Shen <82353723+Source-Shen@users.noreply.github.com>
Co-authored-by: svlandeg <svlandeg@github.com>
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-06-30 11:01:58 +02:00
Daniël de Kok
0ff14aabce
vectors: avoid expensive comparisons between numpy ints and Python ints ()
* vectors: avoid expensive comparisons between numpy ints and Python ints

* vectors: avoid failure on lists of ints

* Convert another numpy int to Python
2022-06-29 12:58:31 +02:00
Peter Baumgartner
dd038b536c
fix to horizontal space () 2022-06-28 20:42:40 +02:00
Adriane Boyd
24f4908fce
Update vector handling in similarity methods ()
Distinguish between vectors that are 0 vs. missing vectors when warning
about missing vectors.

Update `Doc.has_vector` to match `Span.has_vector` and
`Token.has_vector` for cases where the vocab has vectors but none of the
tokens in the container have vectors.
2022-06-28 19:50:47 +02:00
Madeesh Kannan
1d5cad0b42
Example.get_aligned_parse: Handle unit and zero length vectors correctly ()
* `Example.get_aligned_parse`: Do not squeeze gold token idx vector
Correctly handle zero-size vectors passed to `np.vectorize`

* Add tests

* Use `Doc` ctor to initialize attributes

* Remove unintended change

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>

* Remove unused import

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-06-28 19:42:58 +02:00
Richard Hudson
a9559e7435
Handle Cyrillic combining diacritics ()
* Handle Russian, Ukrainian and Bulgarian

* Corrections

* Correction

* Correction to comment

* Changes based on review

* Correction

* Reverted irrelevant change in punctuation.py

* Remove unnecessary group

* Reverted accidental change
2022-06-28 15:35:32 +02:00
Zackere
8ffff18ac4
Try cloning repo from main & master ()
* Try cloning repo from main & master

* fixup! Try cloning repo from main & master

* fixup! fixup! Try cloning repo from main & master

* refactor clone and check for repo:branch existence

* spacing fix

* make mypy happy

* type util function

* Update spacy/cli/project/clone.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

Co-authored-by: Peter Baumgartner <5107405+pmbaumgartner@users.noreply.github.com>
Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-06-28 09:11:15 -04:00
github-actions[bot]
4155a59d47
Auto-format code with black ()
Co-authored-by: explosion-bot <explosion-bot@users.noreply.github.com>
2022-06-27 09:35:35 +02:00
Adriane Boyd
738b38064f
Merge pull request from adrianeboyd/chore/v3.4.0
Set version to v3.4.0
2022-06-24 14:54:16 +02:00
Madeesh Kannan
8f1ba4de58
Backport parser/alignment optimizations from feature/refactor-parser () 2022-06-24 13:39:52 +02:00
Adriane Boyd
d9320db7db Temporarily skip tests that require models/compat 2022-06-24 11:20:53 +02:00