Commit Graph

9315 Commits

Author SHA1 Message Date
Paul O'Leary McCann
2e9dadfda4 Remove orphaned function
This was probably used in the prototyping stage, left as a reference,
and then forgotten. Nothing uses it any more.
2022-07-12 16:06:15 +09:00
Paul O'Leary McCann
07e8556cc3 Remove config from coref tests
This was necessary when the tok2vec_size option was necessary.
2022-07-12 14:08:35 +09:00
Paul O'Leary McCann
1baa334b8a Make get_clusters_from_doc return spans in order
There's no guarantee about the order in which SpanGroup keys will come
out, so access them in sorted order when doing comparisons.
2022-07-12 14:07:40 +09:00
Paul O'Leary McCann
64a0bf4460
Merge branch 'feature/coref' into coref/dimension-inference 2022-07-12 12:56:10 +09:00
Paul O'Leary McCann
0f3c45646b Update error number
This was changed by merge
2022-07-11 20:17:03 +09:00
Paul O'Leary McCann
7792229fa9 Merge branch 'master' into feature/coref 2022-07-11 20:16:23 +09:00
Paul O'Leary McCann
f9c82e249c Update error number
This was changed by merge
2022-07-11 20:14:36 +09:00
Paul O'Leary McCann
5969634e92 Merge branch 'master' into coref/dimension-inference 2022-07-11 20:11:51 +09:00
Paul O'Leary McCann
baeb35f31b Add type annotations for internal models 2022-07-11 20:03:29 +09:00
Paul O'Leary McCann
4d032396b8 Merge branch 'feature/coref' into coref/dimension-inference 2022-07-11 19:18:46 +09:00
Paul O'Leary McCann
6d9eafeb37
Merge branch 'feature/coref' into fix/coref-alignment 2022-07-11 19:14:37 +09:00
Paul O'Leary McCann
1b3db149df Merge branch 'fix/coref-alignment' into feature/coref 2022-07-11 19:12:03 +09:00
Peter Baumgartner
36cb2029a9
displaCy Spans Vertical Alignment Fix 2 (#11092)
* add in span render slot fix

* fix spacing off by one

* rm demo

* adjust comments

* fix whitespace and overlap issue
2022-07-08 19:20:13 +02:00
github-actions[bot]
e7fd06bdbe
Auto-format code with black (#11099)
Co-authored-by: explosion-bot <explosion-bot@users.noreply.github.com>
2022-07-08 18:43:25 +09:00
Paul O'Leary McCann
2eee0d248e Fix types
mypy now exits without an error, except for two apparently unrelated
ones about setup.py.
2022-07-08 18:29:14 +09:00
Paul O'Leary McCann
da81a90d64 Span predictor leftovers 2022-07-06 19:29:27 +09:00
Paul O'Leary McCann
b0800ea855 Do dimension inference in span predictor 2022-07-06 19:22:37 +09:00
Paul O'Leary McCann
b59b924e49 Use normal PyTorchWrapper in coref 2022-07-06 19:22:19 +09:00
Paul O'Leary McCann
f67c1735c5 Remove tok2vec_size from coref 2022-07-06 18:58:57 +09:00
Paul O'Leary McCann
bd17c38b74 It works!
Was missing the serialization-related code from biaffine.
2022-07-06 18:58:22 +09:00
Paul O'Leary McCann
ba1bf8ae72 First take at dimension inference
This follows the pattern used in the Biaffine Parser, which uses an init
function to get the size only after the tok2vec is available.

This works at first, but serialization fails with an error.
2022-07-06 18:40:05 +09:00
Paul O'Leary McCann
ce49136458 Update NotImplementedError for coref component 2022-07-06 17:28:15 +09:00
Paul O'Leary McCann
5e405738d2 Update span predictor docstrings 2022-07-06 17:28:05 +09:00
Paul O'Leary McCann
c4de3e51a2 Remove old TODOs 2022-07-06 17:23:41 +09:00
Paul O'Leary McCann
da9c379355 Update docs
Parameter names in architecture docs were not updated after parameters
were renamed.
2022-07-06 17:13:31 +09:00
Paul O'Leary McCann
6f5cf838ec Remove _spans_to_offsets
Basically the same as get_clusters_from_doc
2022-07-06 14:05:05 +09:00
Paul O'Leary McCann
8f598d7b01 Feedback from code review 2022-07-06 14:03:09 +09:00
Paul O'Leary McCann
63e27b5e44
Update spacy/ml/models/coref_util.py
Co-authored-by: kadarakos <kadar.akos@gmail.com>
2022-07-06 13:46:02 +09:00
Daniël de Kok
a06cbae70d
precompute_hiddens/Parser: do not look up CPU ops (3.4) (#11069)
* precompute_hiddens/Parser: do not look up CPU ops

`get_ops("cpu")` is quite expensive. To avoid this, we want to cache the
result as in #11068. However, for 3.x we do not want to change the ABI.
So we avoid the expensive lookup by using NumpyOps. This should have a
minimal impact, since `get_ops("cpu")` was only used when the model ops
were `CupyOps`. If the ops are `AppleOps`, we are still passing through
the correct BLAS implementation.

* _NUMPY_OPS -> NUMPY_OPS
2022-07-05 10:53:42 +02:00
Madeesh Kannan
d36d66b7ca
Increase test deadline to 30 minutes to prevent spurious test failures (#11070)
* Increase test deadline to 30 minutes to prevent spurious test failures

* Reduce deadline to 2 minutes
2022-07-04 18:37:09 +02:00
kadarakos
5240baccfe
dont use get_array_module (#11056) 2022-07-04 17:15:33 +02:00
Raphael Mitsch
e9eb59699f
NEL confidence threshold (#11016)
* Add base for NEL abstention threshold mechanism.

* Add abstention threshold to entity linker. Add test.

* Fix entity linking tests.

* Changed abstention default threshold from 0 to None.

* Fix default values for abstention thresholds.

* Fix mypy errors.

* Replace assertion with raise of proper error code.

* Simplify threshold check. Remove thresholding from EntityLinker_v1.

* Rename test.

* Update spacy/pipeline/entity_linker.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Update spacy/pipeline/entity_linker.py

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>

* Make E1043 configurable.

* Update docs.

* Rephrase description in docs. Adjusting error code message.

Co-authored-by: Sofie Van Landeghem <svlandeg@users.noreply.github.com>
2022-07-04 17:05:21 +02:00
Madeesh Kannan
59c763eec1
StringStore-related optimizations (#10938)
* `strings`: More roubust type checking of keys/IDs, coerce `int`-like types to `hash_t`

* Preserve existing public API behaviour

* Fix return type

* Replace `bool` with `bint`, rename to `_try_coerce_to_hash`, replace `id` with `hash`

* Avoid unnecessary re-encoding and re-calculation of strings and hashs respectively

* Rename variables named `hash`
Add comment on early return
2022-07-04 15:04:03 +02:00
Paul O'Leary McCann
178feae00a Add tests to give up with whitespace differences
Docs in Examples are allowed to have arbitrarily different whitespace.
Handling that properly would be nice but isn't required, but for now
check for it and blow up.
2022-07-04 19:37:42 +09:00
Paul O'Leary McCann
c7f333d593 Rename spans2ints > _spans_to_offsets 2022-07-04 19:28:35 +09:00
Paul O'Leary McCann
b09bbc7f5e Fix alignment issues
I believe this resolves issues with tokenization mismatches.
2022-07-03 20:11:03 +09:00
Paul O'Leary McCann
cf33b48fe0 Update tests 2022-07-03 20:10:53 +09:00
Paul O'Leary McCann
fd574a89c4 Update overfitting test 2022-07-03 19:34:15 +09:00
Paul O'Leary McCann
a46bc03abb Add failing test with tokenization mismatch
This test only fails due to the explicity assert False at the moment,
but the debug output shows that the learned spans are all off by one due
to misalignment. So the code still needs fixing.
2022-07-03 16:01:27 +09:00
Paul O'Leary McCann
619b1102e6 Use config to specify tok2vec_size 2022-07-03 15:32:35 +09:00
Paul O'Leary McCann
1a4dbb702d Add basic span predictor tests 2022-07-03 15:13:15 +09:00
Paul O'Leary McCann
201731df2d Move spans2ints to util 2022-07-03 15:12:53 +09:00
Paul O'Leary McCann
1dacecbbfb Run black 2022-07-03 14:49:02 +09:00
Paul O'Leary McCann
5192ac1617 Clean tests. 2022-07-03 14:48:42 +09:00
Paul O'Leary McCann
79720886fa Merge branch 'feature/coref' into fix/coref-alignment
Had to renumber error message.
2022-07-01 19:09:29 +09:00
explosion-bot
7e55a51314 Auto-format code with black 2022-07-01 08:04:32 +00:00
Madeesh Kannan
eaf66e7431
Add NVTX ranges to TrainablePipe components (#10965)
* `TrainablePipe`: Add NVTX range decorator

* Annotate `TrainablePipe` subclasses with NVTX ranges

* Export function signature to allow introspection of args in tests

* Revert "Annotate `TrainablePipe` subclasses with NVTX ranges"

This reverts commit d8684f7372.

* Revert "Export function signature to allow introspection of args in tests"

This reverts commit f4405ca3ad.

* Revert "`TrainablePipe`: Add NVTX range decorator"

This reverts commit 26536eb6b8.

* Add `spacy.pipes_with_nvtx_range` pipeline callback

* Show warnings for all missing user-defined pipe functions that need to be annotated
Fix imports, typos

* Rename `DEFAULT_ANNOTATABLE_PIPE_METHODS` to `DEFAULT_NVTX_ANNOTATABLE_PIPE_METHODS`
Reorder import

* Walk model nodes directly whilst applying NVTX ranges
Ignore pipe method wrapper when applying range
2022-06-30 11:28:12 +02:00
Adriane Boyd
3fe9f47de4
Revert "disable failing test because Stanford servers are down (#11015)" (#11054)
This reverts commit f8116078ce.
2022-06-30 11:24:54 +02:00
Shen Qin
be00db6645
Addition of min_max quantifier in matcher {n,m} (#10981)
* Min_max_operators
1. Modified API and Usage for spaCy website to include min_max operator
2. Modified matcher.pyx to include min_max function {n,m} and its variants
3. Modified schemas.py to include min_max validation error
4. Added test cases to test_matcher_api.py, test_matcher_logic.py and test_pattern_validation.py

* attempt to fix mypy/pydantic compat issue

* formatting

* Update spacy/tests/matcher/test_pattern_validation.py

Co-authored-by: Source-Shen <82353723+Source-Shen@users.noreply.github.com>
Co-authored-by: svlandeg <svlandeg@github.com>
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2022-06-30 11:01:58 +02:00
Daniël de Kok
0ff14aabce
vectors: avoid expensive comparisons between numpy ints and Python ints (#10992)
* vectors: avoid expensive comparisons between numpy ints and Python ints

* vectors: avoid failure on lists of ints

* Convert another numpy int to Python
2022-06-29 12:58:31 +02:00