explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-09-22 20:09:18 +03:00
Code Issues Packages Projects Releases Wiki Activity
13,964 Commits 66 Branches 174 Tags 277 MiB
1a77607036
Commit Graph

2094 Commits

Author SHA1 Message Date
Ines Montani
178760855f Merge branch 'develop' into master-tmp 2020-10-15 09:06:03 +02:00
svlandeg
0796401c19 call NumpyOps instead of get_current_ops() 2020-10-14 16:55:00 +02:00
svlandeg
e94a21638e adding tests for trained models to ensure predict reproducibility 2020-10-13 21:07:13 +02:00
svlandeg
ede979d42f formattting 2020-10-13 18:53:17 +02:00
svlandeg
ff83bfae3f naming 2020-10-13 18:52:37 +02:00
svlandeg
6ccacff54e add tests for individual spacy layers 2020-10-13 18:50:07 +02:00
svlandeg
c23041ae60 component tests single or multiple prediction 2020-10-13 16:26:53 +02:00
svlandeg
3a505e7e14 small edit to ensure the new word was indeed new 2020-10-10 21:05:28 +02:00
svlandeg
68d79796c6 add test for vocab after serializing KB 2020-10-10 20:59:48 +02:00
Ines Montani
539b0c10da Tidy up and auto-format 2020-10-10 19:14:48 +02:00
Ines Montani
bfa3931c9d
Revert added_strings change (#6236) 2020-10-10 18:55:07 +02:00
Ines Montani
525f798841 Fix typo in test 2020-10-09 18:00:21 +02:00
Ines Montani
b7cb9d95e4
Merge pull request #6229 from svlandeg/bugfix/disabled 2020-10-09 16:05:11 +02:00
Ines Montani
9fb3244672
Merge pull request #6231 from adrianeboyd/feature/include-static-vectors 2020-10-09 15:54:52 +02:00
Adriane Boyd
727370c633 Remove Span._recalculate_indices 
Remove `Span._recalculate_indices`, which is a remnant from the
deprecated `Span.merge`.
 2020-10-09 14:42:51 +02:00
svlandeg
06b9d213fd formatting 2020-10-09 12:19:47 +02:00
Ines Montani
4771a10503 Make test more explicit [ci skip] 2020-10-09 12:15:26 +02:00
Ines Montani
cc3646b06c Add xfailing test for peculiar spans failure [ci skip] 2020-10-09 12:10:25 +02:00
svlandeg
8316bc7d4a bugfix DisabledPipes 2020-10-09 12:06:20 +02:00
Adriane Boyd
39aabf50ab Also rename to include_static_vectors in CharEmbed 2020-10-09 11:54:48 +02:00
Florijan Stamenković
18f5c309dc Fix Issue 6207 (#6208) 
* Regression test for issue 6207

* Fix issue 6207

* Sign contributor agreement

* Minor adjustments to test

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2020-10-09 10:14:40 +02:00
Duygu Altinok
80fb1bffc9 Ordinal numbers for Turkish (#6142) 
* minor ordinal number addition

* fixed typo

* added corresponding lexical test
 2020-10-09 10:13:15 +02:00
Duygu Altinok
2fad279a44 Turkish language syntax iterators (#6191) 
* added tr_vocab to config

* basic test

* added syntax iterator to Turkish lang class

* first version for Turkish syntax iter, without flat

* added simple tests with nmod, amod, det

* more tests to amod and nmod

* separated noun chunks and parser test

* rearrangement after nchunk parser separation

* added recursive NPs

* tests with complicated recursive NPs

* tests with conjed NPs

* additional tests for conj NP

* small modification for shaving off conj from NP

* added tests with flat

* more tests with flat

* added examples with flats conjed

* added inner func for flat trick

* corrected parse

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2020-10-09 10:10:22 +02:00
Sofie Van Landeghem
d093d6343b
TrainablePipe (#6213) 
* rename Pipe to TrainablePipe

* split functionality between Pipe and TrainablePipe

* remove unnecessary methods from certain components

* cleanup

* hasattr(component, "pipe") should be sufficient again

* remove serialization and vocab/cfg from Pipe

* unify _ensure_examples and validate_examples

* small fixes

* hasattr checks for self.cfg and self.vocab

* make is_resizable and is_trainable properties

* serialize strings.json instead of vocab

* fix KB IO + tests

* fix typos

* more typos

* _added_strings as a set

* few more tests specifically for _added_strings field

* bump to 3.0.0a36
 2020-10-08 21:33:49 +02:00
Ines Montani
8ff73f04db Fix morph in Doc.to_json 2020-10-08 14:44:35 +02:00
Ines Montani
064575d79d
Merge pull request #6216 from svlandeg/feature/nel-initialize 2020-10-08 11:14:12 +02:00
svlandeg
eaf5c265cb set_kb method for entity_linker 2020-10-08 10:34:01 +02:00
Ines Montani
010956d493 Clear rule-based components on initialize 2020-10-08 09:51:31 +02:00
Sofie Van Landeghem
2998131416
Reproducibility for TextCat and Tok2Vec (#6218) 
* ensure fixed seed in HashEmbed layers

* forgot about the joys of python 2
 2020-10-08 00:43:46 +02:00
svlandeg
efedccea8d fix tests 2020-10-07 15:29:52 +02:00
svlandeg
6b8bdb2d39 add init_config to nlp.create_pipe 2020-10-07 14:58:16 +02:00
Duygu Altinok
7e821c2776
Turkish language syntax iterators (#6191) 
* added tr_vocab to config

* basic test

* added syntax iterator to Turkish lang class

* first version for Turkish syntax iter, without flat

* added simple tests with nmod, amod, det

* more tests to amod and nmod

* separated noun chunks and parser test

* rearrangement after nchunk parser separation

* added recursive NPs

* tests with complicated recursive NPs

* tests with conjed NPs

* additional tests for conj NP

* small modification for shaving off conj from NP

* added tests with flat

* more tests with flat

* added examples with flats conjed

* added inner func for flat trick

* corrected parse

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2020-10-07 11:07:52 +02:00
Duygu Altinok
b95a11dd95
Ordinal numbers for Turkish (#6142) 
* minor ordinal number addition

* fixed typo

* added corresponding lexical test
 2020-10-07 10:25:37 +02:00
Rahul Gupta
1a00bff06d
Hindi: Adds tests for lexical attributes (norm and like_num) (#5829) 
* Hindi: Adds tests for lexical attributes (norm and like_num)

* Signs and sdds the contributor agreement

* Add ordinal numbers to be tagged as like_num

* Adds alternate pronunciation for 31 and 39
 2020-10-07 10:23:32 +02:00
Sofie Van Landeghem
fff3f8ccfa
Fix packaging pin (#6212) 
* pin packaging to >=20.0

* ignore spacy-pkuseg in requirements unit test
 2020-10-06 14:16:05 +02:00
Matthew Honnibal
cfb9770a94
Fix empty input into StaticVectors layer (#6211) 
* Add test for empty doc(s)

* Fix empty check in staticvectors

* Remove xfail

* Update spacy/ml/staticvectors.py
 2020-10-06 14:15:41 +02:00
Florijan Stamenković
9db670b996
Fix Issue 6207 (#6208) 
* Regression test for issue 6207

* Fix issue 6207

* Sign contributor agreement

* Minor adjustments to test

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2020-10-06 11:17:37 +02:00
Ines Montani
568e12215d
Merge pull request #6206 from svlandeg/fix/patterns-init 2020-10-06 10:27:23 +02:00
svlandeg
ff9ac39c88 read entity_ruler patterns with srsly.read_jsonl.v1 2020-10-05 22:50:14 +02:00
Ines Montani
126268ce50 Auto-format [ci skip] 2020-10-05 21:58:18 +02:00
Ines Montani
181039bd17
Merge pull request #6205 from explosion/feature/embed-features 2020-10-05 21:49:10 +02:00
Ines Montani
568617af58
Merge pull request #6202 from explosion/feature/project-spacy-version 2020-10-05 21:40:52 +02:00
Ines Montani
6abfc2911d
Merge pull request #6203 from adrianeboyd/feature/zh-spacy-pkuseg 2020-10-05 21:35:57 +02:00
Matthew Honnibal
91d0fbb588 Fix test 2020-10-05 21:13:53 +02:00
Matthew Honnibal
b392d48e76 Fix test 2020-10-05 20:17:07 +02:00
Matthew Honnibal
db84d175c3 Fix test 2020-10-05 19:59:30 +02:00
Matthew Honnibal
6dcc4a0ba6 Simplify MultiHashEmbed signature 2020-10-05 19:57:45 +02:00
Matthew Honnibal
7d93575f35 spacy/tests/ 2020-10-05 15:28:12 +02:00
Matthew Honnibal
f4ca9a39cb spacy/tests/ 2020-10-05 15:27:06 +02:00
Matthew Honnibal
f2f1deca66 spacy/tests/ 2020-10-05 15:24:33 +02:00
Matthew Honnibal
8ec79ad3fa Allow configuration of MultiHashEmbed features 
Update arguments to MultiHashEmbed layer so that the attributes can be
controlled. A kind of tricky scheme is used to allow optional
specification of the rows. I think it's an okay balance between
flexibility and convenience.
 2020-10-05 15:22:00 +02:00
Adriane Boyd
5d19dfc9d3 Update Chinese tokenizer for spacy-pkuseg fork 2020-10-05 14:21:53 +02:00
Ines Montani
6958510bda Include spaCy version check in project CLI 2020-10-05 13:53:07 +02:00
Ines Montani
20f2a17a09 Merge test_misc and test_util 2020-10-05 13:45:57 +02:00
Ines Montani
1c641e41c3 Remove unused import [ci skip] 2020-10-05 11:50:11 +02:00
Adriane Boyd
b0b93854cb Update ru/uk lemmatizers for new nlp.initialize 2020-10-05 09:27:16 +02:00
Ines Montani
549758f67d Adjust test for now 2020-10-04 23:16:09 +02:00
Ines Montani
3c36a57e84
Update data augmenters (#6196) 
* Draft lower-case augmenter

* Make warning a debug log

* Update lowercase augmenter, docs and tests

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
 2020-10-04 17:46:29 +02:00
Ines Montani
496228771d
Merge pull request #6194 from explosion/master-tmp 2020-10-04 15:25:41 +02:00
Ines Montani
0307a228c8
Merge pull request #6193 from explosion/fix/adjust-pipe-init 
Adjust [initialize.components] on Language.remove_pipe and Language.rename_pipe
 2020-10-04 15:20:54 +02:00
Ines Montani
59deeb7da6 Merge branch 'develop' into master-tmp 2020-10-04 14:52:20 +02:00
Ines Montani
8f018e47f8 Adjust [initialize.components] on Language.remove_pipe and Language.rename_pipe 2020-10-04 14:43:45 +02:00
Ines Montani
11347f34da Tidy up, tests and docs 2020-10-04 13:54:05 +02:00
Ines Montani
d3b3663942 Adjust error message and add test 2020-10-04 10:11:27 +02:00
Ines Montani
2110e8f86d Auto-format 2020-10-04 10:06:49 +02:00
Matthew Honnibal
835070cedc Upd test 2020-10-03 19:35:10 +02:00
Ines Montani
c2401fca41 Add tests for Pipe.label_data 2020-10-03 19:12:46 +02:00
Ines Montani
3bc3c05fcc Tidy up and auto-format 2020-10-03 17:20:18 +02:00
Ines Montani
7c4ab7e82c Fix Lemmatizer.get_lookups_config 2020-10-03 17:16:10 +02:00
Ines Montani
dd542ec6a4
Fix label initialization of textcat component (#6190) 2020-10-03 17:07:38 +02:00
Sofie Van Landeghem
09dcb75076
small UX fix for DocBin (#6167) 
* add informative warning when messing up store_user_data DocBin flags

* add informative warning when messing up store_user_data DocBin flags

* cleanup test

* rename to patterns_path
 2020-10-02 15:43:32 +02:00
Ines Montani
f0b30aedad
Make lemmatizers use initialize logic (#6182) 
* Make lemmatizer use initialize logic and tidy up

* Fix typo

* Raise for uninitialized tables
 2020-10-02 15:42:36 +02:00
Ines Montani
d2aa662ab2
Merge pull request #6179 from adrianeboyd/feature/token-morph-refactor-2 [ci skip] 2020-10-02 12:10:27 +02:00
Ines Montani
c41a4332e4 Add test for custom data augmentation 2020-10-02 11:37:56 +02:00
Adriane Boyd
f83dfe62da Fix test 2020-10-02 10:17:26 +02:00
Ines Montani
01c1538c72 Integrate file readers 2020-10-02 01:36:06 +02:00
Adriane Boyd
86c3ec9c2b
Refactor Token morph setting (#6175) 
* Refactor Token morph setting

* Remove `Token.morph_`
* Add `Token.set_morph()`
  * `0` resets `token.c.morph` to unset
  * Any other values are passed to `Morphology.add`

* Add token.morph setter to set from MorphAnalysis
 2020-10-01 22:21:46 +02:00
Ines Montani
d48ddd6c9a Remove default initialize lookups 2020-10-01 21:54:33 +02:00
Adriane Boyd
73538782a0
Switch Doc.__init__(ents=) to IOB tags (#6173) 
* Switch Doc.__init__(ents=) to IOB tags

* Fix check for "-"

* Allow "" or None as missing IOB tag
 2020-10-01 16:22:18 +02:00
Yohei Tamura
3243ddac8f
Fix/span.sent (#6083) 
* add fail test

* fix test

* fix span.sent

* Remove incorrect implicit check

Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
 2020-10-01 14:01:52 +02:00
Ines Montani
381258b75b
Merge pull request #6165 from explosion/feature/update-tokenizers-initialize 2020-10-01 09:49:47 +02:00
Ines Montani
a103ab5f1a Update augmenter lookups and docs 2020-09-30 23:03:47 +02:00
Ines Montani
23c63eefaf Tidy up env vars [ci skip] 2020-09-30 15:15:11 +02:00
Adriane Boyd
6b7bb32834 Refactor Chinese initialization 2020-09-30 11:46:45 +02:00
Ines Montani
34f9c26c62 Add lexeme norm defaults 2020-09-30 10:20:14 +02:00
Ines Montani
1aeef3bfbb Make corpus paths default to None and improve errors 2020-09-29 22:33:46 +02:00
Ines Montani
fa47f87924 Tidy up and auto-format 2020-09-29 21:39:28 +02:00
Ines Montani
6467a560e3 WIP: Test updating Chinese tokenizer 2020-09-29 21:10:22 +02:00
Ines Montani
78021089f9
Merge pull request #6160 from explosion/feature/prepare 2020-09-29 20:55:13 +02:00
Ines Montani
c3f8c09d7d
Merge pull request #6154 from adrianeboyd/bugfix/chinese-tokenizer-pickle 2020-09-29 20:54:59 +02:00
Ines Montani
d3c63b7965 Merge branch 'develop' into feature/prepare 2020-09-29 20:53:05 +02:00
Ines Montani
2be80379ec Fix small issues, resolve_dot_names and debug model 2020-09-29 20:38:35 +02:00
Ines Montani
7851020653 Update tests 2020-09-29 18:14:15 +02:00
Ines Montani
f2352eb701 Test with default value 2020-09-29 17:00:40 +02:00
Ines Montani
63d1598137 Simplify config use in Language.initialize 2020-09-29 16:05:48 +02:00
Ines Montani
56f8bc73ef Add more tests 2020-09-29 15:23:34 +02:00
Ines Montani
591038b1a4 Add test 2020-09-29 12:54:52 +02:00
Matthew Honnibal
e1fdf2b7c5 Upd tests 2020-09-29 12:05:38 +02:00
Ines Montani
ff9a63bfbd begin_training -> initialize 2020-09-28 21:35:09 +02:00
Ines Montani
2e9c9e74af Fix config resolution and interpolation 
TODO: auto-interpolate in Thinc if config is dict (i.e. likely subsection)
 2020-09-28 15:34:00 +02:00