Adriane Boyd
5d19dfc9d3
Update Chinese tokenizer for spacy-pkuseg fork
2020-10-05 14:21:53 +02:00
Ines Montani
549758f67d
Adjust test for now
2020-10-04 23:16:09 +02:00
Ines Montani
3c36a57e84
Update data augmenters ( #6196 )
...
* Draft lower-case augmenter
* Make warning a debug log
* Update lowercase augmenter, docs and tests
Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-10-04 17:46:29 +02:00
Ines Montani
496228771d
Merge pull request #6194 from explosion/master-tmp
2020-10-04 15:25:41 +02:00
Ines Montani
0307a228c8
Merge pull request #6193 from explosion/fix/adjust-pipe-init
...
Adjust [initialize.components] on Language.remove_pipe and Language.rename_pipe
2020-10-04 15:20:54 +02:00
Ines Montani
59deeb7da6
Merge branch 'develop' into master-tmp
2020-10-04 14:52:20 +02:00
Ines Montani
8f018e47f8
Adjust [initialize.components] on Language.remove_pipe and Language.rename_pipe
2020-10-04 14:43:45 +02:00
Ines Montani
11347f34da
Tidy up, tests and docs
2020-10-04 13:54:05 +02:00
Ines Montani
d3b3663942
Adjust error message and add test
2020-10-04 10:11:27 +02:00
Ines Montani
2110e8f86d
Auto-format
2020-10-04 10:06:49 +02:00
Matthew Honnibal
835070cedc
Upd test
2020-10-03 19:35:10 +02:00
Ines Montani
c2401fca41
Add tests for Pipe.label_data
2020-10-03 19:12:46 +02:00
Ines Montani
3bc3c05fcc
Tidy up and auto-format
2020-10-03 17:20:18 +02:00
Ines Montani
7c4ab7e82c
Fix Lemmatizer.get_lookups_config
2020-10-03 17:16:10 +02:00
Ines Montani
dd542ec6a4
Fix label initialization of textcat component ( #6190 )
2020-10-03 17:07:38 +02:00
Sofie Van Landeghem
09dcb75076
small UX fix for DocBin ( #6167 )
...
* add informative warning when messing up store_user_data DocBin flags
* add informative warning when messing up store_user_data DocBin flags
* cleanup test
* rename to patterns_path
2020-10-02 15:43:32 +02:00
Ines Montani
f0b30aedad
Make lemmatizers use initialize logic ( #6182 )
...
* Make lemmatizer use initialize logic and tidy up
* Fix typo
* Raise for uninitialized tables
2020-10-02 15:42:36 +02:00
Ines Montani
d2aa662ab2
Merge pull request #6179 from adrianeboyd/feature/token-morph-refactor-2 [ci skip]
2020-10-02 12:10:27 +02:00
Ines Montani
c41a4332e4
Add test for custom data augmentation
2020-10-02 11:37:56 +02:00
Adriane Boyd
f83dfe62da
Fix test
2020-10-02 10:17:26 +02:00
Ines Montani
01c1538c72
Integrate file readers
2020-10-02 01:36:06 +02:00
Adriane Boyd
86c3ec9c2b
Refactor Token morph setting ( #6175 )
...
* Refactor Token morph setting
* Remove `Token.morph_`
* Add `Token.set_morph()`
* `0` resets `token.c.morph` to unset
* Any other values are passed to `Morphology.add`
* Add token.morph setter to set from MorphAnalysis
2020-10-01 22:21:46 +02:00
Ines Montani
d48ddd6c9a
Remove default initialize lookups
2020-10-01 21:54:33 +02:00
Adriane Boyd
73538782a0
Switch Doc.__init__(ents=) to IOB tags ( #6173 )
...
* Switch Doc.__init__(ents=) to IOB tags
* Fix check for "-"
* Allow "" or None as missing IOB tag
2020-10-01 16:22:18 +02:00
Yohei Tamura
3243ddac8f
Fix/span.sent ( #6083 )
...
* add fail test
* fix test
* fix span.sent
* Remove incorrect implicit check
Co-authored-by: Adriane Boyd <adrianeboyd@gmail.com>
2020-10-01 14:01:52 +02:00
Ines Montani
381258b75b
Merge pull request #6165 from explosion/feature/update-tokenizers-initialize
2020-10-01 09:49:47 +02:00
Ines Montani
a103ab5f1a
Update augmenter lookups and docs
2020-09-30 23:03:47 +02:00
Ines Montani
23c63eefaf
Tidy up env vars [ci skip]
2020-09-30 15:15:11 +02:00
Adriane Boyd
6b7bb32834
Refactor Chinese initialization
2020-09-30 11:46:45 +02:00
Ines Montani
34f9c26c62
Add lexeme norm defaults
2020-09-30 10:20:14 +02:00
Ines Montani
1aeef3bfbb
Make corpus paths default to None and improve errors
2020-09-29 22:33:46 +02:00
Ines Montani
fa47f87924
Tidy up and auto-format
2020-09-29 21:39:28 +02:00
Ines Montani
6467a560e3
WIP: Test updating Chinese tokenizer
2020-09-29 21:10:22 +02:00
Ines Montani
78021089f9
Merge pull request #6160 from explosion/feature/prepare
2020-09-29 20:55:13 +02:00
Ines Montani
c3f8c09d7d
Merge pull request #6154 from adrianeboyd/bugfix/chinese-tokenizer-pickle
2020-09-29 20:54:59 +02:00
Ines Montani
d3c63b7965
Merge branch 'develop' into feature/prepare
2020-09-29 20:53:05 +02:00
Ines Montani
2be80379ec
Fix small issues, resolve_dot_names and debug model
2020-09-29 20:38:35 +02:00
Ines Montani
7851020653
Update tests
2020-09-29 18:14:15 +02:00
Ines Montani
f2352eb701
Test with default value
2020-09-29 17:00:40 +02:00
Ines Montani
63d1598137
Simplify config use in Language.initialize
2020-09-29 16:05:48 +02:00
Ines Montani
56f8bc73ef
Add more tests
2020-09-29 15:23:34 +02:00
Ines Montani
591038b1a4
Add test
2020-09-29 12:54:52 +02:00
Matthew Honnibal
e1fdf2b7c5
Upd tests
2020-09-29 12:05:38 +02:00
Ines Montani
ff9a63bfbd
begin_training -> initialize
2020-09-28 21:35:09 +02:00
Ines Montani
2e9c9e74af
Fix config resolution and interpolation
...
TODO: auto-interpolate in Thinc if config is dict (i.e. likely subsection)
2020-09-28 15:34:00 +02:00
Ines Montani
822ea4ef61
Refactor CLI
2020-09-28 15:09:59 +02:00
Matthew Honnibal
a976da168c
Support data augmentation in Corpus ( #6155 )
...
* Support data augmentation in Corpus
* Note initial docs for data augmentation
* Add augmenter to quickstart
* Fix flake8
* Format
* Fix test
* Update spacy/tests/training/test_training.py
* Improve data augmentation arguments
* Update templates
* Move randomization out into caller
* Refactor
* Update spacy/training/augment.py
* Update spacy/tests/training/test_training.py
* Fix augment
* Fix test
2020-09-28 03:03:27 +02:00
Ines Montani
9016d23cc5
Fix exclude and add test
2020-09-27 23:34:03 +02:00
Ines Montani
7e938ed63e
Update config resolution to use new Thinc
2020-09-27 22:21:31 +02:00
Adriane Boyd
8393dbedad
Minor fixes
...
* Put `cfg` back in serialization
* Add `pickle5` to pytest conf
2020-09-27 15:15:53 +02:00
