Commit Graph

11246 Commits

Author SHA1 Message Date
Sofie Van Landeghem
06f0a8daa0
Default settings to configurations (#4995)
* fix grad_clip naming

* cleaning up pretrained_vectors out of cfg

* further refactoring Model init's

* move Model building out of pipes

* further refactor to require a model config when creating a pipe

* small fixes

* making cfg in nn_parser more consistent

* fixing nr_class for parser

* fixing nn_parser's nO

* fix printing of loss

* architectures in own file per type, consistent naming

* convenience methods default_tagger_config and default_tok2vec_config

* let create_pipe access default config if available for that component

* default_parser_config

* move defaults to separate folder

* allow reading nlp from package or dir with argument 'name'

* architecture spacy.VocabVectors.v1 to read static vectors from file

* cleanup

* default configs for nel, textcat, morphologizer, tensorizer

* fix imports

* fixing unit tests

* fixes and clean up

* fixing defaults, nO, fix unit tests

* restore parser IO

* fix IO

* 'fix' serialization test

* add *.cfg to manifest

* fix example configs with additional arguments

* replace Morpohologizer with Tagger

* add IO bit when testing overfitting of tagger (currently failing)

* fix IO - don't initialize when reading from disk

* expand overfitting tests to also check IO goes OK

* remove dropout from HashEmbed to fix Tagger performance

* add defaults for sentrec

* update thinc

* always pass a Model instance to a Pipe

* fix piped_added statement

* remove obsolete W029

* remove obsolete errors

* restore byte checking tests (work again)

* clean up test

* further test cleanup

* convert from config to Model in create_pipe

* bring back error when component is not initialized

* cleanup

* remove calls for nlp2.begin_training

* use thinc.api in imports

* allow setting charembed's nM and nC

* fix for hardcoded nM/nC + unit test

* formatting fixes

* trigger build
2020-02-27 18:42:27 +01:00
Ines Montani
f39ddda193
Merge pull request #5062 from svlandeg/bugfix/merge-conflicts
Fix sync between master and develop
2020-02-26 13:41:16 +01:00
svlandeg
fc6e34c3a1 fix bugs from porting master to develop 2020-02-26 08:44:22 +01:00
Ines Montani
192b8d45a1
Merge pull request #5008 from svlandeg/fix/build_dependencies
Re-add pyproject.toml and add tests for dependency version consistency
2020-02-25 16:52:18 +01:00
Ines Montani
b6a6cff708 Add blis to pyproject.toml 2020-02-25 16:17:23 +01:00
Ines Montani
912572e04a Only copy if file exists (not if installed from sdist etc.) 2020-02-25 16:01:58 +01:00
Ines Montani
436b26fe0f Revert other changes 2020-02-25 15:48:29 +01:00
Ines Montani
c1a5ece65f Tidy up setup and update requirements tests 2020-02-25 15:46:39 +01:00
Ines Montani
5d21d3e8b9 Merge branch 'develop' into pr/5008 2020-02-25 15:24:47 +01:00
svlandeg
d5bfebe1c5 it's moving day 2020-02-24 10:04:24 +01:00
svlandeg
217c16c7a9 running tests BEFORE deleting them ? 2020-02-24 09:38:43 +01:00
svlandeg
6f846c2cbf removing --pyargs for testing purposes 2020-02-24 09:19:08 +01:00
svlandeg
d821c95eb0 debugging prints 2020-02-23 17:38:33 +01:00
svlandeg
58568bd0cd fix 2020-02-23 16:45:37 +01:00
svlandeg
0f55e51704 assert we found the root_dir 2020-02-23 16:33:58 +01:00
svlandeg
783da088ea avoid try except 2020-02-23 16:21:21 +01:00
svlandeg
9f1447bf71 where areth thou, file ? 2020-02-19 17:09:29 +02:00
svlandeg
9834527f2c hack to switch between CLI folder setup and local setup 2020-02-19 16:22:48 +02:00
svlandeg
5c2f645470 root dir one level up 2020-02-19 16:15:56 +02:00
svlandeg
303c4bcd4c include requirements in manifest 2020-02-19 15:52:55 +02:00
svlandeg
b20351792a assert prints for more clarity 2020-02-19 15:51:53 +02:00
Ines Montani
8137b24928
Merge pull request #5028 from explosion/refactor/remove-symlinks
Remove symlinks, data dir and related stuff
2020-02-19 00:20:23 +01:00
Ines Montani
a3335d36b8 Merge branch 'develop' into refactor/remove-symlinks 2020-02-18 17:22:20 +01:00
Ines Montani
a138acb220
Merge pull request #5027 from explosion/chore/sync-develop-master
Sync develop with master, tidy up, auto-format
2020-02-18 17:22:03 +01:00
Ines Montani
09cbeaef27 Remove symlinks, data dir and related stuff 2020-02-18 17:20:17 +01:00
Ines Montani
e3f40a6a0f Tidy up and auto-format 2020-02-18 15:38:18 +01:00
Ines Montani
1278161f47 Tidy up and fix issues 2020-02-18 15:17:03 +01:00
Ines Montani
de11ea753a Merge branch 'master' into develop 2020-02-18 14:47:23 +01:00
Ines Montani
80e95d02b1 Allow spacy attr in token pattern 2020-02-18 14:32:53 +01:00
Jan Jessewitsch
c7e4fe9c5c
Fix/Improve german stop words (#5024)
* Fix german stop words

Two stop words ("einige" and  "einigen") are sticking together.
Remove three nouns that may serve as stop words in a specific context (e.g. religious or news) but are not applicable for general use.

* Create Jan-711.md
2020-02-17 18:59:22 +01:00
Kabir Khan
f6ed07b85c
Use nlp.pipe in EntityRuler for phrase patterns in add_patterns (#4931)
* Fix ent_ids and labels properties when id attribute used in patterns

* use set for labels

* sort end_ids for comparison in entity_ruler tests

* fixing entity_ruler ent_ids test

* add to set

* Run make_doc optimistically if using phrase matcher patterns.

* remove unused coveragerc I was testing with

* format

* Refactor EntityRuler.add_patterns to use nlp.pipe for phrase patterns. Improves speed substantially.

* Removing old add_patterns function

* Fixing spacing

* Make sure token_patterns loaded as well, before generator was being emptied in from_disk
2020-02-16 18:17:47 +01:00
Sofie Van Landeghem
72c964bcf4
define pretrained_dims which is used by build_text_classifier (#5004) 2020-02-16 17:21:17 +01:00
adrianeboyd
3b22eb651b
Sync Span __eq__ and __hash__ (#5005)
* Sync Span __eq__ and __hash__

Use the same tuple for `__eq__` and `__hash__`, including all attributes
except `vector` and `vector_norm`.

* Update entity comparison in tests

Update `assert_docs_equal()` test util to compare `Span` properties for
ents rather than `Span` objects.
2020-02-16 17:20:36 +01:00
adrianeboyd
0c47a53b5e
Use int only in key2row for better performance (#4990)
Cast all keys and rows to `int` in `vectors.key2row` for more efficient
access and serialization.
2020-02-16 17:19:41 +01:00
adrianeboyd
5b102963bf
Require HEAD for is_parsed in Doc.from_array() (#5011)
Modify flag settings so that `DEP` is not sufficient to set `is_parsed`
and only run `set_children_from_heads()` if `HEAD` is provided.

Then the combination `[SENT_START, DEP]` will set deps and not clobber
sent starts with a lot of one-word sentences.
2020-02-16 17:17:09 +01:00
Sofie Van Landeghem
2572460175
add tok2vec parameters to train script to facilitate init_tok2vec (#5021) 2020-02-16 17:16:41 +01:00
Sofie Van Landeghem
a27c77ce62
add message when cli train script throws exception (#5009)
* add message when cli train script throws exception

* fix formatting
2020-02-15 15:50:17 +01:00
Christos Aridas
ff8e71f46d
Update streamlit app (#5017)
* Update streamlit app [ci skip]

* Add all labels by default

* Tidy up and auto-format

Co-authored-by: Ines Montani <ines@ines.io>
2020-02-15 15:49:09 +01:00
nlptechbook
979a3fd1f5
Update universe.json (#5022)
e-book is available from https://nostarch.com/NLPPython
2020-02-15 15:44:55 +01:00
svlandeg
2729d9164d cleanup 2020-02-12 22:59:37 +01:00
svlandeg
6bbd816569 formatting 2020-02-12 22:50:27 +01:00
svlandeg
34986c7bfd test versions of required libs across different places 2020-02-12 22:49:50 +01:00
svlandeg
2079948711 add build dependencies back to pyproject.toml 2020-02-12 22:49:21 +01:00
Ines Montani
2ed49404e3
Improve setup.py and call into Cython directly (#4952)
* Improve setup.py and call into Cython directly

* Add numpy to setup_requires

* Improve clean helper

* Update setup.cfg

* Try if it builds without pyproject.toml

* Update MANIFEST.in
2020-02-11 17:46:18 -05:00
adrianeboyd
99a543367d
Set GPU before loading any models in train CLI (#4989)
Set the GPU before loading any existing models in the train CLI so that
you can start with a base model and train on GPU.
2020-02-11 17:45:41 -05:00
adrianeboyd
842dfddbb9
Standardize Greek tag map setup (#4997)
* Rename `tag_map.py` to `tag_map_fine.py` to indicate that it's not the
default tag map
* Remove duplicate generic UD tag map and load `../tag_map.py` instead
2020-02-11 17:44:56 -05:00
Sofie Van Landeghem
1c01842588
add pyx and pxd files to the distribution (#5000) 2020-02-11 17:42:17 -05:00
Sofie Van Landeghem
9b84f987bd
fix grad_clip naming (#4967) 2020-02-10 20:33:16 -05:00
Antti Ajanki
e1f777b151
Improvements for Finnish tokenizer (#4985)
* don't split on a colon. Colon is used to attach suffixes for abbreviations
* tokenize on any of LIST_HYPHENS (except a single hyphen), not just on --
* simplify infix rules by merging similar rules
2020-02-10 20:32:43 -05:00
Sofie Van Landeghem
781e95cf53
Ensure doc.similarity returns a float (on develop) (#4969) 2020-02-10 20:31:49 -05:00