Commit Graph

7216 Commits

Author SHA1 Message Date
Ines Montani
5f6f4ff594 Remove object subclassing 2020-07-12 14:03:23 +02:00
Matthew Honnibal
743f7fb73a Set version to v3.0.0a4 2020-07-10 22:40:12 +02:00
Matthew Honnibal
b68216e263
Explicitly delete objects after parser.update to free GPU memory (#5748)
* Try explicitly deleting objects

* Refactor parser model backprop slightly

* Free parser data explicitly after rehearse and update
2020-07-10 22:35:20 +02:00
Sofie Van Landeghem
de6a32315c
debug-model script (#5749)
* adding debug-model to print the internals for debugging purposes

* expend debug-model script with 4 stages: before, init, train, predict

* avoid enforcing to have a seed in the train script

* small fixes
2020-07-10 19:47:53 +02:00
Ines Montani
a60562f208
Update project CLI hashes, directories, skipping (#5741)
* Update project CLI hashes, directories, skipping

* Improve clone success message

* Remove unused context args

* Move project-specific utils to project utils

The hashing/checksum functions may not end up being general-purpose functions and are more designed for the projects, so they shouldn't live in spacy.util

* Improve run help and add workflows

* Add note re: directory checksum speed

* Fix cloning from subdirectories and output messages

* Remove hard-coded dirs
2020-07-09 23:51:18 +02:00
Matthew Honnibal
552d1ad226 Hack at tests 2020-07-09 20:25:51 +02:00
Matthew Honnibal
eb064c59cd Try to fix textcat test 2020-07-09 20:24:53 +02:00
Ines Montani
018319a640 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-07-09 19:44:41 +02:00
Ines Montani
05e182e421 Update CLI args and docstrings 2020-07-09 19:44:28 +02:00
Sofie Van Landeghem
dd207a28be
cleanup components API (#5726)
* add keyword separator for update functions and drop unused "state"

* few more Example tests and various small fixes

* consistently return losses after update call

* eliminate unused tensors field across pipe components

* fix name

* fix arg name
2020-07-09 19:43:39 +02:00
Adriane Boyd
ac4297ee39
Minor refactor to conversion of output docs (#5718)
Minor refactor of conversion of docs to output format to avoid
duplicate conversion steps.
2020-07-09 19:42:32 +02:00
Sofie Van Landeghem
c1ea55307b
Fixing reproducible training (#5735)
* Add initial reproducibility tests

* failing test for default_text_classifier (WIP)

* track trouble to underlying tok2vec layer

* add regression test for Issue 5551

* tests go green with https://github.com/explosion/thinc/pull/359

* update test

* adding fixed seeds to HashEmbed layers, seems to fix the reproducility issue

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-07-09 19:39:31 +02:00
Matthew Honnibal
1827f22f56 Set version to v3.0.0a3 2020-07-09 19:38:04 +02:00
Matthw Honnibal
7010f1a2be Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-07-09 19:34:11 +02:00
Matthw Honnibal
77af0a6bb4 Offer option of padding-sensitive batching 2020-07-09 14:50:20 +02:00
Matthw Honnibal
3a7f275c02 Add extra batch util 2020-07-09 14:38:41 +02:00
Matthw Honnibal
eb0798c421 Add __len__ method for Example 2020-07-09 14:38:26 +02:00
Ines Montani
8f9552d9e7
Refactor project CLI (#5732)
* Make project command a submodule

* Update with WIP

* Add helper for joining commands

* Update docstrins, formatting and types

* Update assets and add support for copying local files

* Fix type

* Update success messages
2020-07-09 01:42:51 +02:00
Adriane Boyd
ad15499b3b
Fix get_loss for values outside of labels in senter (#5730)
* Fix get_loss for None alignments in senter

When converting the `sent_start` values back to `SentenceRecognizer`
labels, handle `None` alignments.

* Handle SENT_START as -1

Handle SENT_START as -1 (or -1 converted to uint64) by treating any
values other than 1 the same as 0 in `SentenceRecognizer.get_loss`.
2020-07-09 01:41:58 +02:00
Matthw Honnibal
1b20ffac38 batch_by_words by default 2020-07-08 21:37:06 +02:00
Matthw Honnibal
93e50da46a Remove auto 'set_annotation' in training to address GPU memory 2020-07-08 21:36:51 +02:00
Matthw Honnibal
fb8a5967c1 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-07-08 15:27:50 +02:00
Ines Montani
0a3d41bb1d
Deprecat model shortcuts and simplify download (#5722) 2020-07-08 14:00:07 +02:00
Adriane Boyd
c9f0f75778
Update get_loss for senter and morphologizer (#5724)
* Update get_loss for senter

Update `SentenceRecognizer.get_loss` to keep it similar to `Tagger`.

* Update get_loss for morphologizer

Update `Morphologizer.get_loss` to keep it similar to `Tagger`.
2020-07-08 13:59:28 +02:00
Matthw Honnibal
ca989f4cc4 Improve cutting logic in parser 2020-07-08 11:27:54 +02:00
Matthw Honnibal
42e1109def Support option to not batch by number of words 2020-07-08 11:26:54 +02:00
Ines Montani
8cb7f9ccff
Improve assets and DVC handling (#5719)
* Improve assets and DVC handling

* Remove outdated comment [ci skip]
2020-07-07 20:51:50 +02:00
Sofie Van Landeghem
a39a110c4e
Few more Example unit tests (#5720)
* small fixes in Example, UX

* add gold tests for aligned_spans and get_aligned_parse

* sentencizer unnecessary
2020-07-07 18:46:00 +02:00
Matthw Honnibal
433dc3c9c9 Simplify PrecomputableAffine slightly 2020-07-07 17:22:47 +02:00
Matthw Honnibal
a4164f67ca Don't normalize gradients 2020-07-07 17:21:58 +02:00
Matthw Honnibal
8177f25b6c Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-07-07 17:21:10 +02:00
Ines Montani
fa00a85828
Merge pull request #5715 from explosion/chore/tidy-regression-tests 2020-07-07 11:22:07 +02:00
Matthw Honnibal
d1fd3438c3 Add dropout to parser hidden layer 2020-07-07 01:38:15 +02:00
Matthw Honnibal
f25761e513 Dont randomize cuts in parser 2020-07-06 17:51:25 +02:00
Matthw Honnibal
709fc5e4ad Clarify dropout and seed in Tok2Vec 2020-07-06 17:50:21 +02:00
Matthew Honnibal
19d42f42de Set version to v3.0.0a2 2020-07-06 17:43:12 +02:00
Matthew Honnibal
cc477be952
Improve gold-standard alignment (#5711)
* Remove previous alignment

* Implement better alignment, using ragged data structure

* Use pytokenizations for alignment

* Fixes

* Fixes

* Fix overlapping entities in alignment

* Fix align split_sents

* Update test

* Commit align.py

* Try to appease setuptools

* Fix flake8

* use realistic entities for testing

* Update tests for better alignment

* Improve alignment heuristic

Co-authored-by: svlandeg <sofie.vanlandeghem@gmail.com>
2020-07-06 17:39:31 +02:00
Ines Montani
b6deef80f8 Fix class to pickling works as expected 2020-07-06 16:43:45 +02:00
Ines Montani
fa261d09e8 Add alternative CLI option 2020-07-06 15:57:38 +02:00
Adriane Boyd
c67fc6aa5b
Make docs_to_json backwards-compatible with v2 (#5714)
* In `spacy convert -t json` output the JSON docs wrapped in a list

* Add back token-level `ner` alongside the doc-level `entities`
2020-07-06 14:15:00 +02:00
Ines Montani
5b7b2a498d Tidy up and merge regression tests 2020-07-06 14:05:59 +02:00
Ines Montani
412dbb1f38
Remove dead and/or deprecated code (#5710)
* Remove dead and/or deprecated code

* Remove n_threads

Co-authored-by: Matthew Honnibal <honnibal+gh@gmail.com>
2020-07-06 13:06:25 +02:00
Sofie Van Landeghem
fcbf899b08
Feature/example only (#5707)
* remove _convert_examples

* fix test_gold, raise TypeError if tuples are used instead of Example's

* throwing proper errors when the wrong type of objects are passed

* fix deprectated format in tests

* fix deprectated format in parser tests

* fix tests for NEL, morph, senter, tagger, textcat

* update regression tests with new Example format

* use make_doc

* more fixes to nlp.update calls

* few more small fixes for rehearse and evaluate

* only import ml_datasets if really necessary
2020-07-06 13:02:36 +02:00
Matthw Honnibal
3f6f087113 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2020-07-04 23:52:12 +02:00
Matthw Honnibal
5642507823 Fix has_unknown_spaces in Doc.copy 2020-07-04 23:52:02 +02:00
Matthw Honnibal
8870a6ded7 Specify seeds in HashEmbed 2020-07-04 23:51:49 +02:00
Ines Montani
37c3bb35e2 Auto-format 2020-07-04 16:25:34 +02:00
Ines Montani
abd173937f Auto-format and update URL 2020-07-04 14:23:44 +02:00
Ines Montani
99aff16d60 Make argument shortcut consistent 2020-07-04 14:23:32 +02:00
Matthew Honnibal
2bd1bf81f1
Refactor pretrain and support character-based objective for v3 (#5706)
* Start adding character-based stuff

* Start adding character-based objective

* Start adding character-based stuff

* Start adding character-based objective

* Remove outdated comment

* Update pretraining models

* Add/fix character-based multi-task models

* Refactor pretrain and support character-based objective

* Update pretrain config

* Remove unused

* Fix flake8 errors

* Clean up imports

* Format

* Format

* Update Thinc version

* Raise error if vectors objective but no vectors
2020-07-03 17:57:28 +02:00