Commit Graph

9547 Commits

Author SHA1 Message Date
Jari Bakken
ba8a840f84 spacy.cli.evaluate: fix TypeError (#3101) 2018-12-28 11:14:28 +01:00
Jari Bakken
0546135fba Set vectors.name when updating meta.json during training (#3100)
* Set vectors.name when updating meta.json during training

* add vectors name to meta in `spacy package`
2018-12-27 19:55:40 +01:00
Jari Bakken
cc95167b6d cli.convert: fix typo in converter arguments (#3099) 2018-12-27 18:08:41 +01:00
Jari Bakken
e172f2478e Add three missing tags from the nb tag map (#3085)
* Contributors agreement for jarib

* Add tags from the UD/NORNE dataset that is missing in the nb tag map. Relates to #3082.
2018-12-27 14:48:40 +01:00
Will Price
4a6af0852a Improve random prefix generation in displaCy arcs (#3096)
* Improve random prefix generation in displaCy arcs

* Add @willprice contributor agreement
2018-12-27 14:46:02 +01:00
Özcan Kasal
b573ebca77 trilyon forgotten (#3083)
* trilyon forgotten

* contributor added
2018-12-27 14:44:23 +01:00
Matthew Honnibal
978d8be8f9 Set version to v2.1.0a5 2018-12-21 00:26:39 +01:00
Matthew Honnibal
d3f03b1668 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-21 00:25:57 +01:00
Ines Montani
bb9ad37e05 Improve entry points and allow custom language classes via entry points (#3080)
* Remove check for overwritten factory

This needs to be handled differently – on first initialization, a new factory will be added and any subsequent initializations will trigger this warning, even if it's a new entry point that doesn't overwrite a built-in.

* Add helper to only load specific entry point

Useful for loading languages via entry points, so that they can be lazy-loaded. Otherwise, all entry point languages would have to be loaded upfront.

* Check entry points for custom languages
2018-12-20 23:58:43 +01:00
Matthew Honnibal
f6ac00fab3 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-20 18:45:44 +01:00
Matthew Honnibal
d8d27f9129 Set version to v2.1.0a5.dev0 2018-12-20 18:45:34 +01:00
Ines Montani
2dc6c52ccc Update displayed Binder version (see #3077) [ci skip] 2018-12-20 17:36:19 +01:00
Ines Montani
ca244f5f84
Small fixes to displaCy (#3076)
## Description
- [x] fix auto-detection of Jupyter notebooks (even if `jupyter=True` isn't set)
- [x] add `displacy.set_render_wrapper` method to define a custom function called around the HTML markup generated in all calls to `displacy.render` (can be used to allow custom integrations, callbacks and page formatting)
- [x] add option to customise host for web server
- [x] show warning if `displacy.serve` is called from within Jupyter notebooks
- [x] move error message to `spacy.errors.Errors`.

### Types of change
enhancement

## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2018-12-20 17:32:04 +01:00
Matthew Honnibal
aeb59f6791 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-20 16:15:01 +01:00
Matthew Honnibal
f57bea8ab6
💫 Prevent parser from predicting unseen classes (#3075)
The output weights often return negative scores for classes, especially
via the bias terms. This means that when we add a new class, we can't
rely on just zeroing the weights, or we'll end up with positive
predictions for those labels.

To solve this, we use nan values as the initial weights for new labels.
This prevents them from ever coming out on top. During backprop, we
replace the nan values with the minimum assigned score, so that we're
still able to learn these classes.
2018-12-20 16:12:22 +01:00
Matthew Honnibal
9ec9f89b99 💫 Raise better error when using uninitialized pipeline component (#3074)
After creating a component, the `.model` attribute is left with the value `True`, to indicate it should be created later during `from_disk()`, `from_bytes()` or `begin_training()`. This had led to confusing errors if you try to use the component without initializing the model.

To fix this, we add a method `require_model()` to the `Pipe` base class. The `require_model()` method needs to be called at the start of the `.predict()` and `.update()` methods of the components. It raises a `ValueError` if the model is not initialized. An error message has been added to `spacy.errors`.
2018-12-20 15:54:53 +01:00
Matthew Honnibal
1788bf1af7 Unbreak progress bar 2018-12-20 13:57:00 +01:00
Muhammad Irfan
2e84ec1513 Fixed ISO code for Urdu. (#3073) 2018-12-20 12:28:53 +01:00
Matthew Honnibal
c315e08e6e Fix formatting of meta.json after spacy package 2018-12-19 14:36:08 +01:00
Matthew Honnibal
b7ce85a6f3 Fix packaging of json schemas 2018-12-19 13:54:02 +01:00
Matthew Honnibal
35ff889852 Fix OSX wheel building 2018-12-19 13:14:57 +01:00
Matthew Honnibal
e24f94ce39 Fix handling of preset entities. closes #2779 2018-12-19 02:13:31 +01:00
Matthew Honnibal
faa8656582 Port parser fix for large label sets from master 2018-12-19 02:11:26 +01:00
Matthew Honnibal
99a84e4d0e Make ParserModel.resize_output idempotent 2018-12-19 02:10:36 +01:00
Matthew Honnibal
9fc8ce0c4d Add schemas to MANIFEST 2018-12-19 01:18:50 +01:00
Matthew Honnibal
a2b75036e9 Try to make sure json schemas are packaged 2018-12-19 01:08:51 +01:00
Matthew Honnibal
0f83b98afa Remove unused code from spacy pretrain 2018-12-18 19:19:26 +01:00
Ken
5f0c5fbfa4 issue #3012: add test (#3021)
* issue #3012: add test

* add contributor aggreement

* Make test work without models and fix typos

ten.pos_ instead of ten.orth_ and comparison against "10" instead of integer 10
2018-12-18 15:02:49 +01:00
Ines Montani
77a47b2b20 Auto-format 2018-12-18 15:02:11 +01:00
Kirill Bulygin
2fb004832f Fix the first nlp call for ja (closes #2901) (#3065)
* Fix the first `nlp` call for `ja` (closes #2901)

* Add unicode declaration, formatting and use relative import
2018-12-18 15:01:06 +01:00
Kirill Bulygin
10189d9092 Fix the first nlp call for ja (closes #2901) (#3065)
* Fix the first `nlp` call for `ja` (closes #2901)

* Add unicode declaration, formatting and use relative import
2018-12-18 14:53:50 +01:00
Ines Montani
ae880ef912 Tidy up merge conflict leftovers 2018-12-18 13:58:30 +01:00
Ines Montani
61d09c481b Merge branch 'master' into develop 2018-12-18 13:48:10 +01:00
Brixjohn
52f3c95004 Added alpha support for Tagalog language (#3062)
I have added alpha support for the Tagalog language from the Philippines. It is the basis for the country's national language Filipino. I have heavily based the format to the EN and ES languages.

I have provided several words in the lemmatizer lookup table, added stop words from a source, translated numeric words to its Tagalog counterpart, added some tokenizer exceptions, and kept the tag map the same as the English language.

While the alpha language passed the preliminary testing that you provided, I think it needs more data to be useful for most cases.

* Added alpha support for Tagalog language

* Edited contributor template

* Included SCA; Reverted templates

* Fixed SCA template

* Fixed changes in SCA template
2018-12-18 13:08:38 +01:00
Matthew Honnibal
92f4b9c8ea set max batch size to 1000 2018-12-17 23:15:39 +00:00
Matthew Honnibal
3c4a2edf4a Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-17 23:08:40 +00:00
Matthew Honnibal
95fc0176d1 Pass tagger options in begin_training 2018-12-17 23:08:31 +00:00
Matthew Honnibal
7c504b6ddb Try to implement more losses for pretraining
* Try to implement cosine loss
This one seems to be correct? Still unsure, but it performs okay

* Try to implement the von Mises-Fisher loss
This one's definitely not right yet.
2018-12-17 14:48:27 +00:00
Ines Montani
e3405f8af3 Don't call begin_training if updating new model (see #3059) [ci skip] 2018-12-17 13:45:49 +01:00
Ines Montani
c9a89bba50 Don't call begin_training if updating new model (see #3059) [ci skip] 2018-12-17 13:45:28 +01:00
Ines Montani
6f1438b5d9 Auto-format example 2018-12-17 13:44:38 +01:00
Matthew Honnibal
ab4b61fb6e Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-16 20:11:43 +01:00
Matthew Honnibal
9ef30b0cde Accept 'text' in matcher as an alternative to ORTH 2018-12-16 20:10:43 +01:00
Amandine Périnet
361554f629 Lemmatization of Adjectives - French : adding rules and vocabulary (#3045)
* modifying FR lemmatisation for Adjectives

* adding contributor agreement for amperinet

* correcting some errors in vocabulary files
2018-12-16 18:11:07 +01:00
Sofie
c6ad557cea French regular expressions instead of extensive exceptions list (on develop) (#3046) (resolves #2679)
* merge changes of PR 3023 into develop branch instead of master

* further deletions from exception list according to PR 3023
2018-12-16 18:04:55 +01:00
Ines Montani
7bbdffd36e Remove pre-set lemma for "cause" (resolves #2165) 2018-12-14 12:51:18 +01:00
Shooter23
6ae8e49bff Fix docstring for is_right_punct(). (#3044) 2018-12-14 10:11:11 +01:00
Matthew Honnibal
ab9494b2a3 Merge branch 'develop' of https://github.com/explosion/spaCy into develop 2018-12-12 21:08:50 +00:00
Matthew Honnibal
fb56028476 Remove b1 and b2 decay 2018-12-12 12:37:07 +01:00
Matthew Honnibal
df15279e88 Reduce batch size during pretrain 2018-12-10 15:30:23 +00:00