Commit Graph

53 Commits

Author SHA1 Message Date
Matthew Honnibal
2c37e0ccf6
💫 Use Blis for matrix multiplications (#2966)
Our epic matrix multiplication odyssey is drawing to a close...

I've now finally got the Blis linear algebra routines in a self-contained Python package, with wheels for Windows, Linux and OSX. The only missing platform at the moment is Windows Python 2.7. The result is at https://github.com/explosion/cython-blis

Thinc v7.0.0 will make the change to Blis. I've put a Thinc v7.0.0.dev0 up on PyPi so that we can test these changes with the CI, and even get them out to spacy-nightly, before Thinc v7.0.0 is released. This PR also updates the other dependencies to be in line with the current versions master is using. I've also resolved the msgpack deprecation problems, and gotten spaCy and Thinc up to date with the latest Cython.

The point of switching to Blis is to have control of how our matrix multiplications are executed across platforms. When we were using numpy for this, a different library would be used on pip and conda, OSX would use Accelerate, etc. This would open up different bugs and performance problems, especially when multi-threading was introduced.

With the change to Blis, we now strictly single-thread the matrix multiplications. This will make it much easier to use multiprocessing to parallelise the runtime, since we won't have nested parallelism problems to deal with.

* Use blis

* Use -2 arg to Cython

* Update dependencies

* Fix requirements

* Update setup dependencies

* Fix requirement typo

* Fix msgpack errors

* Remove Python27 test from Appveyor, until Blis works there

* Auto-format setup.py

* Fix murmurhash version
2018-11-27 00:44:04 +01:00
Matthew Honnibal
4336397ecb Update develop from master 2018-08-14 03:04:28 +02:00
Matthew Honnibal
e05bebce8e Try setting appveyor to Python2 64 2018-07-24 20:47:03 +02:00
Matthew Honnibal
8661218fe8
Refactor parser (#2308)
* Work on refactoring greedy parser

* Compile updated parser

* Fix refactored parser

* Update test

* Fix refactored parser

* Fix refactored parser

* Readd beam search after refactor

* Fix beam search after refactor

* Fix parser

* Fix beam parsing

* Support oracle segmentation in ud-train CLI command

* Avoid relying on final gold check in beam search

* Add a keyword argument sink to GoldParse

* Bug fixes to beam search after refactor

* Avoid importing fused token symbol in ud-run-test, untl that's added

* Avoid importing fused token symbol in ud-run-test, untl that's added

* Don't modify Token in global scope

* Fix error in beam gradient calculation

* Default to beam_update_prob 1

* Set a more aggressive threshold on the max violn update

* Disable some tests to figure out why CI fails

* Disable some tests to figure out why CI fails

* Add some diagnostics to travis.yml to try to figure out why build fails

* Tell Thinc to link against system blas on Travis

* Point thinc to libblas on Travis

* Try running sudo=true for travis

* Unhack travis.sh

* Restore beam_density argument for parser beam

* Require thinc 6.11.1.dev16

* Revert hacks to tests

* Revert hacks to travis.yml

* Update thinc requirement

* Fix parser model loading

* Fix size limits in training data

* Add missing name attribute for parser

* Fix appveyor for Windows
2018-05-15 22:17:29 +02:00
Matthew Honnibal
1f7229f40f Revert "Merge branch 'develop' of https://github.com/explosion/spaCy into develop"
This reverts commit c9ba3d3c2d, reversing
changes made to 92c26a35d4.
2018-03-27 19:23:02 +02:00
Hugo
aa898ab4e4 Drop support for EOL Python 2.6 and 3.3 2017-11-26 19:46:24 +02:00
Matthew Honnibal
a181987061 Try to fix appveyor 2017-09-26 06:55:15 -05:00
Matthew Honnibal
1ffc9a7fbf Fix appveyor 2017-09-15 23:59:36 +02:00
Matthew Honnibal
2432308f3e Build in separate step for appveyor 2017-09-15 23:55:19 +02:00
Matthew Honnibal
02273eeca8 Appveyor 2017-09-15 12:55:33 +02:00
Matthew Honnibal
25ec8935ad Appveyor 2017-09-15 12:53:07 +02:00
Matthew Honnibal
1f840a9211 Appveyor 2017-09-15 12:49:02 +02:00
Matthew Honnibal
86367ab092 Start work on appveyor, for Windows build 2017-09-15 12:41:59 +02:00
Ines Montani
64d1057ad4 Rename appveyor.yml to .appveyor.yml 2017-06-02 12:34:34 +02:00
Henning Peters
7e0d1dd8d3 remove appveyor 2016-02-10 11:28:55 +01:00
Henning Peters
66765d4d8f Update .appveyor.yml 2016-02-10 08:04:11 +01:00
Henning Peters
634ea57876 adapt travis/appveyor to latest sputnik 2016-01-16 09:56:12 +01:00
Matthew Honnibal
8b61d45ed0 * Fix merge conflicts for headers branch 2015-12-27 17:46:25 +01:00
Henning Peters
f5eb3a38f7 fix 2015-12-21 23:28:17 +01:00
Henning Peters
1292c11ece streamline ci/build systems 2015-12-18 18:04:26 +01:00
Henning Peters
762bcc4b1f streamline ci/build systems 2015-12-18 17:47:06 +01:00
Henning Peters
d1f46528ca fix appveyor setup 2015-12-18 11:42:23 +01:00
Henning Peters
55bd1469dc add newer sputnik version 2015-12-15 15:34:27 +01:00
Henning Peters
3597804c7a Update .appveyor.yml 2015-12-09 14:22:26 +01:00
maxirmx
b6f8333cf1 Added project dir to PYTHONPATH 2015-10-25 21:51:33 +03:00
maxirmx
9b53640fdb Added project root to PATH 2015-10-25 21:33:55 +03:00
maxirmx
368e387e80 Appveyor build cleanup 2015-10-25 21:06:22 +03:00
maxirmx
8d13b9d297 Stdint download fix 2015-10-25 21:01:39 +03:00
maxirmx
c62dc940c3 Stdint download fix 2015-10-25 20:33:02 +03:00
maxirmx
5be59f3165 Merge 2015-10-25 20:20:12 +03:00
maxirmx
a728374806 Merge remote-tracking branch 'refs/remotes/honnibal/master'
Conflicts:
	appveyor
2015-10-25 20:09:19 +03:00
maxirmx
00b7a06f09 Added submodule update 2015-10-24 21:51:01 +03:00
maxirmx
1113705085 Switching to appveyor toolkit 2015-10-24 19:46:18 +03:00
maxirmx
64a61d52db Appveyor.yml cleanup 2015-10-22 01:15:53 +03:00
maxirmx
6bb8e05fe2 Win32 build added 2015-10-21 23:13:42 +03:00
maxirmx
685dc89754 Test build #2 2015-10-20 22:53:14 +03:00
maxirmx
f43dee555b Test build #1 2015-10-20 22:52:11 +03:00
Maxim Samsonov
f81fa55db6 Added win32 build to the matrix 2015-10-16 00:04:55 +03:00
maxirmx
d6be51d08f Envorced MSVC environment for pip install cython 2015-10-14 07:47:00 +03:00
maxirmx
ef3f62e92d Added Python 3.4 (latest) to build matrix 2015-10-14 06:52:12 +03:00
maxirmx
805b6ef5a6 Added PYTHONPATH to appveyor build #2 2015-10-13 12:45:32 +03:00
maxirmx
9e6e80b9aa Added PYTHONPATH to appveyor config 2015-10-13 12:42:13 +03:00
maxirmx
62a401fbea Addiding model_init and tests to appveyor build 2015-10-13 12:26:32 +03:00
maxirmx
f8cb17f23e Wordnet download fix #3 2015-10-13 12:08:31 +03:00
maxirmx
635306f860 Wordnet download fix#2 2015-10-13 12:05:02 +03:00
maxirmx
41faa7a375 Wordnet download fix 2015-10-13 12:03:57 +03:00
maxirmx
26071e863e Appveyor build #7 2015-10-13 11:49:04 +03:00
maxirmx
653ed854cd Appveyor build #6 2015-10-13 10:55:48 +03:00
maxirmx
237db7f519 Appveyor build #5
Added Wordnet download
2015-10-13 10:11:56 +03:00
maxirmx
7c5bfc5916 Merging Windows/Linux versions of setup.py 2015-10-13 01:31:59 +03:00