Merge branch 'master' of https://github.com/explosion/spaCy into cite-zenodo

.github/ISSUE_TEMPLATE.md

@@ -1,15 +1,18 @@
 <!--- Please provide a summary in the title and describe your issue here.
 Is this a bug or feature request? If a bug, include all the steps that led to the issue.
 
-If you're looking for help with your code, consider posting a question on Stack Overflow instead:
+If you're looking for help with your code, consider posting a question here:
-http://stackoverflow.com/questions/tagged/spacy -->
-
 
+- GitHub Discussions: https://github.com/explosion/spaCy/discussions
+- Stack Overflow: http://stackoverflow.com/questions/tagged/spacy
+-->
 
 ## Your Environment
-<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type 
+
+<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type
 `python -m spacy info --markdown` and copy-paste the result here.-->
-* Operating System: 
+
-* Python Version Used: 
+- Operating System:
-* spaCy Version Used: 
+- Python Version Used:
-* Environment Information: 
+- spaCy Version Used:
+- Environment Information:

.github/ISSUE_TEMPLATE/03_request.md

@@ -1,11 +0,0 @@
----
-name: "\U0001F381 Feature Request"
-about: Do you have an idea for an improvement, a new feature or a plugin?
-
----
-
-## Feature description
-<!-- Please describe the feature: Which area of the library is it related to? What specific solution would you like? -->
-
-## Could the feature be a [custom component](https://spacy.io/usage/processing-pipelines#custom-components) or [spaCy plugin](https://spacy.io/universe)?
-If so, we will tag it as [`project idea`](https://github.com/explosion/spaCy/labels/project%20idea) so other users can take it on.

.github/ISSUE_TEMPLATE/04_other.md

@@ -0,0 +1,19 @@
+---
+name: "\U0001F4AC Anything else?"
+about: For feature and project ideas, general usage questions or help with your code, please post on the GitHub Discussions board instead.
+---
+
+<!-- Describe your issue here. Please keep in mind that the GitHub issue tracker is mostly intended for reports related to the spaCy code base and source, and for bugs and enhancements. If you're looking for help with your code, consider posting a question here:
+
+- GitHub Discussions: https://github.com/explosion/spaCy/discussions
+- Stack Overflow: http://stackoverflow.com/questions/tagged/spacy
+-->
+
+## Your Environment
+
+<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type `python -m spacy info --markdown` and copy-paste the result here.-->
+
+- Operating System:
+- Python Version Used:
+- spaCy Version Used:
+- Environment Information:

.github/ISSUE_TEMPLATE/05_other.md

@@ -1,15 +0,0 @@
----
-name: "\U0001F4AC Anything else?"
-about: For general usage questions or help with your code, please consider
-  posting on Stack Overflow instead.
-
----
-
-<!-- Describe your issue here. Please keep in mind that the GitHub issue tracker is mostly intended for reports related to the spaCy code base and source, and for bugs and feature requests. If you're looking for help with your code, consider posting a question on Stack Overflow instead: http://stackoverflow.com/questions/tagged/spacy -->
-
-## Your Environment
-<!-- Include details of your environment. If you're using spaCy 1.7+, you can also type `python -m spacy info --markdown` and copy-paste the result here.-->
-* Operating System:
-* Python Version Used:
-* spaCy Version Used:
-* Environment Information:

CONTRIBUTING.md

@@ -26,11 +26,11 @@ also often include helpful tips and solutions to common problems. You should
 also check the [troubleshooting guide](https://spacy.io/usage/#troubleshooting)
 to see if your problem is already listed there.
 
-If you're looking for help with your code, consider posting a question on
+If you're looking for help with your code, consider posting a question on the
-[Stack Overflow](http://stackoverflow.com/questions/tagged/spacy) instead. If you
+[GitHub Discussions board](https://github.com/explosion/spaCy/discussions) or
-tag it `spacy` and `python`, more people will see it and hopefully be able to
+[Stack Overflow](http://stackoverflow.com/questions/tagged/spacy). Please
-help. Please understand that we won't be able to provide individual support via
+understand that we won't be able to provide individual support via email. We
-email. We also believe that help is much more valuable if it's **shared publicly**,
+also believe that help is much more valuable if it's **shared publicly**,
 so that more people can benefit from it.
 
 ### Submitting issues

README.md

@@ -61,17 +61,16 @@ be able to provide individual support via email. We also believe that help is
 much more valuable if it's shared publicly, so that more people can benefit from
 it.
 
-| Type                     | Platforms                                              |
+| Type                            | Platforms                               |
-| ------------------------ | ------------------------------------------------------ |
+| ------------------------------- | --------------------------------------- |
-| 🚨 **Bug Reports**       | [GitHub Issue Tracker]                                 |
+| 🚨 **Bug Reports**              | [GitHub Issue Tracker]                  |
-| 🎁 **Feature Requests**  | [GitHub Issue Tracker]                                 |
+| 🎁 **Feature Requests & Ideas** | [GitHub Discussions]                    |
-| 👩‍💻 **Usage Questions**   | [Stack Overflow] · [Gitter Chat] · [Reddit User Group] |
+| 👩‍💻 **Usage Questions**          | [GitHub Discussions] · [Stack Overflow] |
-| 🗯 **General Discussion** | [Gitter Chat] · [Reddit User Group]                    |
+| 🗯 **General Discussion**        | [GitHub Discussions]                    |
 
 [github issue tracker]: https://github.com/explosion/spaCy/issues
+[github discussions]: https://github.com/explosion/spaCy/discussions
 [stack overflow]: https://stackoverflow.com/questions/tagged/spacy
-[gitter chat]: https://gitter.im/explosion/spaCy
-[reddit user group]: https://www.reddit.com/r/spacynlp
 
 ## Features
 
@@ -111,16 +110,16 @@ For detailed installation instructions, see the
 
 Using pip, spaCy releases are available as source packages and binary wheels (as
 of `v2.0.13`). Before you install spaCy and its dependencies, make sure that
-`pip` and `setuptools` are up to date.
+`pip`, `setuptools` and `wheel` are up to date.
 
 ```bash
-pip install -U pip setuptools
+pip install -U pip setuptools wheel
 pip install spacy
 ```
 
-For installation on python 3.5 where binary wheels are not provided for the most
+For installation on python 2.7 or 3.5 where binary wheels are not provided for
-recent versions of the dependencies, you can prefer older binary wheels over
+the most recent versions of the dependencies, you can prefer older binary
-newer source packages with `--prefer-binary`:
+wheels over newer source packages with `--prefer-binary`:
 
 ```bash
 pip install spacy --prefer-binary
@@ -140,6 +139,7 @@ environment to avoid modifying system state:
 ```bash
 python -m venv .env
 source .env/bin/activate
+pip install -U pip setuptools wheel
 pip install spacy
 ```
 
@@ -238,16 +238,28 @@ do that depends on your system. See notes on Ubuntu, OS X and Windows for
 details.
 
 ```bash
-# make sure you are using the latest pip
-python -m pip install -U pip
 git clone https://github.com/explosion/spaCy
 cd spaCy
 
 python -m venv .env
 source .env/bin/activate
-export PYTHONPATH=`pwd`
+
+# make sure you are using the latest pip
+python -m pip install -U pip setuptools wheel
+
+pip install .
+```
+
+To install with extras:
+
+```bash
+pip install .[lookups,cuda102]
+```
+
+To install all dependencies required for development:
+
+```bash
 pip install -r requirements.txt
-python setup.py build_ext --inplace
 ```
 
 Compared to regular install via pip, [requirements.txt](requirements.txt)
@@ -287,14 +299,13 @@ tests, you'll usually want to clone the repository and build spaCy from source.
 This will also install the required development dependencies and test utilities
 defined in the `requirements.txt`.
 
-Alternatively, you can find out where spaCy is installed and run `pytest` on
+Alternatively, you can run `pytest` on the tests from within the installed
-that directory. Don't forget to also install the test utilities via spaCy's
+`spacy` package. Don't forget to also install the test utilities via spaCy's
 `requirements.txt`:
 
 ```bash
-python -c "import os; import spacy; print(os.path.dirname(spacy.__file__))"
+pip install -r requirements.txt
-pip install -r path/to/requirements.txt
+python -m pytest --pyargs spacy
-python -m pytest <spacy-directory>
 ```
 
 See [the documentation](https://spacy.io/usage#tests) for more details and

setup.cfg

@@ -63,21 +63,25 @@ install_requires =
 lookups =
     spacy_lookups_data>=0.3.2,<0.4.0
 cuda =
-    cupy>=5.0.0b4,<8.0.0
+    cupy>=5.0.0b4
 cuda80 =
-    cupy-cuda80>=5.0.0b4,<8.0.0
+    cupy-cuda80>=5.0.0b4
 cuda90 =
-    cupy-cuda90>=5.0.0b4,<8.0.0
+    cupy-cuda90>=5.0.0b4
 cuda91 =
-    cupy-cuda91>=5.0.0b4,<8.0.0
+    cupy-cuda91>=5.0.0b4
 cuda92 =
-    cupy-cuda92>=5.0.0b4,<8.0.0
+    cupy-cuda92>=5.0.0b4
 cuda100 =
-    cupy-cuda100>=5.0.0b4,<8.0.0
+    cupy-cuda100>=5.0.0b4
 cuda101 =
-    cupy-cuda101>=5.0.0b4,<8.0.0
+    cupy-cuda101>=5.0.0b4
 cuda102 =
-    cupy-cuda102>=5.0.0b4,<8.0.0
+    cupy-cuda102>=5.0.0b4
+cuda110 =
+    cupy-cuda110>=5.0.0b4
+cuda111 =
+    cupy-cuda111>=5.0.0b4
 # Language tokenizers with external dependencies
 ja =
     sudachipy>=0.4.5

spacy/about.py

@@ -1,6 +1,6 @@
 # fmt: off
 __title__ = "spacy"
-__version__ = "2.3.4"
+__version__ = "2.3.5"
 __release__ = True
 __download_url__ = "https://github.com/explosion/spacy-models/releases/download"
 __compatibility__ = "https://raw.githubusercontent.com/explosion/spacy-models/master/compatibility.json"

website/UNIVERSE.md

@@ -11,8 +11,8 @@ suggest it by submitting a pull request to this repository. The Universe
 database is open-source and collected in a simple JSON file.
 
 Looking for inspiration for your own spaCy plugin or extension? Check out the
-[`project idea`](https://github.com/explosion/spaCy/labels/project%20idea) label
+[`project ideas`](https://github.com/explosion/spaCy/discussions?discussions_q=category%3A%22New+Features+%26+Project+Ideas%22) 
-on the issue tracker.
+discussion forum.
 
 ## Checklist
 

website/docs/usage/index.md

@@ -31,9 +31,10 @@ import QuickstartInstall from 'widgets/quickstart-install.js'
 ### pip {#pip}
 
 Using pip, spaCy releases are available as source packages and binary wheels (as
-of v2.0.13).
+of v2.0.13). For the most recent releases, pip 19.3 or newer is recommended.
 
 ```bash
+$ pip install -U pip setuptools wheel
 $ pip install -U spacy
 ```
 
@@ -66,6 +67,7 @@ environment to avoid modifying system state:
 ```bash
 python -m venv .env
 source .env/bin/activate
+pip install -U pip setuptools wheel
 pip install spacy
 ```
 
@@ -122,10 +124,11 @@ support, we've been grateful to use the work of Chainer's
 interface for GPU arrays.
 
 spaCy can be installed on GPU by specifying `spacy[cuda]`, `spacy[cuda90]`,
-`spacy[cuda91]`, `spacy[cuda92]`, `spacy[cuda100]`, `spacy[cuda101]` or
+`spacy[cuda91]`, `spacy[cuda92]`, `spacy[cuda100]`, `spacy[cuda101]`,
-`spacy[cuda102]`. If you know your cuda version, using the more explicit
+`spacy[cuda102]`, `spacy[cuda110]` or `spacy[cuda111]`. If you know your cuda
-specifier allows cupy to be installed via wheel, saving some compilation time.
+version, using the more explicit specifier allows cupy to be installed via
-The specifiers should install [`cupy`](https://cupy.chainer.org).
+wheel, saving some compilation time. The specifiers should install
+[`cupy`](https://cupy.chainer.org).
 
 ```bash
 $ pip install -U spacy[cuda92]
@@ -158,15 +161,25 @@ system. See notes on [Ubuntu](#source-ubuntu), [macOS / OS X](#source-osx) and
 [Windows](#source-windows) for details.
 
 ```bash
-python -m pip install -U pip                   # update pip
 git clone https://github.com/explosion/spaCy   # clone spaCy
 cd spaCy                                       # navigate into directory
 
 python -m venv .env                            # create environment in .env
 source .env/bin/activate                       # activate virtual environment
-\export PYTHONPATH=`pwd`                        # set Python path to spaCy directory
+python -m pip install -U pip setuptools wheel  # update build tools
-pip install -r requirements.txt                # install all requirements
+pip install .                                  # compile and install spaCy
-python setup.py build_ext --inplace            # compile spaCy
+```
+
+To install with extras:
+
+```bash
+pip install .[lookups,cuda102]                 # install spaCy with extras
+```
+
+To install all dependencies required for development:
+
+```bash
+pip install -r requirements.txt
 ```
 
 Compared to regular install via pip, the
@@ -204,6 +217,36 @@ official distributions these are:
 | Python 3.4   | Visual Studio 2010 |
 | Python 3.5+  | Visual Studio 2015 |
 
+#### Additional options for developers {#source-developers}
+
+Some additional options may be useful for spaCy developers who are editing the
+source code and recompiling frequently.
+
+- Install in editable mode. Changes to `.py` files will be reflected as soon as
+  the files are saved, but edits to Cython files (`.pxd`, `.pyx`) will require
+  the `pip install` or `python setup.py build_ext` command below to be run
+  again. Before installing in editable mode, be sure you have removed any
+  previous installs with `pip uninstall spacy`, which you may need to run
+  multiple times to remove all traces of earlier installs.
+
+  ```diff
+    pip install -U pip setuptools wheel
+  - pip install .
+  + pip install -r requirements.txt
+  + pip install --no-build-isolation --editable .
+  ```
+
+- Build in parallel using `N` CPUs to speed up compilation and then install in
+  editable mode:
+
+  ```diff
+    pip install -U pip setuptools wheel
+  - pip install .
+  + pip install -r requirements.txt
+  + python setup.py build_ext --inplace -j N
+  + python setup.py develop
+  ```
+
 ### Run tests {#run-tests}
 
 spaCy comes with an
@@ -213,14 +256,12 @@ In order to run the tests, you'll usually want to clone the
 [build spaCy from source](#source). This will also install the required
 development dependencies and test utilities defined in the `requirements.txt`.
 
-Alternatively, you can find out where spaCy is installed and run `pytest` on
+Alternatively, you can run `pytest` on the tests packaged with the install
-that directory. Don't forget to also install the test utilities via spaCy's
+`spacy package. Don't forget to also install the test utilities via spaCy's [`requirements.txt`](https://github.com/explosion/spaCy/tree/master/requirements.txt):
-[`requirements.txt`](https://github.com/explosion/spaCy/tree/master/requirements.txt):
 
 ```bash
-python -c "import os; import spacy; print(os.path.dirname(spacy.__file__))"
+pip install -r requirements.txt
-pip install -r path/to/requirements.txt
+python -m pytest --pyargs spacy
-python -m pytest [spacy directory]
 ```
 
 Calling `pytest` on the spaCy directory will run only the basic tests. The flag
@@ -230,8 +271,8 @@ Calling `pytest` on the spaCy directory will run only the basic tests. The flag
 # make sure you are using recent pytest version
 python -m pip install -U pytest
 
-python -m pytest [spacy directory]                 # basic tests
+python -m pytest --pyargs spacy                # basic tests
-python -m pytest [spacy directory] --slow          # basic and slow tests
+python -m pytest --pyargs spacy --slow         # basic and slow tests
 ```
 
 ## Troubleshooting guide {#troubleshooting}

website/docs/usage/spacy-101.md

@@ -915,9 +915,10 @@ via the following platforms:
   questions** and everything related to problems with your specific code. The
   Stack Overflow community is much larger than ours, so if your problem can be
   solved by others, you'll receive help much quicker.
-- [Gitter chat](https://gitter.im/explosion/spaCy): **General discussion** about
+- [GitHub discussions](https://github.com/explosion/spaCy/discussions): **General 
-  spaCy, meeting other community members and exchanging **tips, tricks and best
+  discussion**, **project ideas** and **usage questions**. Meet other community 
-  practices**.
+  members to get help with a specific code implementation, discuss ideas for new 
+  projects/plugins, support more languages, and share best practices.
 - [GitHub issue tracker](https://github.com/explosion/spaCy/issues): **Bug
   reports** and **improvement suggestions**, i.e. everything that's likely
   spaCy's fault. This also includes problems with the models beyond statistical

website/meta/site.json

@@ -44,20 +44,23 @@
             "items": [
                 { "text": "Usage", "url": "/usage" },
                 { "text": "Models", "url": "/models" },
-                { "text": "API", "url": "/api" },
+                { "text": "API Reference", "url": "/api" },
-                { "text": "Universe", "url": "/universe" }
+                { "text": "Online Course", "url": "https://course.spacy.io" }
             ]
         },
         {
-            "label": "Support",
+            "label": "Community",
             "items": [
+                { "text": "Universe", "url": "/universe" },
+                {
+                    "text": "GitHub Discussions",
+                    "url": "https://github.com/explosion/spaCy/discussions"
+                },
                 { "text": "Issue Tracker", "url": "https://github.com/explosion/spaCy/issues" },
                 {
                     "text": "Stack Overflow",
                     "url": "http://stackoverflow.com/questions/tagged/spacy"
-                },
+                }
-                { "text": "Reddit User Group", "url": "https://www.reddit.com/r/spacynlp/" },
-                { "text": "Gitter Chat", "url": "https://gitter.im/explosion/spaCy" }
             ]
         },
         {

website/src/widgets/quickstart-install.js

@@ -88,17 +88,15 @@ const QuickstartInstall = ({ id, title }) => (
                     <QS config="venv" os="windows">
                         .env\Scripts\activate
                     </QS>
-                    <QS package="pip">pip install -U spacy</QS>
+                    <QS package="pip">
+                        pip install -U pip setuptools wheel
+                    </QS>
+                    <QS package="pip">
+                        pip install -U spacy
+                    </QS>
                     <QS package="conda">conda install -c conda-forge spacy</QS>
                     <QS package="source">git clone https://github.com/{repo}</QS>
                     <QS package="source">cd spaCy</QS>
-                    <QS package="source" os="linux">
-                        export PYTHONPATH=`pwd`
-                    </QS>
-                    <QS package="source" os="windows">
-                        set PYTHONPATH=/path/to/spaCy
-                    </QS>
-                    <QS package="source">pip install -r requirements.txt</QS>
                     <QS data="lookups" package="pip">
                         pip install -U spacy-lookups-data
                     </QS>
@@ -108,7 +106,10 @@ const QuickstartInstall = ({ id, title }) => (
                     <QS data="lookups" package="conda">
                         conda install -c conda-forge spacy-lookups-data
                     </QS>
-                    <QS package="source">python setup.py build_ext --inplace</QS>
+                    <QS package="source">
+                        pip install -U pip setuptools wheel
+                    </QS>
+                    <QS package="source">pip install .</QS>
                     {models.map(({ code, models: modelOptions }) => (
                         <QS models={code} key={code}>
                             python -m spacy download {modelOptions[0]}