💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
---
|
|
|
|
|
title: Install spaCy
|
|
|
|
|
next: /usage/models
|
|
|
|
|
menu:
|
|
|
|
|
- ['Quickstart', 'quickstart']
|
|
|
|
|
- ['Instructions', 'installation']
|
|
|
|
|
- ['Troubleshooting', 'troubleshooting']
|
|
|
|
|
- ['Changelog', 'changelog']
|
|
|
|
|
---
|
|
|
|
|
|
2019-12-22 03:53:56 +03:00
|
|
|
|
spaCy is compatible with **64-bit CPython 3.6+** and runs on **Unix/Linux**,
|
|
|
|
|
**macOS/OS X** and **Windows**. The latest spaCy releases are available over
|
|
|
|
|
[pip](https://pypi.python.org/pypi/spacy) and
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
[conda](https://anaconda.org/conda-forge/spacy).
|
|
|
|
|
|
|
|
|
|
> #### 📖 Looking for the old docs?
|
|
|
|
|
>
|
2020-07-01 22:26:39 +03:00
|
|
|
|
> To help you make the transition from v2.x to v3.0, we've uploaded the old
|
|
|
|
|
> website to [**v2.spacy.io**](https://v2.spacy.io/docs). To see what's changed
|
|
|
|
|
> and how to migrate, see the guide on [v3.0 guide](/usage/v3).
|
Generalize handling of tokenizer special cases (#4259)
* Generalize handling of tokenizer special cases
Handle tokenizer special cases more generally by using the Matcher
internally to match special cases after the affix/token_match
tokenization is complete.
Instead of only matching special cases while processing balanced or
nearly balanced prefixes and suffixes, this recognizes special cases in
a wider range of contexts:
* Allows arbitrary numbers of prefixes/affixes around special cases
* Allows special cases separated by infixes
Existing tests/settings that couldn't be preserved as before:
* The emoticon '")' is no longer a supported special case
* The emoticon ':)' in "example:)" is a false positive again
When merged with #4258 (or the relevant cache bugfix), the affix and
token_match properties should be modified to flush and reload all
special cases to use the updated internal tokenization with the Matcher.
* Remove accidentally added test case
* Really remove accidentally added test
* Reload special cases when necessary
Reload special cases when affixes or token_match are modified. Skip
reloading during initialization.
* Update error code number
* Fix offset and whitespace in Matcher special cases
* Fix offset bugs when merging and splitting tokens
* Set final whitespace on final token in inserted special case
* Improve cache flushing in tokenizer
* Separate cache and specials memory (temporarily)
* Flush cache when adding special cases
* Repeated `self._cache = PreshMap()` and `self._specials = PreshMap()`
are necessary due to this bug:
https://github.com/explosion/preshed/issues/21
* Remove reinitialized PreshMaps on cache flush
* Update UD bin scripts
* Update imports for `bin/`
* Add all currently supported languages
* Update subtok merger for new Matcher validation
* Modify blinded check to look at tokens instead of lemmas (for corpora
with tokens but not lemmas like Telugu)
* Use special Matcher only for cases with affixes
* Reinsert specials cache checks during normal tokenization for special
cases as much as possible
* Additionally include specials cache checks while splitting on infixes
* Since the special Matcher needs consistent affix-only tokenization
for the special cases themselves, introduce the argument
`with_special_cases` in order to do tokenization with or without
specials cache checks
* After normal tokenization, postprocess with special cases Matcher for
special cases containing affixes
* Replace PhraseMatcher with Aho-Corasick
Replace PhraseMatcher with the Aho-Corasick algorithm over numpy arrays
of the hash values for the relevant attribute. The implementation is
based on FlashText.
The speed should be similar to the previous PhraseMatcher. It is now
possible to easily remove match IDs and matches don't go missing with
large keyword lists / vocabularies.
Fixes #4308.
* Restore support for pickling
* Fix internal keyword add/remove for numpy arrays
* Add test for #4248, clean up test
* Improve efficiency of special cases handling
* Use PhraseMatcher instead of Matcher
* Improve efficiency of merging/splitting special cases in document
* Process merge/splits in one pass without repeated token shifting
* Merge in place if no splits
* Update error message number
* Remove UD script modifications
Only used for timing/testing, should be a separate PR
* Remove final traces of UD script modifications
* Update UD bin scripts
* Update imports for `bin/`
* Add all currently supported languages
* Update subtok merger for new Matcher validation
* Modify blinded check to look at tokens instead of lemmas (for corpora
with tokens but not lemmas like Telugu)
* Add missing loop for match ID set in search loop
* Remove cruft in matching loop for partial matches
There was a bit of unnecessary code left over from FlashText in the
matching loop to handle partial token matches, which we don't have with
PhraseMatcher.
* Replace dict trie with MapStruct trie
* Fix how match ID hash is stored/added
* Update fix for match ID vocab
* Switch from map_get_unless_missing to map_get
* Switch from numpy array to Token.get_struct_attr
Access token attributes directly in Doc instead of making a copy of the
relevant values in a numpy array.
Add unsatisfactory warning for hash collision with reserved terminal
hash key. (Ideally it would change the reserved terminal hash and redo
the whole trie, but for now, I'm hoping there won't be collisions.)
* Restructure imports to export find_matches
* Implement full remove()
Remove unnecessary trie paths and free unused maps.
Parallel to Matcher, raise KeyError when attempting to remove a match ID
that has not been added.
* Switch to PhraseMatcher.find_matches
* Switch to local cdef functions for span filtering
* Switch special case reload threshold to variable
Refer to variable instead of hard-coded threshold
* Move more of special case retokenize to cdef nogil
Move as much of the special case retokenization to nogil as possible.
* Rewrap sort as stdsort for OS X
* Rewrap stdsort with specific types
* Switch to qsort
* Fix merge
* Improve cmp functions
* Fix realloc
* Fix realloc again
* Initialize span struct while retokenizing
* Temporarily skip retokenizing
* Revert "Move more of special case retokenize to cdef nogil"
This reverts commit 0b7e52c797cd8ff1548f214bd4186ebb3a7ce8b1.
* Revert "Switch to qsort"
This reverts commit a98d71a942fc9bca531cf5eb05cf89fa88153b60.
* Fix specials check while caching
* Modify URL test with emoticons
The multiple suffix tests result in the emoticon `:>`, which is now
retokenized into one token as a special case after the suffixes are
split off.
* Refactor _apply_special_cases()
* Use cdef ints for span info used in multiple spots
* Modify _filter_special_spans() to prefer earlier
Parallel to #4414, modify _filter_special_spans() so that the earlier
span is preferred for overlapping spans of the same length.
* Replace MatchStruct with Entity
Replace MatchStruct with Entity since the existing Entity struct is
nearly identical.
* Replace Entity with more general SpanC
* Replace MatchStruct with SpanC
* Add error in debug-data if no dev docs are available (see #4575)
* Update azure-pipelines.yml
* Revert "Update azure-pipelines.yml"
This reverts commit ed1060cf59e5895b5fe92ad5b894fd1078ec4c49.
* Use latest wasabi
* Reorganise install_requires
* add dframcy to universe.json (#4580)
* Update universe.json [ci skip]
* Fix multiprocessing for as_tuples=True (#4582)
* Fix conllu script (#4579)
* force extensions to avoid clash between example scripts
* fix arg order and default file encoding
* add example config for conllu script
* newline
* move extension definitions to main function
* few more encodings fixes
* Add load_from_docbin example [ci skip]
TODO: upload the file somewhere
* Update README.md
* Add warnings about 3.8 (resolves #4593) [ci skip]
* Fixed typo: Added space between "recognize" and "various" (#4600)
* Fix DocBin.merge() example (#4599)
* Replace function registries with catalogue (#4584)
* Replace functions registries with catalogue
* Update __init__.py
* Fix test
* Revert unrelated flag [ci skip]
* Bugfix/dep matcher issue 4590 (#4601)
* add contributor agreement for prilopes
* add test for issue #4590
* fix on_match params for DependencyMacther (#4590)
* Minor updates to language example sentences (#4608)
* Add punctuation to Spanish example sentences
* Combine multilanguage examples for lang xx
* Add punctuation to nb examples
* Always realloc to a larger size
Avoid potential (unlikely) edge case and cymem error seen in #4604.
* Add error in debug-data if no dev docs are available (see #4575)
* Update debug-data for GoldCorpus / Example
* Ignore None label in misaligned NER data
2019-11-13 23:24:35 +03:00
|
|
|
|
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
## Quickstart {hidden="true"}
|
|
|
|
|
|
|
|
|
|
import QuickstartInstall from 'widgets/quickstart-install.js'
|
|
|
|
|
|
|
|
|
|
<QuickstartInstall title="Quickstart" id="quickstart" />
|
|
|
|
|
|
|
|
|
|
## Installation instructions {#installation}
|
|
|
|
|
|
2019-03-14 19:56:53 +03:00
|
|
|
|
### pip {#pip}
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
2020-07-04 17:47:24 +03:00
|
|
|
|
Using pip, spaCy releases are available as source packages and binary wheels.
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
$ pip install -U spacy
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
> #### Download models
|
|
|
|
|
>
|
|
|
|
|
> After installation you need to download a language model. For more info and
|
|
|
|
|
> available models, see the [docs on models](/models).
|
|
|
|
|
>
|
2020-08-19 01:28:37 +03:00
|
|
|
|
> ```cli
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
> $ python -m spacy download en_core_web_sm
|
|
|
|
|
>
|
|
|
|
|
> >>> import spacy
|
|
|
|
|
> >>> nlp = spacy.load("en_core_web_sm")
|
|
|
|
|
> ```
|
|
|
|
|
|
2019-10-01 14:22:13 +03:00
|
|
|
|
<Infobox variant="warning">
|
|
|
|
|
|
2020-07-04 17:47:24 +03:00
|
|
|
|
To install additional data tables for lemmatization you can run
|
|
|
|
|
`pip install spacy[lookups]` or install
|
2019-10-01 14:22:13 +03:00
|
|
|
|
[`spacy-lookups-data`](https://github.com/explosion/spacy-lookups-data)
|
2019-10-01 15:19:34 +03:00
|
|
|
|
separately. The lookups package is needed to create blank models with
|
2019-10-01 15:20:44 +03:00
|
|
|
|
lemmatization data, and to lemmatize in languages that don't yet come with
|
2019-10-02 11:37:39 +03:00
|
|
|
|
pretrained models and aren't powered by third-party libraries.
|
2019-10-01 14:22:13 +03:00
|
|
|
|
|
|
|
|
|
</Infobox>
|
|
|
|
|
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
When using pip it is generally recommended to install packages in a virtual
|
|
|
|
|
environment to avoid modifying system state:
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ python -m venv .env
|
|
|
|
|
$ source .env/bin/activate
|
|
|
|
|
$ pip install spacy
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
2019-03-14 19:56:53 +03:00
|
|
|
|
### conda {#conda}
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
Thanks to our great community, we've been able to re-add conda support. You can
|
|
|
|
|
also install spaCy via `conda-forge`:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
$ conda install -c conda-forge spacy
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
For the feedstock including the build recipe and configuration, check out
|
|
|
|
|
[this repository](https://github.com/conda-forge/spacy-feedstock). Improvements
|
|
|
|
|
and pull requests to the recipe and setup are always appreciated.
|
|
|
|
|
|
|
|
|
|
### Upgrading spaCy {#upgrading}
|
|
|
|
|
|
2020-07-01 22:26:39 +03:00
|
|
|
|
> #### Upgrading from v2 to v3
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
>
|
|
|
|
|
> Although we've tried to keep breaking changes to a minimum, upgrading from
|
2020-07-01 22:26:39 +03:00
|
|
|
|
> spaCy v2.x to v3.x may still require some changes to your code base. For
|
|
|
|
|
> details see the sections on [backwards incompatibilities](/usage/v3#incompat)
|
|
|
|
|
> and [migrating](/usage/v3#migrating). Also remember to download the new
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
> models, and retrain your own models.
|
|
|
|
|
|
|
|
|
|
When updating to a newer version of spaCy, it's generally recommended to start
|
|
|
|
|
with a clean virtual environment. If you're upgrading to a new major version,
|
|
|
|
|
make sure you have the latest **compatible models** installed, and that there
|
2020-07-01 22:26:39 +03:00
|
|
|
|
are no old and incompatible model packages left over in your environment, as
|
|
|
|
|
this can often lead to unexpected results and errors. If you've trained your own
|
|
|
|
|
models, keep in mind that your train and runtime inputs must match. This means
|
|
|
|
|
you'll have to **retrain your models** with the new version.
|
|
|
|
|
|
|
|
|
|
spaCy also provides a [`validate`](/api/cli#validate) command, which lets you
|
|
|
|
|
verify that all installed models are compatible with your spaCy version. If
|
|
|
|
|
incompatible models are found, tips and installation instructions are printed.
|
|
|
|
|
The command is also useful to detect out-of-sync model links resulting from
|
|
|
|
|
links created in different virtual environments. It's recommended to run the
|
|
|
|
|
command with `python -m` to make sure you're executing the correct version of
|
|
|
|
|
spaCy.
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
2020-08-19 01:28:37 +03:00
|
|
|
|
```cli
|
|
|
|
|
$ pip install -U spacy
|
|
|
|
|
$ python -m spacy validate
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Run spaCy with GPU {#gpu new="2.0.14"}
|
|
|
|
|
|
|
|
|
|
As of v2.0, spaCy comes with neural network models that are implemented in our
|
|
|
|
|
machine learning library, [Thinc](https://github.com/explosion/thinc). For GPU
|
|
|
|
|
support, we've been grateful to use the work of Chainer's
|
|
|
|
|
[CuPy](https://cupy.chainer.org) module, which provides a numpy-compatible
|
|
|
|
|
interface for GPU arrays.
|
|
|
|
|
|
|
|
|
|
spaCy can be installed on GPU by specifying `spacy[cuda]`, `spacy[cuda90]`,
|
2020-04-29 13:51:12 +03:00
|
|
|
|
`spacy[cuda91]`, `spacy[cuda92]`, `spacy[cuda100]`, `spacy[cuda101]` or
|
|
|
|
|
`spacy[cuda102]`. If you know your cuda version, using the more explicit
|
|
|
|
|
specifier allows cupy to be installed via wheel, saving some compilation time.
|
|
|
|
|
The specifiers should install [`cupy`](https://cupy.chainer.org).
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
$ pip install -U spacy[cuda92]
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Once you have a GPU-enabled installation, the best way to activate it is to call
|
|
|
|
|
[`spacy.prefer_gpu`](/api/top-level#spacy.prefer_gpu) or
|
|
|
|
|
[`spacy.require_gpu()`](/api/top-level#spacy.require_gpu) somewhere in your
|
|
|
|
|
script before any models have been loaded. `require_gpu` will raise an error if
|
|
|
|
|
no GPU is available.
|
|
|
|
|
|
|
|
|
|
```python
|
|
|
|
|
import spacy
|
|
|
|
|
|
|
|
|
|
spacy.prefer_gpu()
|
|
|
|
|
nlp = spacy.load("en_core_web_sm")
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
### Compile from source {#source}
|
|
|
|
|
|
|
|
|
|
The other way to install spaCy is to clone its
|
|
|
|
|
[GitHub repository](https://github.com/explosion/spaCy) and build it from
|
|
|
|
|
source. That is the common way if you want to make changes to the code base.
|
|
|
|
|
You'll need to make sure that you have a development environment consisting of a
|
|
|
|
|
Python distribution including header files, a compiler,
|
|
|
|
|
[pip](https://pip.pypa.io/en/latest/installing/),
|
|
|
|
|
[virtualenv](https://virtualenv.pypa.io/) and [git](https://git-scm.com)
|
|
|
|
|
installed. The compiler part is the trickiest. How to do that depends on your
|
|
|
|
|
system. See notes on [Ubuntu](#source-ubuntu), [macOS / OS X](#source-osx) and
|
|
|
|
|
[Windows](#source-windows) for details.
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ python -m pip install -U pip # update pip
|
|
|
|
|
$ git clone https://github.com/explosion/spaCy # clone spaCy
|
|
|
|
|
$ cd spaCy # navigate into dir
|
|
|
|
|
|
|
|
|
|
$ python -m venv .env # create environment in .env
|
|
|
|
|
$ source .env/bin/activate # activate virtual env
|
|
|
|
|
$ export PYTHONPATH=`pwd` # set Python path to spaCy dir
|
|
|
|
|
$ pip install -r requirements.txt # install all requirements
|
|
|
|
|
$ python setup.py build_ext --inplace # compile spaCy
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Compared to regular install via pip, the
|
|
|
|
|
[`requirements.txt`](https://github.com/explosion/spaCy/tree/master/requirements.txt)
|
2020-08-21 14:49:18 +03:00
|
|
|
|
additionally installs developer dependencies such as Cython. See the
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
[quickstart widget](#quickstart) to get the right commands for your platform and
|
|
|
|
|
Python version.
|
|
|
|
|
|
|
|
|
|
#### Ubuntu {#source-ubuntu}
|
|
|
|
|
|
|
|
|
|
Install system-level dependencies via `apt-get`:
|
|
|
|
|
|
|
|
|
|
```bash
|
|
|
|
|
$ sudo apt-get install build-essential python-dev git
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
#### macOS / OS X {#source-osx}
|
|
|
|
|
|
|
|
|
|
Install a recent version of [XCode](https://developer.apple.com/xcode/),
|
|
|
|
|
including the so-called "Command Line Tools". macOS and OS X ship with Python
|
|
|
|
|
and git preinstalled.
|
|
|
|
|
|
|
|
|
|
#### Windows {#source-windows}
|
|
|
|
|
|
|
|
|
|
Install a version of the
|
|
|
|
|
[Visual C++ Build Tools](https://visualstudio.microsoft.com/visual-cpp-build-tools/)
|
|
|
|
|
or
|
|
|
|
|
[Visual Studio Express](https://www.visualstudio.com/vs/visual-studio-express/)
|
2019-12-22 03:53:56 +03:00
|
|
|
|
that matches the version that was used to compile your Python interpreter.
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
2019-03-14 19:56:53 +03:00
|
|
|
|
### Run tests {#run-tests}
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
spaCy comes with an
|
2019-09-29 18:32:12 +03:00
|
|
|
|
[extensive test suite](https://github.com/explosion/spaCy/tree/master/spacy/tests).
|
|
|
|
|
In order to run the tests, you'll usually want to clone the
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
[repository](https://github.com/explosion/spaCy/tree/master/) and
|
|
|
|
|
[build spaCy from source](#source). This will also install the required
|
|
|
|
|
development dependencies and test utilities defined in the `requirements.txt`.
|
|
|
|
|
|
|
|
|
|
Alternatively, you can find out where spaCy is installed and run `pytest` on
|
|
|
|
|
that directory. Don't forget to also install the test utilities via spaCy's
|
|
|
|
|
[`requirements.txt`](https://github.com/explosion/spaCy/tree/master/requirements.txt):
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ python -c "import os; import spacy; print(os.path.dirname(spacy.__file__))"
|
|
|
|
|
$ pip install -r path/to/requirements.txt
|
|
|
|
|
$ python -m pytest [spacy directory]
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
Calling `pytest` on the spaCy directory will run only the basic tests. The flag
|
|
|
|
|
`--slow` is optional and enables additional tests that take longer.
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ python -m pip install -U pytest # update pytest
|
|
|
|
|
$ python -m pytest [spacy directory] # basic tests
|
|
|
|
|
$ python -m pytest [spacy directory] --slow # basic and slow tests
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Troubleshooting guide {#troubleshooting}
|
|
|
|
|
|
|
|
|
|
This section collects some of the most common errors you may come across when
|
|
|
|
|
installing, loading and using spaCy, as well as their solutions.
|
|
|
|
|
|
|
|
|
|
> #### Help us improve this guide
|
|
|
|
|
>
|
|
|
|
|
> Did you come across a problem like the ones listed here and want to share the
|
|
|
|
|
> solution? You can find the "Suggest edits" button at the bottom of this page
|
|
|
|
|
> that points you to the source. We always appreciate
|
|
|
|
|
> [pull requests](https://github.com/explosion/spaCy/pulls)!
|
|
|
|
|
|
|
|
|
|
<Accordion title="No compatible model found" id="compatible-model">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
No compatible model found for [lang] (spaCy vX.X.X).
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This usually means that the model you're trying to download does not exist, or
|
|
|
|
|
isn't available for your version of spaCy. Check the
|
|
|
|
|
[compatibility table](https://github.com/explosion/spacy-models/tree/master/compatibility.json)
|
|
|
|
|
to see which models are available for your spaCy version. If you're using an old
|
|
|
|
|
version, consider upgrading to the latest release. Note that while spaCy
|
|
|
|
|
supports tokenization for [a variety of languages](/usage/models#languages), not
|
|
|
|
|
all of them come with statistical models. To only use the tokenizer, import the
|
|
|
|
|
language's `Language` class instead, for example
|
|
|
|
|
`from spacy.lang.fr import French`.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="No such option: --no-cache-dir" id="no-cache-dir">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
no such option: --no-cache-dir
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
The `download` command uses pip to install the models and sets the
|
|
|
|
|
`--no-cache-dir` flag to prevent it from requiring too much memory.
|
|
|
|
|
[This setting](https://pip.pypa.io/en/stable/reference/pip_install/#caching)
|
|
|
|
|
requires pip v6.0 or newer. Run `pip install -U pip` to upgrade to the latest
|
|
|
|
|
version of pip. To see which version you have installed, run `pip --version`.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
2019-03-19 12:27:02 +03:00
|
|
|
|
<Accordion title="sre_constants.error: bad character range" id="narrow-unicode">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
sre_constants.error: bad character range
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
In [v2.1](/usage/v2-1), spaCy changed its implementation of regular expressions
|
|
|
|
|
for tokenization to make it up to 2-3 times faster. But this also means that
|
|
|
|
|
it's very important now that you run spaCy with a wide unicode build of Python.
|
|
|
|
|
This means that the build has 1114111 unicode characters available, instead of
|
|
|
|
|
only 65535 in a narrow unicode build. You can check this by running the
|
|
|
|
|
following command:
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ python -c "import sys; print(sys.maxunicode)"
|
2019-03-19 12:27:02 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
If you're running a narrow unicode build, reinstall Python and use a wide
|
|
|
|
|
unicode build instead. You can also rebuild Python and set the
|
|
|
|
|
`--enable-unicode=ucs4` flag.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
<Accordion title="Unknown locale: UTF-8" id="unknown-locale">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
ValueError: unknown locale: UTF-8
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This error can sometimes occur on OSX and is likely related to a still
|
|
|
|
|
unresolved [Python bug](https://bugs.python.org/issue18378). However, it's easy
|
|
|
|
|
to fix: just add the following to your `~/.bash_profile` or `~/.zshrc` and then
|
|
|
|
|
run `source ~/.bash_profile` or `source ~/.zshrc`. Make sure to add **both
|
|
|
|
|
lines** for `LC_ALL` and `LANG`.
|
|
|
|
|
|
|
|
|
|
```bash
|
2020-08-19 01:28:37 +03:00
|
|
|
|
$ export LC_ALL=en_US.UTF-8
|
|
|
|
|
$ export LANG=en_US.UTF-8
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="Import error: No module named spacy" id="import-error">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
Import Error: No module named spacy
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This error means that the spaCy module can't be located on your system, or in
|
|
|
|
|
your environment. Make sure you have spaCy installed. If you're using a virtual
|
|
|
|
|
environment, make sure it's activated and check that spaCy is installed in that
|
|
|
|
|
environment – otherwise, you're trying to load a system installation. You can
|
|
|
|
|
also run `which python` to find out where your Python executable is located.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="Import error: No module named [model]" id="import-error-models">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
ImportError: No module named 'en_core_web_sm'
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
As of spaCy v1.7, all models can be installed as Python packages. This means
|
2020-07-01 22:26:39 +03:00
|
|
|
|
that they'll become importable modules of your application. If this fails, it's
|
|
|
|
|
usually a sign that the package is not installed in the current environment. Run
|
|
|
|
|
`pip list` or `pip freeze` to check which model packages you have installed, and
|
|
|
|
|
install the [correct models](/models) if necessary. If you're importing a model
|
|
|
|
|
manually at the top of a file, make sure to use the name of the package, not the
|
|
|
|
|
shortcut link you've created.
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="Command not found: spacy" id="command-not-found">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
command not found: spacy
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
This error may occur when running the `spacy` command from the command line.
|
|
|
|
|
spaCy does not currently add an entry to your `PATH` environment variable, as
|
|
|
|
|
this can lead to unexpected results, especially when using a virtual
|
|
|
|
|
environment. Instead, spaCy adds an auto-alias that maps `spacy` to
|
|
|
|
|
`python -m spacy]`. If this is not working as expected, run the command with
|
2019-03-17 13:48:18 +03:00
|
|
|
|
`python -m`, yourself – for example `python -m spacy download en_core_web_sm`.
|
|
|
|
|
For more info on this, see the [`download`](/api/cli#download) command.
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="'module' object has no attribute 'load'" id="module-load">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
AttributeError: 'module' object has no attribute 'load'
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
While this could technically have many causes, including spaCy being broken, the
|
|
|
|
|
most likely one is that your script's file or directory name is "shadowing" the
|
|
|
|
|
module – e.g. your file is called `spacy.py`, or a directory you're importing
|
|
|
|
|
from is called `spacy`. So, when using spaCy, never call anything else `spacy`.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
2020-08-21 14:49:18 +03:00
|
|
|
|
<Accordion title="NER model doesn't recognize other entities anymore after training" id="catastrophic-forgetting">
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
If your training data only contained new entities and you didn't mix in any
|
|
|
|
|
examples the model previously recognized, it can cause the model to "forget"
|
|
|
|
|
what it had previously learned. This is also referred to as the "catastrophic
|
|
|
|
|
forgetting problem". A solution is to pre-label some text, and mix it with the
|
|
|
|
|
new text in your updates. You can also do this by running spaCy over some text,
|
|
|
|
|
extracting a bunch of entities the model previously recognized correctly, and
|
|
|
|
|
adding them to your training examples.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
|
|
|
|
<Accordion title="Unhashable type: 'list'" id="unhashable-list">
|
|
|
|
|
|
|
|
|
|
```
|
|
|
|
|
TypeError: unhashable type: 'list'
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
If you're training models, writing them to disk, and versioning them with git,
|
|
|
|
|
you might encounter this error when trying to load them in a Windows
|
|
|
|
|
environment. This happens because a default install of Git for Windows is
|
|
|
|
|
configured to automatically convert Unix-style end-of-line characters (LF) to
|
|
|
|
|
Windows-style ones (CRLF) during file checkout (and the reverse when
|
|
|
|
|
committing). While that's mostly fine for text files, a trained model written to
|
|
|
|
|
disk has some binary files that should not go through this conversion. When they
|
|
|
|
|
do, you get the error above. You can fix it by either changing your
|
|
|
|
|
[`core.autocrlf`](https://git-scm.com/book/en/v2/Customizing-Git-Git-Configuration)
|
|
|
|
|
setting to `"false"`, or by committing a
|
|
|
|
|
[`.gitattributes`](https://git-scm.com/docs/gitattributes) file] to your
|
|
|
|
|
repository to tell git on which files or folders it shouldn't do LF-to-CRLF
|
|
|
|
|
conversion, with an entry like `path/to/spacy/model/** -text`. After you've done
|
|
|
|
|
either of these, clone your repository again.
|
|
|
|
|
|
|
|
|
|
</Accordion>
|
|
|
|
|
|
2019-03-14 19:56:53 +03:00
|
|
|
|
## Changelog {#changelog}
|
💫 Update website (#3285)
<!--- Provide a general summary of your changes in the title. -->
## Description
The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on.
This PR also includes various new docs pages and content.
Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837.
### Types of change
enhancement
## Checklist
<!--- Before you submit the PR, go over this checklist and make sure you can
tick off all the boxes. [] -> [x] -->
- [x] I have submitted the spaCy Contributor Agreement.
- [x] I ran the tests, and all new and existing tests passed.
- [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
2019-02-17 21:31:19 +03:00
|
|
|
|
|
|
|
|
|
import Changelog from 'widgets/changelog.js'
|
|
|
|
|
|
|
|
|
|
<Changelog />
|