e597110d31
<!--- Provide a general summary of your changes in the title. --> ## Description The new website is implemented using [Gatsby](https://www.gatsbyjs.org) with [Remark](https://github.com/remarkjs/remark) and [MDX](https://mdxjs.com/). This allows authoring content in **straightforward Markdown** without the usual limitations. Standard elements can be overwritten with powerful [React](http://reactjs.org/) components and wherever Markdown syntax isn't enough, JSX components can be used. Hopefully, this update will also make it much easier to contribute to the docs. Once this PR is merged, I'll implement auto-deployment via [Netlify](https://netlify.com) on a specific branch (to avoid building the website on every PR). There's a bunch of other cool stuff that the new setup will allow us to do – including writing front-end tests, service workers, offline support, implementing a search and so on. This PR also includes various new docs pages and content. Resolves #3270. Resolves #3222. Resolves #2947. Resolves #2837. ### Types of change enhancement ## Checklist <!--- Before you submit the PR, go over this checklist and make sure you can tick off all the boxes. [] -> [x] --> - [x] I have submitted the spaCy Contributor Agreement. - [x] I ran the tests, and all new and existing tests passed. - [x] My changes don't require a change to the documentation, or if they do, I've added all required information.
7.2 KiB
| title | teaser | menu | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Examples | Full code examples you can modify and run |
|
Information Extraction
Using spaCy's phrase matcher
This example shows how to use the new PhraseMatcher to
efficiently find entities from a large terminology list.
https://github.com/explosion/spaCy/tree/master/examples/information_extraction/phrase_matcher.py
Extracting entity relations
A simple example of extracting relations between phrases and entities using
spaCy's named entity recognizer and the dependency parse. Here, we extract money
and currency values (entities labelled as MONEY) and then check the dependency
tree to find the noun phrase they are referring to – for example:
"$9.4 million" → "Net income".
https://github.com/explosion/spaCy/tree/master/examples/information_extraction/entity_relations.py
Navigating the parse tree and subtrees
This example shows how to navigate the parse tree including subtrees attached to a word.
https://github.com/explosion/spaCy/tree/master/examples/information_extraction/parse_subtrees.py
Pipeline
Custom pipeline components and attribute extensions
This example shows the implementation of a pipeline component that sets entity
annotations based on a list of single or multiple-word company names, merges
entities into one token and sets custom attributes on the Doc, Span and
Token.
https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_component_entities.py
Custom pipeline components and attribute extensions via a REST API
This example shows the implementation of a pipeline component that fetches
country meta data via the REST Countries API sets
entity annotations for countries, merges entities into one token and sets custom
attributes on the Doc, Span and Token – for example, the capital,
latitude/longitude coordinates and the country flag.
https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_component_countries_api.py
Custom method extensions
A collection of snippets showing examples of extensions adding custom methods to
the Doc, Token and Span.
https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_attr_methods.py
Multi-processing with Joblib
This example shows how to use multiple cores to process text using spaCy and Joblib. We're exporting part-of-speech-tagged, true-cased, (very roughly) sentence-separated text, with each "sentence" on a newline, and spaces between tokens. Data is loaded from the IMDB movie reviews dataset and will be loaded automatically via Thinc's built-in dataset loader.
https://github.com/explosion/spaCy/tree/master/examples/pipeline/multi_processing.py
Training
Training spaCy's Named Entity Recognizer
This example shows how to update spaCy's entity recognizer with your own
examples, starting off with an existing, pre-trained model, or from scratch
using a blank Language class.
https://github.com/explosion/spaCy/tree/master/examples/training/train_ner.py
Training an additional entity type
This script shows how to add a new entity type to an existing pre-trained NER model. To keep the example short and simple, only four sentences are provided as examples. In practice, you'll need many more — a few hundred would be a good start.
https://github.com/explosion/spaCy/tree/master/examples/training/train_new_entity_type.py
Training spaCy's Dependency Parser
This example shows how to update spaCy's dependency parser, starting off with an
existing, pre-trained model, or from scratch using a blank Language class.
https://github.com/explosion/spaCy/tree/master/examples/training/train_parser.py
Training spaCy's Part-of-speech Tagger
In this example, we're training spaCy's part-of-speech tagger with a custom tag map, mapping our own tags to the mapping those tags to the Universal Dependencies scheme.
https://github.com/explosion/spaCy/tree/master/examples/training/train_tagger.py
Training a custom parser for chat intent semantics
spaCy's parser component can be used to trained to predict any type of tree
structure over your input text. You can also predict trees over whole documents
or chat logs, with connections between the sentence-roots used to annotate
discourse structure. In this example, we'll build a message parser for a common
"chat intent": finding local businesses. Our message semantics will have the
following types of relations: ROOT, PLACE, QUALITY, ATTRIBUTE, TIME
and LOCATION.
https://github.com/explosion/spaCy/tree/master/examples/training/train_intent_parser.py
Training spaCy's text classifier
This example shows how to train a multi-label convolutional neural network text
classifier on IMDB movie reviews, using spaCy's new
TextCategorizer component. The dataset will be loaded
automatically via Thinc's built-in dataset loader. Predictions are available via
Doc.cats.
https://github.com/explosion/spaCy/tree/master/examples/training/train_textcat.py
Vectors
Visualizing spaCy vectors in TensorBoard
These two scripts let you load any spaCy model containing word vectors into TensorBoard to create an embedding visualization. The first example uses TensorBoard, the second example TensorBoard's standalone embedding projector.
https://github.com/explosion/spaCy/tree/master/examples/vectors_tensorboard.py
https://github.com/explosion/spaCy/tree/master/examples/vectors_tensorboard_standalone.py
Deep Learning
Text classification with Keras
This example shows how to use a Keras LSTM sentiment classification model in spaCy. spaCy splits the document into sentences, and each sentence is classified using the LSTM. The scores for the sentences are then aggregated to give the document score. This kind of hierarchical model is quite difficult in "pure" Keras or TensorFlow, but it's very effective. The Keras example on this dataset performs quite poorly, because it cuts off the documents so that they're a fixed size. This hurts review accuracy a lot, because people often summarize their rating in the final sentence.
https://github.com/explosion/spaCy/tree/master/examples/deep_learning_keras.py