spaCy/website/docs/usage/examples.md

title	teaser	menu
Examples	Full code examples you can modify and run	Information Extraction information-extraction Pipeline pipeline Training training Vectors & Similarity vectors Deep Learning deep-learning

Information Extraction

Using spaCy's phrase matcher

This example shows how to use the new PhraseMatcher to efficiently find entities from a large terminology list.

https://github.com/explosion/spaCy/tree/master/examples/information_extraction/phrase_matcher.py

Extracting entity relations

A simple example of extracting relations between phrases and entities using spaCy's named entity recognizer and the dependency parse. Here, we extract money and currency values (entities labelled as MONEY) and then check the dependency tree to find the noun phrase they are referring to – for example: "$9.4 million" → "Net income".

https://github.com/explosion/spaCy/tree/master/examples/information_extraction/entity_relations.py

Navigating the parse tree and subtrees

This example shows how to navigate the parse tree including subtrees attached to a word.

https://github.com/explosion/spaCy/tree/master/examples/information_extraction/parse_subtrees.py

Pipeline

Custom pipeline components and attribute extensions

This example shows the implementation of a pipeline component that sets entity annotations based on a list of single or multiple-word company names, merges entities into one token and sets custom attributes on the Doc, Span and Token.

https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_component_entities.py

Custom pipeline components and attribute extensions via a REST API

This example shows the implementation of a pipeline component that fetches country meta data via the REST Countries API sets entity annotations for countries, merges entities into one token and sets custom attributes on the Doc, Span and Token – for example, the capital, latitude/longitude coordinates and the country flag.

https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_component_countries_api.py

Custom method extensions

A collection of snippets showing examples of extensions adding custom methods to the Doc, Token and Span.

https://github.com/explosion/spaCy/tree/master/examples/pipeline/custom_attr_methods.py

Multi-processing with Joblib

This example shows how to use multiple cores to process text using spaCy and Joblib. We're exporting part-of-speech-tagged, true-cased, (very roughly) sentence-separated text, with each "sentence" on a newline, and spaces between tokens. Data is loaded from the IMDB movie reviews dataset and will be loaded automatically via Thinc's built-in dataset loader.

https://github.com/explosion/spaCy/tree/master/examples/pipeline/multi_processing.py

Training

Training spaCy's Named Entity Recognizer

This example shows how to update spaCy's entity recognizer with your own examples, starting off with an existing, pretrained model, or from scratch using a blank Language class.

https://github.com/explosion/spaCy/tree/master/examples/training/train_ner.py

Training an additional entity type

This script shows how to add a new entity type to an existing pretrained NER model. To keep the example short and simple, only four sentences are provided as examples. In practice, you'll need many more — a few hundred would be a good start.

https://github.com/explosion/spaCy/tree/master/examples/training/train_new_entity_type.py

Creating a Knowledge Base for Named Entity Linking

This example shows how to create a knowledge base in spaCy, which is needed to implement entity linking functionality. It requires as input a spaCy model with pretrained word vectors, and it stores the KB to file (if an output_dir is provided).

https://github.com/explosion/spaCy/tree/master/examples/training/create_kb.py

Training spaCy's Named Entity Linker

This example shows how to train spaCy's entity linker with your own custom examples, starting off with a predefined knowledge base and its vocab, and using a blank English class.

https://github.com/explosion/spaCy/tree/master/examples/training/train_entity_linker.py

Training spaCy's Dependency Parser

This example shows how to update spaCy's dependency parser, starting off with an existing, pretrained model, or from scratch using a blank Language class.

https://github.com/explosion/spaCy/tree/master/examples/training/train_parser.py

Training spaCy's Part-of-speech Tagger

In this example, we're training spaCy's part-of-speech tagger with a custom tag map, mapping our own tags to the mapping those tags to the Universal Dependencies scheme.

https://github.com/explosion/spaCy/tree/master/examples/training/train_tagger.py

Training a custom parser for chat intent semantics

spaCy's parser component can be used to trained to predict any type of tree structure over your input text. You can also predict trees over whole documents or chat logs, with connections between the sentence-roots used to annotate discourse structure. In this example, we'll build a message parser for a common "chat intent": finding local businesses. Our message semantics will have the following types of relations: ROOT, PLACE, QUALITY, ATTRIBUTE, TIME and LOCATION.

https://github.com/explosion/spaCy/tree/master/examples/training/train_intent_parser.py

Training spaCy's text classifier

This example shows how to train a multi-label convolutional neural network text classifier on IMDB movie reviews, using spaCy's new TextCategorizer component. The dataset will be loaded automatically via Thinc's built-in dataset loader. Predictions are available via Doc.cats.

https://github.com/explosion/spaCy/tree/master/examples/training/train_textcat.py

Vectors

Visualizing spaCy vectors in TensorBoard

This script lets you load any spaCy model containing word vectors into TensorBoard to create an embedding visualization.

https://github.com/explosion/spaCy/tree/master/examples/vectors_tensorboard.py

Deep Learning

Text classification with Keras

This example shows how to use a Keras LSTM sentiment classification model in spaCy. spaCy splits the document into sentences, and each sentence is classified using the LSTM. The scores for the sentences are then aggregated to give the document score. This kind of hierarchical model is quite difficult in "pure" Keras or TensorFlow, but it's very effective. The Keras example on this dataset performs quite poorly, because it cuts off the documents so that they're a fixed size. This hurts review accuracy a lot, because people often summarize their rating in the final sentence.

https://github.com/explosion/spaCy/tree/master/examples/deep_learning_keras.py