spaCy/website

spacy.io website and docs

This page contains the documentation and styleguide for the spaCy website. Its rendered version is available at https://spacy.io/styleguide.

---

The spacy.io website is implemented using Gatsby with Remark and MDX. This allows authoring content in straightforward Markdown without the usual limitations. Standard elements can be overwritten with powerful React components and wherever Markdown syntax isn't enough, JSX components can be used.

Contributing to the site

The docs can always use another example or more detail, and they should always be up to date and not misleading. We always appreciate a pull request. To quickly find the correct file to edit, simply click on the "Suggest edits" button at the bottom of a page.

For more details on editing the site locally, see the installation instructions and markdown reference below.

Logo

import { Logos } from 'widgets/styleguide'

If you would like to use the spaCy logo on your site, please get in touch and ask us first. However, if you want to show support and tell others that your project is using spaCy, you can grab one of our spaCy badges.

Colors

import { Colors, Patterns } from 'widgets/styleguide'

Patterns

Typography

import { H1, H2, H3, H4, H5, Label, InlineList, Comment } from 'components/typography'

Markdown

## Headline 2
## Headline 2 {#some_id}
## Headline 2 {#some_id tag="method"}

JSX

<H2>Headline 2</H2>
<H2 id="some_id">Headline 2</H2>
<H2 id="some_id" tag="method">Headline 2</H2>

Headlines are set in HK Grotesk by Hanken Design. All other body text and code uses the best-matching default system font to provide a "native" reading experience. All code uses the JetBrains Mono typeface by JetBrains.

Level 2 headings are automatically wrapped in <section> elements at compile time, using a custom Markdown transformer. This makes it easier to highlight the section that's currently in the viewpoint in the sidebar menu.

Headline 1

Headline 2

Headline 3

Headline 4

Headline 5

Label

---

The following optional attributes can be set on the headline to modify it. For example, to add a tag for the documented type or mark features that have been introduced in a specific version or require statistical models to be loaded. Tags are also available as standalone <Tag /> components.

Argument	Example	Result
tag	{tag="method"}	method
new	{new="3"}	3
model	{model="tagger, parser"}	tagger, parser
hidden	{hidden="true"}

Elements

Links

Markdown

[I am a link](https://spacy.io)

JSX

<Link to="https://spacy.io">I am a link</Link>

Special link styles are used depending on the link URL.

• I am a regular external link
• I am a link to the documentation
• I am a link to an architecture
• I am a link to a model
• I am a link to GitHub

Abbreviations

import { Abbr } from 'components/typography'

JSX

<Abbr title="Explanation">Abbreviation</Abbr>

Some text with an abbreviation. On small screens, I collapse and the explanation text is displayed next to the abbreviation.

Tags

import Tag from 'components/tag'

<Tag>method</Tag>
<Tag variant="new">4</Tag>
<Tag variant="model">tagger, parser</Tag>

Tags can be used together with headlines, or next to properties across the documentation, and combined with tooltips to provide additional information. An optional variant argument can be used for special tags. variant="new" makes the tag take a version number to mark new features. Using the component, visibility of this tag can later be toggled once the feature isn't considered new anymore. Setting variant="model" takes a description of model capabilities and can be used to mark features that require a respective model to be installed.

method 4 tagger, parser

Buttons

import Button from 'components/button'

<Button to="#" variant="primary">Primary small</Button>
<Button to="#" variant="secondary">Secondary small</Button>

Link buttons come in two variants, primary and secondary and two sizes, with an optional large size modifier. Since they're mostly used as enhanced links, the buttons are implemented as styled links instead of native button elements.

Primary small Secondary small

Primary large Secondary large

Components

Table

Markdown

| Header 1 | Header 2 |
| -------- | -------- |
| Column 1 | Column 2 |

JSX

<Table>
    <Tr><Th>Header 1</Th><Th>Header 2</Th></Tr></thead>
    <Tr><Td>Column 1</Td><Td>Column 2</Td></Tr>
</Table>

Tables are used to present data and API documentation. Certain keywords can be used to mark a footer row with a distinct style, for example to visualize the return values of a documented function.

Header 1	Header 2	Header 3	Header 4
Column 1	Column 2	Column 3	Column 4
Column 1	Column 2	Column 3	Column 4
Column 1	Column 2	Column 3	Column 4
Column 1	Column 2	Column 3	Column 4
RETURNS	Column 2	Column 3	Column 4

Tables also support optional "divider" rows that are typically used to denote keyword-only arguments in API documentation. To turn a row into a dividing headline, it should only include content in its first cell, and its value should be italicized:

Markdown

| Header 1 | Header 2 | Header 3 |
| -------- | -------- | -------- |
| Column 1 | Column 2 | Column 3 |
| _Hello_  |          |          |
| Column 1 | Column 2 | Column 3 |

Header 1	Header 2	Header 3
Column 1	Column 2	Column 3
Hello
Column 1	Column 2	Column 3

Type Annotations

Markdown

~~Model[List[Doc], Floats2d]~~

JSX

<TypeAnnotation>Model[List[Doc], Floats2d]</Typeannotation>

Type annotations are special inline code blocks are used to describe Python types in the type hints format. The special component will split the type, apply syntax highlighting and link all types that specify links in meta/type-annotations.json. Types can link to internal or external documentation pages. To make it easy to represent the type annotations in Markdown, the rendering "hijacks" the ~~ tags that would typically be converted to a <del> element – but in this case, text surrounded by ~~ becomes a type annotation.

• Dict[str, List[Union[Doc, Span]]]
• Model[List[Doc], List[numpy.ndarray]]

Type annotations support a special visual style in tables and will render as a separate row, under the cell text. This allows the API docs to display complex types without taking up too much space in the cell. The type annotation should always be the last element in the row.

Markdown

| Header 1 | Header 2                |
| -------- | ----------------------- |
| Column 1 | Column 2 ~~List[Doc]~~ |

Name	Description
vocab	The shared vocabulary. Vocab
model	The Thinc Model wrapping the transformer. Model[List[Doc], FullTransformerBatch]
set_extra_annotations	Function that takes a batch of Doc objects and transformer outputs and can set additional annotations on the Doc. Callable[[List[Doc], FullTransformerBatch], None]

List

Markdown

1. One
2. Two

JSX

<Ol>
    <Li>One</Li>
    <Li>Two</Li>
</Ol>

Lists are available as bulleted and numbered. Markdown lists are transformed automatically.

• I am a bulleted list
• I have nice bullets
• Lorem ipsum dolor
• consectetur adipiscing elit

1. I am an ordered list
2. I have nice numbers
3. Lorem ipsum dolor
4. consectetur adipiscing elit

Aside

Markdown

> #### Aside title
> This is aside text.

JSX

<Aside title="Aside title">This is aside text.</Aside>

Asides can be used to display additional notes and content in the right-hand column. Asides can contain text, code and other elements if needed. Visually, asides are moved to the side on the X-axis, and displayed at the same level they were inserted. On small screens, they collapse and are rendered in their original position, in between the text.

To make them easier to use in Markdown, paragraphs formatted as blockquotes will turn into asides by default. Level 4 headlines (with a leading ####) will become aside titles.

Code Block

Markdown

```python
### This is a title
import spacy
```

JSX

<CodeBlock title="This is a title" lang="python">
  import spacy
</CodeBlock>

Code blocks use the Prism syntax highlighter with a custom theme. The language can be set individually on each block, and defaults to raw text with no highlighting. An optional label can be added as the first line with the prefix #### (Python-like) and /// (JavaScript-like). the indented block as plain text and preserve whitespace.

### Using spaCy
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("This is a sentence.")
for token in doc:
    print(token.text, token.pos_)

Code blocks and also specify an optional range of line numbers to highlight by adding {highlight="..."} to the headline. Acceptable ranges are spans like 5-7, but also 5-7,10 or 5-7,10,13-14.

Markdown

```python
### This is a title {highlight="1-2"}
import spacy
nlp = spacy.load("en_core_web_sm")
```

### Using the matcher {highlight="5-7"}
import spacy
from spacy.matcher import Matcher

nlp = spacy.load('en_core_web_sm')
matcher = Matcher(nlp.vocab)
pattern = [{"LOWER": "hello"}, {"IS_PUNCT": True}, {"LOWER": "world"}]
matcher.add("HelloWorld", None, pattern)
doc = nlp("Hello, world! Hello world!")
matches = matcher(doc)

Adding {executable="true"} to the title turns the code into an executable block, powered by Binder and Juniper. If JavaScript is disabled, the interactive widget defaults to a regular code block.

Markdown

```python
### {executable="true"}
import spacy
nlp = spacy.load("en_core_web_sm")
```

### {executable="true"}
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("This is a sentence.")
for token in doc:
    print(token.text, token.pos_)

If a code block only contains a URL to a GitHub file, the raw file contents are embedded automatically and syntax highlighting is applied. The link to the original file is shown at the top of the widget.

Markdown

```python
https://github.com/...
```

JSX

<GitHubCode url="https://github.com/..." lang="python" />

https://github.com/explosion/spaCy/tree/master/spacy/language.py

Infobox

import Infobox from 'components/infobox'

JSX

<Infobox title="Information">Regular infobox</Infobox>
<Infobox title="Important note" variant="warning">This is a warning.</Infobox>
<Infobox title="Be careful!" variant="danger">This is dangerous.</Infobox>

Infoboxes can be used to add notes, updates, warnings or additional information to a page or section. Semantically, they're implemented and interpreted as an aside element. Infoboxes can take an optional title argument, as well as an optional variant (either "warning" or "danger").

If needed, an infobox can contain regular text, inline code, lists and other blocks.

If needed, an infobox can contain regular text, inline code, lists and other blocks.

If needed, an infobox can contain regular text, inline code, lists and other blocks.

Accordion

import Accordion from 'components/accordion'

JSX

<Accordion title="This is an accordion">
  Accordion content goes here.
</Accordion>

Accordions are collapsible sections that are mostly used for lengthy tables, like the tag and label annotation schemes for different languages. They all need to be presented – but chances are the user doesn't actually care about all of them, especially not at the same time. So it's fairly reasonable to hide them begin a click. This particular implementation was inspired by the amazing Inclusive Components blog.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Quisque enim ante, pretium a orci eget, varius dignissim augue. Nam eu dictum mauris, id tincidunt nisi. Integer commodo pellentesque tincidunt. Nam at turpis finibus tortor gravida sodales tincidunt sit amet est. Nullam euismod arcu in tortor auctor, sit amet dignissim justo congue.

Setup and installation

Before running the setup, make sure your versions of Node and npm are up to date. Node v10.15 or later is required.

# Clone the repository
git clone https://github.com/explosion/spaCy
cd spaCy/website

# Install Gatsby's command-line tool
npm install --global gatsby-cli

# Install the dependencies
npm install

# Start the development server
npm run dev

If you are planning on making edits to the site, you should also set up the Prettier code formatter. It takes care of formatting Markdown and other files automatically. See here for the available extensions for your code editor. The .prettierrc file in the root defines the settings used in this codebase.

Building & developing the site with Docker

Sometimes it's hard to get a local environment working due to rapid updates to node dependencies, so it may be easier to use docker for building the docs.

If you'd like to do this, be sure you do not include your local node_modules folder, since there are some dependencies that need to be built for the image system. Rename it before using.

docker run -it \
  -v $(pwd):/spacy-io/website \
  -p 8000:8000 \
  ghcr.io/explosion/spacy-io \
  gatsby develop -H 0.0.0.0

This will allow you to access the built website at http://0.0.0.0:8000/ in your browser, and still edit code in your editor while having the site reflect those changes.

Note: If you're working on a Mac with an M1 processor, you might see segfault errors from qemu if you use the default image. To fix this use the arm64 tagged image in the docker run command (ghcr.io/explosion/spacy-io:arm64).

Building the Docker image

If you'd like to build the image locally, you can do so like this:

docker build -t spacy-io .

This will take some time, so if you want to use the prebuilt image you'll save a bit of time.

Markdown reference

All page content and page meta lives in the .md files in the /docs directory. The frontmatter block at the top of each file defines the page title and other settings like the sidebar menu.

---
title: Page title
---

## Headline starting a section {#some_id}

This is a regular paragraph with a [link](https://spacy.io) and **bold text**.

> #### This is an aside title
>
> This is aside text.

### Subheadline

| Header 1 | Header 2 |
| -------- | -------- |
| Column 1 | Column 2 |

```python
### Code block title {highlight="2-3"}
import spacy
nlp = spacy.load("en_core_web_sm")
doc = nlp("Hello world")
```

<Infobox title="Important note" variant="warning">

This is content in the infobox.

</Infobox>

In addition to the native markdown elements, you can use the components <Infobox />, <Accordion />, <Abbr /> and <Tag /> via their JSX syntax.

Project structure

### Directory structure
├── docs                 # the actual markdown content
├── meta                 # JSON-formatted site metadata
|   ├── languages.json   # supported languages and statistical models
|   ├── sidebars.json    # sidebar navigations for different sections
|   ├── site.json        # general site metadata
|   └── universe.json    # data for the spaCy universe section
├── public               # compiled site
├── src                  # source
|   ├── components       # React components
|   ├── fonts            # webfonts
|   ├── images           # images used in the layout
|   ├── plugins          # custom plugins to transform Markdown
|   ├── styles           # CSS modules and global styles
|   ├── templates        # page layouts
|   |   ├── docs.js      # layout template for documentation pages
|   |   ├── index.js     # global layout template
|   |   ├── models.js    # layout template for model pages
|   |   └── universe.js  # layout templates for universe
|   └── widgets          # non-reusable components with content, e.g. changelog
├── gatsby-browser.js    # browser-specific hooks for Gatsby
├── gatsby-config.js     # Gatsby configuration
├── gatsby-node.js       # Node-specific hooks for Gatsby
└── package.json         # package settings and dependencies

Editorial

• "spaCy" should always be spelled with a lowercase "s" and a capital "C", unless it specifically refers to the Python package or Python import spacy (in which case it should be formatted as code).
  • ✅ spaCy is a library for advanced NLP in Python.
  • ❌ Spacy is a library for advanced NLP in Python.
  • ✅ First, you need to install the spacy package from pip.
• Mentions of code, like function names, classes, variable names etc. in inline text should be formatted as code.
  • ✅ "Calling the nlp object on a text returns a Doc."
• Objects that have pages in the API docs should be linked – for example, Doc or Language.to_disk. The mentions should still be formatted as code within the link. Links pointing to the API docs will automatically receive a little icon. However, if a paragraph includes many references to the API, the links can easily get messy. In that case, we typically only link the first mention of an object and not any subsequent ones.
  • ✅ The Span and Token objects are views of a Doc. Span.as_doc creates a Doc object from a Span.
  • ❌ The Span and Token objects are views of a Doc. Span.as_doc creates a Doc object from a Span.

• Other things we format as code are: references to trained pipeline packages like en_core_web_sm or file names like code.py or meta.json.

  • ✅ After training, the config.cfg is saved to disk.

• Type annotations are a special type of code formatting, expressed by wrapping the text in ~~ instead of backticks. The result looks like this: List[Doc]. All references to known types will be linked automatically.

  • ✅ The model has the input type List[Doc] and it outputs a List[Array2d].

• We try to keep links meaningful but short.

  • ✅ For details, see the usage guide on training with custom code.
  • ❌ For details, see the usage guide on training with custom code.
  • ❌ For details, see the usage guide on training with custom code here.