//- 💫 DOCS > USAGE > SPACY'S DATA MODEL
include ../../_includes/_mixins
p After reading this page, you should be able to:
+list
+item Understand how spaCy's Doc, Span, Token and Lexeme object work
+item Start using spaCy's Cython API
+item Use spaCy more efficiently
+h(2, "design-considerations") Design considerations
+h(3, "no-job-too-big") No job too big
p
| When writing spaCy, one of my mottos was #[em no job too big]. I wanted
| to make sure that if Google or Facebook were founded tomorrow, spaCy
| would be the obvious choice for them. I wanted spaCy to be the obvious
| choice for web-scale NLP. This meant sweating about performance, because
| for web-scale tasks, Moore's law can't save you.
p
| Most computational work gets less expensive over time. If you wrote a
| program to solve fluid dynamics in 2008, and you ran it again in 2014,
| you would expect it to be cheaper. For NLP, it often doesn't work out
| that way. The problem is that we're writing programs where the task is
| something like "Process all articles in the English Wikipedia". Sure,
| compute prices dropped from $0.80 per hour to $0.20 per hour on AWS in
| 2008-2014. But the size of Wikipedia grew from 3GB to 11GB. Maybe the
| job is a #[em little] cheaper in 2014 — but not by much.
+h(3, "annotation-layers") Multiple layers of annotation
p
| When I tell a certain sort of person that I'm a computational linguist,
| this comic is often the first thing that comes to their mind:
+image("http://i.imgur.com/n3DTzqx.png", 450)
+image-caption © #[+a("http://xkcd.com") xkcd]
p
| I've thought a lot about what this comic is really trying to say. It's
| probably not talking about #[em data models] — but in that sense at
| least, it really rings true.
p
| You'll often need to model a document as a sequence of sentences. Other
| times you'll need to model it as a sequence of words. Sometimes you'll
| care about paragraphs, other times you won't. Sometimes you'll care
| about extracting quotes, which can cross paragraph boundaries. A quote
| can also occur within a sentence. When we consider sentence structure,
| things get even more complicated and contradictory. We have syntactic
| trees, sequences of entities, sequences of phrases, sub-word units,
| multi-word units...
p
| Different applications are going to need to query different,
| overlapping, and often contradictory views of the document. They're
| often going to need to query them jointly. You need to be able to get
| the syntactic head of a named entity, or the sentiment of a paragraph.
+h(2, "solutions") Solutions
+h(3) Fat types, thin tokens
+h(3) Static model, dynamic views
p
| Different applications are going to need to query different,
| overlapping, and often contradictory views of the document. For this
| reason, I think it's a bad idea to have too much of the document
| structure reflected in the data model. If you structure the data
| according to the needs of one layer of annotation, you're going to need
| to copy the data and transform it in order to use a different layer of
| annotation. You'll soon have lots of copies, and no single source of
| truth.
+h(3) Never go full stand-off
+h(3) Implementation
+h(3) Cython 101
+h(3) #[code cdef class Doc]
p
| Let's start at the top. Here's the memory layout of the
| #[+api("doc") #[code Doc]] class, minus irrelevant details:
+code.
from cymem.cymem cimport Pool
from ..vocab cimport Vocab
from ..structs cimport TokenC
cdef class Doc:
cdef Pool mem
cdef Vocab vocab
cdef TokenC* c
cdef int length
cdef int max_length
p
| So, our #[code Doc] class is a wrapper around a TokenC* array — that's
| where the actual document content is stored. Here's the #[code TokenC]
| struct, in its entirety:
+h(3) #[code cdef struct TokenC]
+code.
cdef struct TokenC:
const LexemeC* lex
uint64_t morph
univ_pos_t pos
bint spacy
int tag
int idx
int lemma
int sense
int head
int dep
bint sent_start
uint32_t l_kids
uint32_t r_kids
uint32_t l_edge
uint32_t r_edge
int ent_iob
int ent_type # TODO: Is there a better way to do this? Multiple sources of truth..
hash_t ent_id
p
| The token owns all of its linguistic annotations, and holds a const
| pointer to a #[code LexemeC] struct. The #[code LexemeC] struct owns all
| of the #[em vocabulary] data about the word — all the dictionary
| definition stuff that we want to be shared by all instances of the type.
| Here's the #[code LexemeC] struct, in its entirety:
+h(3) #[code cdef struct LexemeC]
+code.
cdef struct LexemeC:
int32_t id
int32_t orth # Allows the string to be retrieved
int32_t length # Length of the string
uint64_t flags # These are the most useful parts.
int32_t cluster # Distributional similarity cluster
float prob # Probability
float sentiment # Slot for sentiment
int32_t lang
int32_t lower # These string views made sense
int32_t norm # when NLP meant linear models.
int32_t shape # Now they're less relevant, and
int32_t prefix # will probably be revised.
int32_t suffix
float* vector # <-- This was a design mistake, and will change.
+h(2, "dynamic-views") Dynamic views
+h(3) Text
p
| You might have noticed that in all of the structs above, there's not a
| string to be found. The strings are all stored separately, in the
| #[+api("stringstore") #[code StringStore]] class. The lexemes don't know
| the strings — they only know their integer IDs. The document string is
| never stored anywhere, either. Instead, it's reconstructed by iterating
| over the tokens, which look up the #[code orth] attribute of their
| underlying lexeme. Once we have the orth ID, we can fetch the string
| from the vocabulary. Finally, each token knows whether a single
| whitespace character (#[code ' ']) should be used to separate it from
| the subsequent tokens. This allows us to preserve whitespace.
+code.
cdef print_text(Vocab vocab, const TokenC* tokens, int length):
for i in range(length):
word_string = vocab.strings[tokens.lex.orth]
if tokens.lex.spacy:
word_string += ' '
print(word_string)
p
| This is why you get whitespace tokens in spaCy — we need those tokens,
| so that we can reconstruct the document string. I also think you should
| have those tokens anyway. Most NLP libraries strip them, making it very
| difficult to recover the paragraph information once you're at the token
| level. You'll never have that sort of problem with spaCy — because
| there's a single source of truth.
+h(3) #[code cdef class Token]
p When you do...
+code.
doc[i]
p
| ...you get back an instance of class #[code spacy.tokens.token.Token].
| This instance owns no data. Instead, it holds the information
| #[code (doc, i)], and uses these to retrieve all information via the
| parent container.
+h(3) #[code cdef class Span]
p When you do...
+code.
doc[i : j]
p
| ...you get back an instance of class #[code spacy.tokens.span.Span].
| #[code Span] instances are also returned by the #[code .sents],
| #[code .ents] and #[code .noun_chunks] iterators of the #[code Doc]
| object. A #[code Span] is a slice of tokens, with an optional label
| attached. Its data model is:
+code.
cdef class Span:
cdef readonly Doc doc
cdef int start
cdef int end
cdef int start_char
cdef int end_char
cdef int label
p
| Once again, the #[code Span] owns almost no data. Instead, it refers
| back to the parent #[code Doc] container.
p
| The #[code start] and #[code end] attributes refer to token positions,
| while #[code start_char] and #[code end_char] record the character
| positions of the span. By recording the character offsets, we can still
| use the #[code Span] object if the tokenization of the document changes.
+h(3) #[code cdef class Lexeme]
p When you do...
+code.
vocab[u'the']
p
| ...you get back an instance of class #[code spacy.lexeme.Lexeme]. The
| #[code Lexeme]'s data model is:
+code.
cdef class Lexeme:
cdef LexemeC* c
cdef readonly Vocab vocab