mirror of
https://github.com/explosion/spaCy.git
synced 2025-01-12 18:26:30 +03:00
Update docs for Token.morph / Token.set_morph
This commit is contained in:
Adriane Boyd
parent
65dfaa4f4b
commit
fd09e6b140
|
|
@ -172,6 +172,25 @@ Get a neighboring token.
|
|||
| `i` | The relative position of the token to get. Defaults to `1`. ~~int~~ |
|
||||
| **RETURNS** | The token at position `self.doc[self.i+i]`. ~~Token~~ |
|
||||
|
||||
## Token.set_morph {#set_morph tag="method"}
|
||||
|
||||
Set the morphological analysis from a UD FEATS string, hash value of a UD FEATS
|
||||
string, features dict or `MorphAnalysis`. The value `None` can be used to reset
|
||||
the morph to an unset state.
|
||||
|
||||
> #### Example
|
||||
>
|
||||
> ```python
|
||||
> doc = nlp("Give it back! He pleaded.")
|
||||
> doc[0].set_morph("Mood=Imp|VerbForm=Fin")
|
||||
> assert "Mood=Imp" in doc[0].morph
|
||||
> assert doc[0].morph.get("Mood") == ["Imp"]
|
||||
> ```
|
||||
|
||||
| Name | Description |
|
||||
| -------- | --------------------------------------------------------------------------------- |
|
||||
| features | The morphological features to set. ~~Union[int, dict, str, MorphAnalysis, None]~~ |
|
||||
|
||||
## Token.is_ancestor {#is_ancestor tag="method" model="parser"}
|
||||
|
||||
Check whether this token is a parent, grandparent, etc. of another in the
|
||||
|
|
@ -392,74 +411,73 @@ The L2 norm of the token's vector representation.
|
|||
|
||||
## Attributes {#attributes}
|
||||
|
||||
| Name | Description |
|
||||
| -------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `doc` | The parent document. ~~Doc~~ |
|
||||
| `lex` <Tag variant="new">3</Tag> | The underlying lexeme. ~~Lexeme~~ |
|
||||
| `sent` <Tag variant="new">2.0.12</Tag> | The sentence span that this token is a part of. ~~Span~~ |
|
||||
| `text` | Verbatim text content. ~~str~~ |
|
||||
| `text_with_ws` | Text content, with trailing space character if present. ~~str~~ |
|
||||
| `whitespace_` | Trailing space character if present. ~~str~~ |
|
||||
| `orth` | ID of the verbatim text content. ~~int~~ |
|
||||
| `orth_` | Verbatim text content (identical to `Token.text`). Exists mostly for consistency with the other attributes. ~~str~~ |
|
||||
| `vocab` | The vocab object of the parent `Doc`. ~~vocab~~ |
|
||||
| `tensor` <Tag variant="new">2.1.7</Tag> | The tokens's slice of the parent `Doc`'s tensor. ~~numpy.ndarray~~ |
|
||||
| `head` | The syntactic parent, or "governor", of this token. ~~Token~~ |
|
||||
| `left_edge` | The leftmost token of this token's syntactic descendants. ~~Token~~ |
|
||||
| `right_edge` | The rightmost token of this token's syntactic descendants. ~~Token~~ |
|
||||
| `i` | The index of the token within the parent document. ~~int~~ |
|
||||
| `ent_type` | Named entity type. ~~int~~ |
|
||||
| `ent_type_` | Named entity type. ~~str~~ |
|
||||
| `ent_iob` | IOB code of named entity tag. `3` means the token begins an entity, `2` means it is outside an entity, `1` means it is inside an entity, and `0` means no entity tag is set. ~~int~~ |
|
||||
| `ent_iob_` | IOB code of named entity tag. "B" means the token begins an entity, "I" means it is inside an entity, "O" means it is outside an entity, and "" means no entity tag is set. ~~str~~ |
|
||||
| `ent_kb_id` <Tag variant="new">2.2</Tag> | Knowledge base ID that refers to the named entity this token is a part of, if any. ~~int~~ |
|
||||
| `ent_kb_id_` <Tag variant="new">2.2</Tag> | Knowledge base ID that refers to the named entity this token is a part of, if any. ~~str~~ |
|
||||
| `ent_id` | ID of the entity the token is an instance of, if any. Currently not used, but potentially for coreference resolution. ~~int~~ |
|
||||
| `ent_id_` | ID of the entity the token is an instance of, if any. Currently not used, but potentially for coreference resolution. ~~str~~ |
|
||||
| `lemma` | Base form of the token, with no inflectional suffixes. ~~int~~ |
|
||||
| `lemma_` | Base form of the token, with no inflectional suffixes. ~~str~~ |
|
||||
| `norm` | The token's norm, i.e. a normalized form of the token text. Can be set in the language's [tokenizer exceptions](/usage/adding-languages#tokenizer-exceptions). ~~int~~ |
|
||||
| `norm_` | The token's norm, i.e. a normalized form of the token text. Can be set in the language's [tokenizer exceptions](/usage/adding-languages#tokenizer-exceptions). ~~str~~ |
|
||||
| `lower` | Lowercase form of the token. ~~int~~ |
|
||||
| `lower_` | Lowercase form of the token text. Equivalent to `Token.text.lower()`. ~~str~~ |
|
||||
| Name | Description |
|
||||
| -------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `doc` | The parent document. ~~Doc~~ |
|
||||
| `lex` <Tag variant="new">3</Tag> | The underlying lexeme. ~~Lexeme~~ |
|
||||
| `sent` <Tag variant="new">2.0.12</Tag> | The sentence span that this token is a part of. ~~Span~~ |
|
||||
| `text` | Verbatim text content. ~~str~~ |
|
||||
| `text_with_ws` | Text content, with trailing space character if present. ~~str~~ |
|
||||
| `whitespace_` | Trailing space character if present. ~~str~~ |
|
||||
| `orth` | ID of the verbatim text content. ~~int~~ |
|
||||
| `orth_` | Verbatim text content (identical to `Token.text`). Exists mostly for consistency with the other attributes. ~~str~~ |
|
||||
| `vocab` | The vocab object of the parent `Doc`. ~~vocab~~ |
|
||||
| `tensor` <Tag variant="new">2.1.7</Tag> | The tokens's slice of the parent `Doc`'s tensor. ~~numpy.ndarray~~ |
|
||||
| `head` | The syntactic parent, or "governor", of this token. ~~Token~~ |
|
||||
| `left_edge` | The leftmost token of this token's syntactic descendants. ~~Token~~ |
|
||||
| `right_edge` | The rightmost token of this token's syntactic descendants. ~~Token~~ |
|
||||
| `i` | The index of the token within the parent document. ~~int~~ |
|
||||
| `ent_type` | Named entity type. ~~int~~ |
|
||||
| `ent_type_` | Named entity type. ~~str~~ |
|
||||
| `ent_iob` | IOB code of named entity tag. `3` means the token begins an entity, `2` means it is outside an entity, `1` means it is inside an entity, and `0` means no entity tag is set. ~~int~~ |
|
||||
| `ent_iob_` | IOB code of named entity tag. "B" means the token begins an entity, "I" means it is inside an entity, "O" means it is outside an entity, and "" means no entity tag is set. ~~str~~ |
|
||||
| `ent_kb_id` <Tag variant="new">2.2</Tag> | Knowledge base ID that refers to the named entity this token is a part of, if any. ~~int~~ |
|
||||
| `ent_kb_id_` <Tag variant="new">2.2</Tag> | Knowledge base ID that refers to the named entity this token is a part of, if any. ~~str~~ |
|
||||
| `ent_id` | ID of the entity the token is an instance of, if any. Currently not used, but potentially for coreference resolution. ~~int~~ |
|
||||
| `ent_id_` | ID of the entity the token is an instance of, if any. Currently not used, but potentially for coreference resolution. ~~str~~ |
|
||||
| `lemma` | Base form of the token, with no inflectional suffixes. ~~int~~ |
|
||||
| `lemma_` | Base form of the token, with no inflectional suffixes. ~~str~~ |
|
||||
| `norm` | The token's norm, i.e. a normalized form of the token text. Can be set in the language's [tokenizer exceptions](/usage/adding-languages#tokenizer-exceptions). ~~int~~ |
|
||||
| `norm_` | The token's norm, i.e. a normalized form of the token text. Can be set in the language's [tokenizer exceptions](/usage/adding-languages#tokenizer-exceptions). ~~str~~ |
|
||||
| `lower` | Lowercase form of the token. ~~int~~ |
|
||||
| `lower_` | Lowercase form of the token text. Equivalent to `Token.text.lower()`. ~~str~~ |
|
||||
| `shape` | Transform of the tokens's string to show orthographic features. Alphabetic characters are replaced by `x` or `X`, and numeric characters are replaced by `d`, and sequences of the same character are truncated after length 4. For example,`"Xxxx"`or`"dd"`. ~~int~~ |
|
||||
| `shape_` | Transform of the tokens's string to show orthographic features. Alphabetic characters are replaced by `x` or `X`, and numeric characters are replaced by `d`, and sequences of the same character are truncated after length 4. For example,`"Xxxx"`or`"dd"`. ~~str~~ |
|
||||
| `prefix` | Hash value of a length-N substring from the start of the token. Defaults to `N=1`. ~~int~~ |
|
||||
| `prefix_` | A length-N substring from the start of the token. Defaults to `N=1`. ~~str~~ |
|
||||
| `suffix` | Hash value of a length-N substring from the end of the token. Defaults to `N=3`. ~~int~~ |
|
||||
| `suffix_` | Length-N substring from the end of the token. Defaults to `N=3`. ~~str~~ |
|
||||
| `is_alpha` | Does the token consist of alphabetic characters? Equivalent to `token.text.isalpha()`. ~~bool~~ |
|
||||
| `is_ascii` | Does the token consist of ASCII characters? Equivalent to `all(ord(c) < 128 for c in token.text)`. ~~bool~~ |
|
||||
| `is_digit` | Does the token consist of digits? Equivalent to `token.text.isdigit()`. ~~bool~~ |
|
||||
| `is_lower` | Is the token in lowercase? Equivalent to `token.text.islower()`. ~~bool~~ |
|
||||
| `is_upper` | Is the token in uppercase? Equivalent to `token.text.isupper()`. ~~bool~~ |
|
||||
| `is_title` | Is the token in titlecase? Equivalent to `token.text.istitle()`. ~~bool~~ |
|
||||
| `is_punct` | Is the token punctuation? ~~bool~~ |
|
||||
| `is_left_punct` | Is the token a left punctuation mark, e.g. `"("` ? ~~bool~~ |
|
||||
| `is_right_punct` | Is the token a right punctuation mark, e.g. `")"` ? ~~bool~~ |
|
||||
| `is_space` | Does the token consist of whitespace characters? Equivalent to `token.text.isspace()`. ~~bool~~ |
|
||||
| `is_bracket` | Is the token a bracket? ~~bool~~ |
|
||||
| `is_quote` | Is the token a quotation mark? ~~bool~~ |
|
||||
| `is_currency` <Tag variant="new">2.0.8</Tag> | Is the token a currency symbol? ~~bool~~ |
|
||||
| `like_url` | Does the token resemble a URL? ~~bool~~ |
|
||||
| `like_num` | Does the token represent a number? e.g. "10.9", "10", "ten", etc. ~~bool~~ |
|
||||
| `like_email` | Does the token resemble an email address? ~~bool~~ |
|
||||
| `is_oov` | Does the token have a word vector? ~~bool~~ |
|
||||
| `is_stop` | Is the token part of a "stop list"? ~~bool~~ |
|
||||
| `pos` | Coarse-grained part-of-speech from the [Universal POS tag set](https://universaldependencies.org/docs/u/pos/). ~~int~~ |
|
||||
| `pos_` | Coarse-grained part-of-speech from the [Universal POS tag set](https://universaldependencies.org/docs/u/pos/). ~~str~~ |
|
||||
| `tag` | Fine-grained part-of-speech. ~~int~~ |
|
||||
| `tag_` | Fine-grained part-of-speech. ~~str~~ |
|
||||
| `morph` <Tag variant="new">3</Tag> | Morphological analysis. ~~MorphAnalysis~~ |
|
||||
| `morph_` <Tag variant="new">3</Tag> | Morphological analysis in the Universal Dependencies [FEATS](https://universaldependencies.org/format.html#morphological-annotation) format. ~~str~~ |
|
||||
| `dep` | Syntactic dependency relation. ~~int~~ |
|
||||
| `dep_` | Syntactic dependency relation. ~~str~~ |
|
||||
| `lang` | Language of the parent document's vocabulary. ~~int~~ |
|
||||
| `lang_` | Language of the parent document's vocabulary. ~~str~~ |
|
||||
| `prob` | Smoothed log probability estimate of token's word type (context-independent entry in the vocabulary). ~~float~~ |
|
||||
| `idx` | The character offset of the token within the parent document. ~~int~~ |
|
||||
| `sentiment` | A scalar value indicating the positivity or negativity of the token. ~~float~~ |
|
||||
| `lex_id` | Sequential ID of the token's lexical type, used to index into tables, e.g. for word vectors. ~~int~~ |
|
||||
| `rank` | Sequential ID of the token's lexical type, used to index into tables, e.g. for word vectors. ~~int~~ |
|
||||
| `cluster` | Brown cluster ID. ~~int~~ |
|
||||
| `_` | User space for adding custom [attribute extensions](/usage/processing-pipelines#custom-components-attributes). ~~Underscore~~ |
|
||||
| `prefix` | Hash value of a length-N substring from the start of the token. Defaults to `N=1`. ~~int~~ |
|
||||
| `prefix_` | A length-N substring from the start of the token. Defaults to `N=1`. ~~str~~ |
|
||||
| `suffix` | Hash value of a length-N substring from the end of the token. Defaults to `N=3`. ~~int~~ |
|
||||
| `suffix_` | Length-N substring from the end of the token. Defaults to `N=3`. ~~str~~ |
|
||||
| `is_alpha` | Does the token consist of alphabetic characters? Equivalent to `token.text.isalpha()`. ~~bool~~ |
|
||||
| `is_ascii` | Does the token consist of ASCII characters? Equivalent to `all(ord(c) < 128 for c in token.text)`. ~~bool~~ |
|
||||
| `is_digit` | Does the token consist of digits? Equivalent to `token.text.isdigit()`. ~~bool~~ |
|
||||
| `is_lower` | Is the token in lowercase? Equivalent to `token.text.islower()`. ~~bool~~ |
|
||||
| `is_upper` | Is the token in uppercase? Equivalent to `token.text.isupper()`. ~~bool~~ |
|
||||
| `is_title` | Is the token in titlecase? Equivalent to `token.text.istitle()`. ~~bool~~ |
|
||||
| `is_punct` | Is the token punctuation? ~~bool~~ |
|
||||
| `is_left_punct` | Is the token a left punctuation mark, e.g. `"("` ? ~~bool~~ |
|
||||
| `is_right_punct` | Is the token a right punctuation mark, e.g. `")"` ? ~~bool~~ |
|
||||
| `is_space` | Does the token consist of whitespace characters? Equivalent to `token.text.isspace()`. ~~bool~~ |
|
||||
| `is_bracket` | Is the token a bracket? ~~bool~~ |
|
||||
| `is_quote` | Is the token a quotation mark? ~~bool~~ |
|
||||
| `is_currency` <Tag variant="new">2.0.8</Tag> | Is the token a currency symbol? ~~bool~~ |
|
||||
| `like_url` | Does the token resemble a URL? ~~bool~~ |
|
||||
| `like_num` | Does the token represent a number? e.g. "10.9", "10", "ten", etc. ~~bool~~ |
|
||||
| `like_email` | Does the token resemble an email address? ~~bool~~ |
|
||||
| `is_oov` | Does the token have a word vector? ~~bool~~ |
|
||||
| `is_stop` | Is the token part of a "stop list"? ~~bool~~ |
|
||||
| `pos` | Coarse-grained part-of-speech from the [Universal POS tag set](https://universaldependencies.org/docs/u/pos/). ~~int~~ |
|
||||
| `pos_` | Coarse-grained part-of-speech from the [Universal POS tag set](https://universaldependencies.org/docs/u/pos/). ~~str~~ |
|
||||
| `tag` | Fine-grained part-of-speech. ~~int~~ |
|
||||
| `tag_` | Fine-grained part-of-speech. ~~str~~ |
|
||||
| `morph` <Tag variant="new">3</Tag> | Morphological analysis. ~~MorphAnalysis~~ |
|
||||
| `dep` | Syntactic dependency relation. ~~int~~ |
|
||||
| `dep_` | Syntactic dependency relation. ~~str~~ |
|
||||
| `lang` | Language of the parent document's vocabulary. ~~int~~ |
|
||||
| `lang_` | Language of the parent document's vocabulary. ~~str~~ |
|
||||
| `prob` | Smoothed log probability estimate of token's word type (context-independent entry in the vocabulary). ~~float~~ |
|
||||
| `idx` | The character offset of the token within the parent document. ~~int~~ |
|
||||
| `sentiment` | A scalar value indicating the positivity or negativity of the token. ~~float~~ |
|
||||
| `lex_id` | Sequential ID of the token's lexical type, used to index into tables, e.g. for word vectors. ~~int~~ |
|
||||
| `rank` | Sequential ID of the token's lexical type, used to index into tables, e.g. for word vectors. ~~int~~ |
|
||||
| `cluster` | Brown cluster ID. ~~int~~ |
|
||||
| `_` | User space for adding custom [attribute extensions](/usage/processing-pipelines#custom-components-attributes). ~~Underscore~~ |
|
||||
|
|
|
|||
|
|
@ -56,16 +56,13 @@ create a surface form. Here are some examples:
|
|||
|
||||
Morphological features are stored in the [`MorphAnalysis`](/api/morphanalysis)
|
||||
under `Token.morph`, which allows you to access individual morphological
|
||||
features. The attribute `Token.morph_` provides the morphological analysis in
|
||||
the Universal Dependencies
|
||||
[FEATS](https://universaldependencies.org/format.html#morphological-annotation)
|
||||
format.
|
||||
features.
|
||||
|
||||
> #### 📝 Things to try
|
||||
>
|
||||
> 1. Change "I" to "She". You should see that the morphological features change
|
||||
> and express that it's a pronoun in the third person.
|
||||
> 2. Inspect `token.morph_` for the other tokens.
|
||||
> 2. Inspect `token.morph` for the other tokens.
|
||||
|
||||
```python
|
||||
### {executable="true"}
|
||||
|
|
@ -75,7 +72,7 @@ nlp = spacy.load("en_core_web_sm")
|
|||
print("Pipeline:", nlp.pipe_names)
|
||||
doc = nlp("I was reading the paper.")
|
||||
token = doc[0] # 'I'
|
||||
print(token.morph_) # 'Case=Nom|Number=Sing|Person=1|PronType=Prs'
|
||||
print(token.morph) # 'Case=Nom|Number=Sing|Person=1|PronType=Prs'
|
||||
print(token.morph.get("PronType")) # ['Prs']
|
||||
```
|
||||
|
||||
|
|
@ -91,7 +88,7 @@ import spacy
|
|||
|
||||
nlp = spacy.load("de_core_news_sm")
|
||||
doc = nlp("Wo bist du?") # English: 'Where are you?'
|
||||
print(doc[2].morph_) # 'Case=Nom|Number=Sing|Person=2|PronType=Prs'
|
||||
print(doc[2].morph) # 'Case=Nom|Number=Sing|Person=2|PronType=Prs'
|
||||
print(doc[2].pos_) # 'PRON'
|
||||
```
|
||||
|
||||
|
|
@ -117,7 +114,7 @@ import spacy
|
|||
|
||||
nlp = spacy.load("en_core_web_sm")
|
||||
doc = nlp("Where are you?")
|
||||
print(doc[2].morph_) # 'Case=Nom|Person=2|PronType=Prs'
|
||||
print(doc[2].morph) # 'Case=Nom|Person=2|PronType=Prs'
|
||||
print(doc[2].pos_) # 'PRON'
|
||||
```
|
||||
|
||||
|
|
|
|||
Loading…
Reference in New Issue
Block a user