spaCy/spacy/lang
Adriane Boyd f42c9026f5
Update v2.3.x branch (#5636)
* Fix typos and auto-format [ci skip]

* Add pkuseg warnings and auto-format [ci skip]

* Update Binder URL [ci skip]

* Update Binder version [ci skip]

* Update alignment example for new gold.align

* Update POS in tagging example

* Fix numpy.zeros() dtype for Doc.from_array

* Change example title to Dr.

Change example title to Dr. so the current model does exclude the title
in the initial example.

* Fix spacy convert argument

* Warning for sudachipy 0.4.5 (#5611)

* Create myavrum.md (#5612)

* Update lex_attrs.py (#5608)

* Create mahnerak.md (#5615)

* Some changes for Armenian (#5616)

* Fixing numericals

* We need a Armenian question sign to make the sentence a question

* Add Nepali Language  (#5622)

* added support for nepali lang

* added examples and test files

* added spacy contributor agreement

* Japanese model: add user_dict entries and small refactor (#5573)

* user_dict fields: adding inflections, reading_forms, sub_tokens
deleting: unidic_tags
improve code readability around the token alignment procedure

* add test cases, replace fugashi with sudachipy in conftest

* move bunsetu.py to spaCy Universe as a pipeline component BunsetuRecognizer

* tag is space -> both surface and tag are spaces

* consider len(text)==0

* Add warnings example in v2.3 migration guide (#5627)

* contribute (#5632)

* Fix polarity of Token.is_oov and Lexeme.is_oov (#5634)

Fix `Token.is_oov` and `Lexeme.is_oov` so they return `True` when the
lexeme does **not** have a vector.

* Extend what's new in v2.3 with vocab / is_oov (#5635)

* Skip vocab in component config overrides (#5624)

* Fix backslashes in warnings config diff (#5640)

Fix backslashes in warnings config diff in v2.3 migration section.

* Disregard special tag  _SP in check for new tag map (#5641)

* Skip special tag  _SP in check for new tag map

In `Tagger.begin_training()` check for new tags aside from `_SP` in the
new tag map initialized from the provided gold tuples when determining
whether to reinitialize the morphology with the new tag map.

* Simplify _SP check

Co-authored-by: Ines Montani <ines@ines.io>
Co-authored-by: Marat M. Yavrumyan <myavrum@ysu.am>
Co-authored-by: Karen Hambardzumyan <mahnerak@gmail.com>
Co-authored-by: Rameshh <30867740+rameshhpathak@users.noreply.github.com>
Co-authored-by: Hiroshi Matsuda <40782025+hiroshi-matsuda-rit@users.noreply.github.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-06-29 14:13:12 +02:00
..
af 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
ar Add writing_system to ArabicDefaults (experimental) 2019-03-11 14:22:23 +01:00
bg Update examples and languages.json [ci skip] 2019-09-15 17:56:40 +02:00
bn Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
ca Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
cs 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
da Tidy up and auto-format 2020-05-21 14:14:01 +02:00
de Rename argument: doc_or_span/obj -> doclike (#5463) 2020-05-21 15:17:39 +02:00
el span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
en Add "c'mon" token exception (#5570) 2020-06-10 21:54:06 +02:00
es Spanish tokenizer exception and examples improvement (#5531) 2020-06-01 18:18:34 +02:00
et 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
eu Tidy up and auto-format 2020-03-25 12:28:12 +01:00
fa span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
fi add two abbreviations and some additional unit tests (#5040) 2020-02-22 14:12:32 +01:00
fr corrected issue #5524 changed <U+009C> 'STRING TERMINATOR' for <U+0153> LATIN SMALL LIGATURE OE' (#5526) 2020-05-31 22:08:12 +02:00
ga 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
gu Tidy up and auto-format 2020-05-21 14:14:01 +02:00
he Auto-format [ci skip] 2019-03-11 17:10:50 +01:00
hi Fix example sentences in Hindi for grammatical errors (#4343) 2019-09-30 23:32:49 +02:00
hr Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
hu Add tokenizer option for token match with affixes 2020-05-05 10:35:33 +02:00
hy Update v2.3.x branch (#5636) 2020-06-29 14:13:12 +02:00
id span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
is 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
it Tidy up and auto-format 2020-03-25 12:28:12 +01:00
ja Update v2.3.x branch (#5636) 2020-06-29 14:13:12 +02:00
kn Add kannada examples (#5162) 2020-03-29 13:54:42 +02:00
ko Very minor issues in Korean example sentences (#5446) 2020-05-17 13:43:34 +02:00
lb Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
lij Tidy up and auto-format 2020-03-25 12:28:12 +01:00
lt Tidy up and auto-format 2020-03-25 12:28:12 +01:00
lv 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
ml Tidy up and auto-format 2020-05-21 14:14:01 +02:00
mr Tidy up [ci skip] 2019-06-12 13:38:23 +02:00
nb span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
ne Update v2.3.x branch (#5636) 2020-06-29 14:13:12 +02:00
nl Improve tokenization for UD Dutch corpora (#5259) 2020-04-06 13:18:07 +02:00
pl Fix Polish lemmatizer for deserialized models 2020-05-26 09:56:12 +02:00
pt Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
ro Improved Romanian tokenization for UD RRT (#5036) 2020-02-19 16:15:59 +01:00
ru Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
si 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
sk Tidy up and auto-format 2020-03-25 12:28:12 +01:00
sl 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
sq Update languages and examples (see #1107) 2019-06-26 16:19:17 +02:00
sr Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
sv span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
ta Added Tamil Example Sentences (#5583) 2020-06-13 15:56:26 +02:00
te 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
th Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
tl Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
tr Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
tt Tidy up and auto-format 2019-08-20 17:36:34 +02:00
uk Update Ukrainian lemmatizer with new lookups (#4359) 2019-10-02 12:04:06 +02:00
ur Tidy up and auto-format 2020-05-21 14:14:01 +02:00
vi 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
xx Minor updates to language example sentences (#4608) 2019-11-07 22:34:58 +01:00
yo Adding support for Yoruba Language (#4614) 2019-12-21 14:11:50 +01:00
zh Map NR to PROPN (#5512) 2020-05-26 22:30:53 +02:00
__init__.py Remove imports in /lang/__init__.py 2017-05-08 23:58:07 +02:00
char_classes.py Add CJK to character classes (#4884) 2020-01-08 16:50:19 +01:00
lex_attrs.py Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
norm_exceptions.py Update norm_exceptions.py (#3778) 2019-05-27 11:52:52 +02:00
punctuation.py Allow period as suffix following punctuation (#4248) 2019-09-09 19:19:22 +02:00
tag_map.py 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
tokenizer_exceptions.py Rename to url_match 2020-05-22 12:41:03 +02:00