explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2024-11-11 04:08:09 +03:00
Code Issues Packages Projects Releases Wiki Activity
45c9a68828
spaCy/spacy/lang
History
Duygu Altinok 0e55f806dd
Turkish tokenization improvements (#6268) 
* added single and paired orth variants

* added token match

* added long text tokenization test

* inverted init

* normalized lemmas to lowercase

* more abbrevs

* tests for ordinals and abbrevs

* separated period abbvrevs to another list

* fiex typo

* added ordinal and abbrev tests

* added number tests for dates

* minor refinement

* added inflected abbrevs regex

* added percentage and inflection

* cosmetics

* added token match

* added url inflection tests

* excluded url tokens from custom pattern

* removed url match import
2020-10-29 09:43:17 +01:00
..
af 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
ar Add writing_system to ArabicDefaults (experimental) 2019-03-11 14:22:23 +01:00
bg Update examples and languages.json [ci skip] 2019-09-15 17:56:40 +02:00
bn Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
ca Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
cs Create examples.py (#5985) 2020-09-04 11:00:14 +02:00
da Tidy up and auto-format 2020-05-21 14:14:01 +02:00
de Fix overlapping German noun chunks (#6112) 2020-09-22 21:52:42 +02:00
el span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
en Hebrew like num (#5952) 2020-08-24 14:30:05 +02:00
es Fix span boundary handling in Spanish noun_chunks (#5860) 2020-08-03 13:53:15 +02:00
et 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
eu Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
fa span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
fi add two abbreviations and some additional unit tests (#5040) 2020-02-22 14:12:32 +01:00
fr Remove is_base_form from French lemmatizer (#5733) 2020-07-09 22:11:13 +02:00
ga 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
gu Tidy up and auto-format 2020-05-21 14:14:01 +02:00
he Hebrew like num (#5952) 2020-08-24 14:30:05 +02:00
hi Hindi: Adds tests for lexical attributes (norm and like_num) (#5829) 2020-10-07 10:23:32 +02:00
hr Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
hu Add tokenizer option for token match with affixes 2020-05-05 10:35:33 +02:00
hy Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
id Update Indonesian Example Phrases (#6124) 2020-09-23 14:02:26 +02:00
is 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
it Tidy up and auto-format 2020-03-25 12:28:12 +01:00
ja fix ja leading spaces (#5969) 2020-08-25 14:16:24 +02:00
kn Add kannada examples (#5162) 2020-03-29 13:54:42 +02:00
ko fix bug in Korean language, resulting in 100x speedup by reducing overhead of mecab (#5701) 2020-07-06 17:03:33 +02:00
lb Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
lij Tidy up and auto-format 2020-03-25 12:28:12 +01:00
lt Tidy up and auto-format 2020-03-25 12:28:12 +01:00
lv 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
mk Include Macedonian language (#6230) 2020-10-15 15:55:01 +02:00
ml Tidy up and auto-format 2020-05-21 14:14:01 +02:00
mr Tidy up [ci skip] 2019-06-12 13:38:23 +02:00
nb span / noun chunk has +1 because end is exclusive 2020-05-21 19:56:56 +02:00
ne Add Nepali Language (#5622) 2020-06-22 10:25:46 +02:00
nl Improve tokenization for UD Dutch corpora (#5259) 2020-04-06 13:18:07 +02:00
pl Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
pt Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
ro Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
ru Update invalid tag maps (#5796) 2020-07-22 16:02:51 +02:00
sa Added support for Sanskrit language (#5956) 2020-08-25 10:56:29 +02:00
si 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
sk Tidy up and auto-format 2020-03-25 12:28:12 +01:00
sl 💫 Add base Language classes for more languages (#3276) 2019-02-15 01:31:19 +11:00
sq Update languages and examples (see #1107) 2019-06-26 16:19:17 +02:00
sr Reduce stored lexemes data, move feats to lookups (#5238) 2020-05-19 15:59:14 +02:00
sv Update morph_rules.py (#6102) 2020-10-06 15:14:47 +02:00
ta Update spacy/lang/ta/examples.py 2020-10-13 11:03:35 +02:00
te 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
th Add Thai tag map (LST20 Corpus) (#6163) 2020-10-07 11:12:01 +02:00
tl Move lookup tables out of the core library (#4346) 2019-10-01 00:01:27 +02:00
tr Turkish tokenization improvements (#6268) 2020-10-29 09:43:17 +01:00
tt Tidy up and auto-format 2019-08-20 17:36:34 +02:00
uk Update Ukrainian lemmatizer with new lookups (#4359) 2019-10-02 12:04:06 +02:00
ur Tidy up and auto-format 2020-05-21 14:14:01 +02:00
vi 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
xx Minor updates to language example sentences (#4608) 2019-11-07 22:34:58 +01:00
yo Adding support for Yoruba Language (#4614) 2019-12-21 14:11:50 +01:00
zh Update pkuseg version (#5774) 2020-07-19 11:09:49 +02:00
__init__.py Remove imports in /lang/__init__.py 2017-05-08 23:58:07 +02:00
char_classes.py Include Macedonian language (#6230) 2020-10-15 15:55:01 +02:00
lex_attrs.py Hebrew like num (#5952) 2020-08-24 14:30:05 +02:00
norm_exceptions.py Update norm_exceptions.py (#3778) 2019-05-27 11:52:52 +02:00
punctuation.py Allow period as suffix following punctuation (#4248) 2019-09-09 19:19:22 +02:00
tag_map.py 💫 Tidy up and auto-format .py files (#2983) 2018-11-30 17:03:03 +01:00
tokenizer_exceptions.py Fix raw strings in URL pattern (#5972) 2020-08-26 04:00:49 +02:00