Paul O'Leary McCann f0e3e606a6 Replace python-mecab3 with fugashi for Japanese (#4621) ... * Switch from mecab-python3 to fugashi mecab-python3 has been the best MeCab binding for a long time but it's not very actively maintained, and since it's based on old SWIG code distributed with MeCab there's a limit to how effectively it can be maintained. Fugashi is a new Cython-based MeCab wrapper I wrote. Since it's not based on the old SWIG code it's easier to keep it current and make small deviations from the MeCab C/C++ API where that makes sense. * Change mecab-python3 to fugashi in setup.cfg * Change "mecab tags" to "unidic tags" The tags come from MeCab, but the tag schema is specified by Unidic, so it's more proper to refer to it that way. * Update conftest * Add fugashi link to external deps list for Japanese		2019-11-23 14:31:04 +01:00
..
af	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
ar	Add writing_system to ArabicDefaults (experimental)	2019-03-11 14:22:23 +01:00
bg	Update examples and languages.json [ci skip]	2019-09-15 17:56:40 +02:00
bn	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
ca	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
cs	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
da	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
de	Update tag maps and docs for English and German (#4501)	2019-10-24 12:56:05 +02:00
el	Refactor lemmatizer and data table integration (#4353)	2019-10-01 21:36:03 +02:00
en	Tidy up and auto-format [ci skip]	2019-10-24 16:20:48 +02:00
es	Minor updates to language example sentences (#4608)	2019-11-07 22:34:58 +01:00
et	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
fa	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
fi	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
fr	Refactor lemmatizer and data table integration (#4353)	2019-10-01 21:36:03 +02:00
ga	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
he	Auto-format [ci skip]	2019-03-11 17:10:50 +01:00
hi	Fix example sentences in Hindi for grammatical errors (#4343)	2019-09-30 23:32:49 +02:00
hr	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
hu	Improve URL_PATTERN and handling in tokenizer (#4374)	2019-10-05 13:00:09 +02:00
id	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
is	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
it	Remove unnecessary newline replace	2019-11-15 16:19:01 +01:00
ja	Replace python-mecab3 with fugashi for Japanese (#4621)	2019-11-23 14:31:04 +01:00
kn	Enhancing Kannada language Resources (#3755)	2019-05-20 12:56:10 +02:00
ko	Auto-format	2019-11-20 13:15:24 +01:00
lb	Auto-format	2019-11-20 13:15:24 +01:00
lt	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
lv	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
mr	Tidy up [ci skip]	2019-06-12 13:38:23 +02:00
nb	Minor updates to language example sentences (#4608)	2019-11-07 22:34:58 +01:00
nl	Refactor lemmatizer and data table integration (#4353)	2019-10-01 21:36:03 +02:00
pl	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
pt	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
ro	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
ru	Refactor lemmatizer and data table integration (#4353)	2019-10-01 21:36:03 +02:00
si	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
sk	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
sl	💫 Add base Language classes for more languages (#3276)	2019-02-15 01:31:19 +11:00
sq	Update languages and examples (see #1107)	2019-06-26 16:19:17 +02:00
sr	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
sv	Adding noun_chunks to the Swedish language model (sv) (#4422)	2019-10-21 12:57:06 +02:00
ta	Remove stray print statement (closes #3342)	2019-02-27 15:35:04 +01:00
te	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
th	fix thai bug (#3693)	2019-05-10 14:21:34 +02:00
tl	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
tr	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
tt	Tidy up and auto-format	2019-08-20 17:36:34 +02:00
uk	Update Ukrainian lemmatizer with new lookups (#4359)	2019-10-02 12:04:06 +02:00
ur	Move lookup tables out of the core library (#4346)	2019-10-01 00:01:27 +02:00
vi	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
xx	Minor updates to language example sentences (#4608)	2019-11-07 22:34:58 +01:00
zh	Auto-format	2019-11-20 13:15:24 +01:00
__init__.py	Remove imports in /lang/__init__.py	2017-05-08 23:58:07 +02:00
char_classes.py	Auto-format	2019-11-20 13:15:24 +01:00
lex_attrs.py	Replacing regex library with re to increase tokenization speed (#3218)	2019-02-01 18:05:22 +11:00
norm_exceptions.py	Update norm_exceptions.py (#3778)	2019-05-27 11:52:52 +02:00
punctuation.py	Allow period as suffix following punctuation (#4248)	2019-09-09 19:19:22 +02:00
tag_map.py	💫 Tidy up and auto-format .py files (#2983)	2018-11-30 17:03:03 +01:00
tokenizer_exceptions.py	Improve URL_PATTERN and handling in tokenizer (#4374)	2019-10-05 13:00:09 +02:00