spaCy/spacy
adrianeboyd 86c43e55fa
Improve Lithuanian tokenization (#5205)
* Improve Lithuanian tokenization

Modify Lithuanian tokenization to improve performance for
UD_Lithuanian-ALKSNIS.

* Update Lithuanian tokenizer tests
2020-03-25 11:28:12 +01:00
..
cli Fix formatting and update docs for v2.2.4 2020-03-09 11:17:20 +01:00
data Make spacy/data a package 2017-03-18 20:04:22 +01:00
displacy add lemma option to displacy 'dep' visualiser (#5041) 2020-02-22 14:11:51 +01:00
lang Improve Lithuanian tokenization (#5205) 2020-03-25 11:28:12 +01:00
matcher Normalize IS_SENT_START to SENT_START for Matcher (#5080) 2020-03-03 12:22:39 +01:00
ml Replace function registries with catalogue (#4584) 2019-11-07 11:45:22 +01:00
pipeline Bugfix/get doc (#5049) 2020-03-02 11:49:28 +01:00
syntax prevent updating cfg if the Model was already defined (#5078) 2020-03-03 13:58:56 +01:00
tests Improve Lithuanian tokenization (#5205) 2020-03-25 11:28:12 +01:00
tokens additional information if doc is empty 2020-03-09 18:08:18 +01:00
__init__.pxd * Seems to be working after refactor. Need to wire up more POS tag features, and wire up save/load of POS tags. 2014-10-24 02:23:42 +11:00
__init__.py Replace function registries with catalogue (#4584) 2019-11-07 11:45:22 +01:00
__main__.py Use latest wasabi 2019-11-04 02:38:45 +01:00
_align.pyx Fixes typos (#4843) 2019-12-29 14:24:13 +01:00
_ml.py take care of global vectors in multiprocessing (#5081) 2020-03-03 13:58:22 +01:00
about.py Set version to v2.2.4 2020-03-12 11:30:41 +01:00
analysis.py Support span._. in component decorator attrs (#4555) 2019-10-30 17:19:36 +01:00
attrs.pxd make idx available via to_array (#5030) 2020-02-22 14:13:06 +01:00
attrs.pyx make idx available via to_array (#5030) 2020-02-22 14:13:06 +01:00
compat.py Replace function registries with catalogue (#4584) 2019-11-07 11:45:22 +01:00
errors.py Improve token head verification (#5079) 2020-03-03 21:44:51 +01:00
glossary.py Update tag maps and docs for English and German (#4501) 2019-10-24 12:56:05 +02:00
gold.pxd Merge changes from master 2019-08-21 14:18:52 +02:00
gold.pyx Initialize all values in a2b/b2a in new align (#5063) 2020-02-27 18:43:00 +01:00
kb.pxd rename entity frequency 2019-07-19 17:40:28 +02:00
kb.pyx More robust set entities method in KB (#4794) 2019-12-13 10:45:29 +01:00
language.py Remove unnecessary iterator in Language.pipe (#5101) 2020-03-08 13:22:25 +01:00
lemmatizer.py Refactor lemmatizer and data table integration (#4353) 2019-10-01 21:36:03 +02:00
lexeme.pxd 💫 Support lexical attributes in retokenizer attrs (closes #2390) (#3325) 2019-02-24 21:13:51 +01:00
lexeme.pyx Alphanumeric -> alphabetic [ci skip] 2019-10-06 13:30:01 +02:00
lookups.py Refactor lemmatizer and data table integration (#4353) 2019-10-01 21:36:03 +02:00
morphology.pxd annotate kb_id through ents in doc 2019-03-22 11:36:44 +01:00
morphology.pyx Improve Morphology errors (#4314) 2019-09-21 14:37:06 +02:00
parts_of_speech.pxd Add support for Universal Dependencies v2.0 2017-03-03 13:17:34 +01:00
parts_of_speech.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
scorer.py Auto-format 2019-11-20 13:15:24 +01:00
strings.pxd Try to fix StringStore clean up (see #1506) 2017-11-11 03:11:27 +03:00
strings.pyx Merge branch 'master' into feature/lemmatizer 2019-03-16 13:44:22 +01:00
structs.pxd Replace Entity/MatchStruct with SpanC (#4459) 2019-10-18 11:01:47 +02:00
symbols.pxd make idx available via to_array (#5030) 2020-02-22 14:13:06 +01:00
symbols.pyx make idx available via to_array (#5030) 2020-02-22 14:13:06 +01:00
tokenizer.pxd Flush tokenizer cache when necessary (#4258) 2019-09-08 20:52:46 +02:00
tokenizer.pyx Tokenizer to_disk and from_disk now ensure paths (#5116) 2020-03-08 13:25:56 +01:00
typedefs.pxd Work on changing StringStore to return hashes. 2017-05-28 12:36:27 +02:00
typedefs.pyx Tidy up rest 2017-10-27 21:07:59 +02:00
util.py Auto-exclude disabled when calling from_disk during load (#4708) 2019-11-25 16:01:22 +01:00
vectors.pyx Use int only in key2row for better performance (#4990) 2020-02-16 17:19:41 +01:00
vocab.pxd 💫 WIP: Basic lookup class scaffolding and JSON for all lemmati… (#4167) 2019-08-22 14:21:32 +02:00
vocab.pyx Agnostic vocab array fix (#4680) 2019-11-23 14:59:52 +01:00