explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-08-05 04:40:20 +03:00
Code Issues Packages Projects Releases Wiki Activity

fix: match hyphenated words to lemmas in index_table (e.g. "co-authored" -> "co-author")

Browse Source
This commit is contained in:
d0ngw 2025-05-09 18:42:08 +08:00 committed by GitHub
parent 98a19df91a
commit 12292b36f9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 4 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

5
spacy/pipeline/lemmatizer.py
View File

@ -241,7 +241,10 @@ class Lemmatizer(Pipe):
if not form: if not form:
pass pass
elif form in index or not form.isalpha(): elif form in index or not form.isalpha():
forms.append(form) if form in index:
forms.insert(0, form)
else:
forms.append(form)
else: else:
oov_forms.append(form) oov_forms.append(form)
# Remove duplicates but preserve the ordering of applied "rules" # Remove duplicates but preserve the ordering of applied "rules"