explosion/spaCy
1
1
Fork 0
mirror of https://github.com/explosion/spaCy.git synced 2025-01-10 01:06:33 +03:00
Code Issues Packages Projects Releases Wiki Activity
364aaf5bc2
spaCy/spacy/lang/el/lemmatizer/_nouns.json

1 line
956 KiB
JSON
Raw Normal View History

Reduce size of language data (#4141) * Move Turkish lemmas to a json file Rather than a large dict in Python source, the data is now a big json file. This includes a method for loading the json file, falling back to a compressed file, and an update to MANIFEST.in that excludes json in the spacy/lang directory. This focuses on Turkish specifically because it has the most language data in core. * Transition all lemmatizer.py files to json This covers all lemmatizer.py files of a significant size (>500k or so). Small files were left alone. None of the affected files have logic, so this was pretty straightforward. One unusual thing is that the lemma data for Urdu doesn't seem to be used anywhere. That may require further investigation. * Move large lang data to json for fr/nb/nl/sv These are the languages that use a lemmatizer directory (rather than a single file) and are larger than English. For most of these languages there were many language data files, in which case only the large ones (>500k or so) were converted to json. It may or may not be a good idea to migrate the remaining Python files to json in the future. * Fix id lemmas.json The contents of this file were originally just copied from the Python source, but that used single quotes, so it had to be properly converted to json first. * Add .json.gz to gitignore This covers the json.gz files built as part of distribution. * Add language data gzip to build process Currently this gzip data on every build; it works, but it should be changed to only gzip when the source file has been updated. * Remove Danish lemmatizer.py Missed this when I added the json. * Update to match latest explosion/srsly#9 The way gzipped json is loaded/saved in srsly changed a bit. * Only compress language data if necessary If a .json.gz file exists and is newer than the corresponding json file, it's not recompressed. * Move en/el language data to json This only affected files >500kb, which was nouns for both languages and the generic lookup table for English. * Remove empty files in Norwegian tokenizer It's unclear why, but the Norwegian (nb) tokenizer had empty files for adj/adv/noun/verb lemmas. This may have been a result of copying the structure of the English lemmatizer. This removed the files, but still creates the empty sets in the lemmatizer. That may not actually be necessary. * Remove dubious entries in English lookup.json " furthest" and " skilled" - both prefixed with a space - were in the English lookup table. That seems obviously wrong so I have removed them. * Fix small issues with en/fr lemmatizers The en tokenizer was including the removed _nouns.py file, so that's removed. The fr tokenizer is unusual in that it has a lemmatizer directory with both __init__.py and lemmatizer.py. lemmatizer.py had not been converted to load the json language data, so that was fixed. * Auto-format * Auto-format * Update srsly pin * Consistently use pathlib paths
2019-08-20 15:54:11 +03:00
["(ιρλανδικά)", "(σκωτικά)", "(σοράνι)", "-αλγία", "-βατώ", "-βατῶ", "-ούλα", "-πληξία", "-ώνυμο", "sofa", "table", "άβακας", "άβατο", "άβατον", "άβυσσος", "άγανο", "άγαρ", "άγγελμα", "άγγελος", "άγγιγμα", "άγγισμα", "άγγλος", "άγημα", "άγιασμα", "άγιο", "άγκλισμα", "άγκυρα", "άγμα", "άγνοια", "άγνωστος", "άγονο", "άγος", "άγουρος", "άγουσα", "άγρα", "άγρευμα", "άγρευσις", "άγρωστη", "άγχος", "άγχωση", "άδεια", "άδειασμα", "άδικο", "άδραγμα", "άδυτο", "άζωτο", "άζωτον", "άθεος", "άθλημα", "άθληση", "άθλησις", "άθλο", "άθλον", "άθλος", "άθος", "άθροιση", "άθροισμα", "άθυρμα", "άκανθα", "άκανθος", "άκαρι", "άκατος", "άκμονας", "άκμων", "άκουσμα", "άκρα", "άκρη", "άκρια", "άκρο", "άκρον", "άλβατρο", "άλβεδο", "άλγεβρα", "άλγη", "άλγος", "άλειμμα", "άλειψη", "άλεση", "άλεσις", "άλεσμα", "άλευρο", "άλικο", "άλκα", "άλκαλι", "άλκη", "άλλαγμα", "άλλαντα", "άλλεν", "άλλιο", "άλλοθι", "άλμα", "άλμη", "άλμπατρος", "άλμπουρο", "άλογο", "άλσος", "άλτης", "άλτο", "άλτρια", "άλυσις", "άλυσος", "άλφα", "άλφιτο", "άλωση", "άλωσις", "άμαξα", "άμαχος", "άμβικας", "άμβλυνση", "άμβλωση", "άμβλωσις", "άμβυκας", "άμβωνας", "άμιλλα", "άμμος", "άμνιο", "άμπακας", "άμπακος", "άμπελος", "άμπικας", "άμπωτη", "άμπωτις", "άμυλο", "άμυλον", "άμφιο", "άναμμα", "άναξ", "άνασσα", "άνδηρο", "άνδηρον", "άνδρας", "άνεμος", "άνεση", "άνεσις", "άνηθο", "άνθημα", "άνθηση", "άνθησις", "άνθι", "άνθιση", "άνθισις", "άνθισμα", "άνθος", "άνθρακας", "άνθραξ", "άνθρωπος", "άνοδος", "άνοια", "άνοιγμα", "άνοιξη", "άνοιξις", "άντερα", "άντερο", "άντζα", "άντληση", "άντλιον", "άντρακλας", "άντραρος", "άντρας", "άντρο", "άντρον", "άντωση", "άνυσμα", "άνω", "άνωση", "άνωσις", "άξονας", "άξων", "άουτ", "άουτο", "άπαις", "άπαν", "άπαντα", "άπαρση", "άπαρσις", "άπηξ", "άπιον", "άπλα", "άπλοια", "άπλυτα", "άπλωμα", "άπνοια", "άποικος", "άποψη", "άποψις", "άππαρος", "άπωση", "άρα", "άραβας", "άραγμα", "άραψ", "άρβυκας", "άρβυλο", "άργασμα", "άργητα", "άργιλος", "άρδευση", "άρδευσις", "άρθρα/αρχείο2", "άρθρο", "άρθρον", "άρθρωμα", "άρθρωση", "άρθρωσις", "άρια", "άρκαλος", "άρκευθος", "άρκος", "άρκτος", "άρμα", "άρμεγμα", "άρμενα", "άρμενο", "άρμενον", "άρμη", "άρμοση", "άρμοσις", "άρνηση", "άρνησις", "άροση", "άροτρο", "άρουλα", "άρουρα", "άρπα", "άρπαγας", "άρπαγμα", "άρπασμα", "άρρενας", "άρσις", "άρτος", "άρτυμα", "άρχοντας", "άρχος", "άρχων", "άρωμα", "άσβεστος", "άσθμα", "άσιος", "άσκαυλος", "άσκηση", "άσκησις", "άσμα", "άσος", "άσπρη", "άσπρισμα", "άσπρο", "άσπρουγας", "άσσος", "άστατο", "άστραμμα", "άστριος", "άστρο", "άστυ", "άσυλο", "άσφαλτος", "άτα", "άτι", "άτλαντας", "άτλας",
Reference in New Issue Copy Permalink