spaCy/spacy/lang/fr/lemmatizer/_lemma_rules.py

# coding: utf8
from __future__ import unicode_literals


ADJECTIVE_RULES = [
    ["s", ""],
    ["e", ""],
    ["es", ""] 
]
    
    
NOUN_RULES = [
    ["s", ""]
]


VERB_RULES = [
    ["é", "er"],
    ["és", "er"],
    ["ée", "er"],
    ["ées", "er"],
    ["é", "er"],
    ["es", "er"],
    ["ons", "er"],
    ["ez", "er"],
    ["ent", "er"],
    ["ais", "er"],
    ["ait", "er"],
    ["ions", "er"],
    ["iez", "er"],
    ["aient", "er"],
    ["ai", "er"],
    ["as", "er"],
    ["a", "er"],
    ["âmes", "er"],
    ["âtes", "er"],
    ["èrent", "er"],
    ["erai", "er"],
    ["eras", "er"],
    ["era", "er"],
    ["erons", "er"],
    ["erez", "er"],
    ["eront", "er"],
    ["erais", "er"],
    ["erait", "er"],
    ["erions", "er"],
    ["eriez", "er"],
    ["eraient", "er"],
    ["asse", "er"],
    ["asses", "er"],
    ["ât", "er"],
    ["assions", "er"],
    ["assiez", "er"],
    ["assent", "er"],
    ["ant", "er"]
]
Rule-based French Lemmatizer (#2818) <!--- Provide a general summary of your changes in the title. --> ## Description <!--- Use this section to describe your changes. If your changes required testing, include information about the testing environment and the tests you ran. If your test fixes a bug reported in an issue, don't forget to include the issue number. If your PR is still a work in progress, that's totally fine – just include a note to let us know. --> Add a rule-based French Lemmatizer following the english one and the excellent PR for [greek language optimizations](https://github.com/explosion/spaCy/pull/2558) to adapt the Lemmatizer class. ### Types of change <!-- What type of change does your PR cover? Is it a bug fix, an enhancement or new feature, or a change to the documentation? --> - Lemma dictionary used can be found [here](http://infolingu.univ-mlv.fr/DonneesLinguistiques/Dictionnaires/telechargement.html), I used the XML version. - Add several files containing exhaustive list of words for each part of speech - Add some lemma rules - Add POS that are not checked in the standard Lemmatizer, i.e PRON, DET, ADV and AUX - Modify the Lemmatizer class to check in lookup table as a last resort if POS not mentionned - Modify the lemmatize function to check in lookup table as a last resort - Init files are updated so the model can support all the functionalities mentioned above - Add words to tokenizer_exceptions_list.py in respect to regex used in tokenizer_exceptions.py ## Checklist <!--- Before you submit the PR, go over this checklist and make sure you can tick off all the boxes. [] -> [x] --> - [X] I have submitted the spaCy Contributor Agreement. - [X] I ran the tests, and all new and existing tests passed. - [X] My changes don't require a change to the documentation, or if they do, I've added all required information. 2018-10-13 17:38:21 +03:00			`# coding: utf8`
			`from __future__ import unicode_literals`


			`ADJECTIVE_RULES = [`
			`["s", ""],`
			`["e", ""],`
			`["es", ""]`
			`]`


			`NOUN_RULES = [`
			`["s", ""]`
			`]`


			`VERB_RULES = [`
			`["é", "er"],`
			`["és", "er"],`
			`["ée", "er"],`
			`["ées", "er"],`
			`["é", "er"],`
			`["es", "er"],`
			`["ons", "er"],`
			`["ez", "er"],`
			`["ent", "er"],`
			`["ais", "er"],`
			`["ait", "er"],`
			`["ions", "er"],`
			`["iez", "er"],`
			`["aient", "er"],`
			`["ai", "er"],`
			`["as", "er"],`
			`["a", "er"],`
			`["âmes", "er"],`
			`["âtes", "er"],`
			`["èrent", "er"],`
			`["erai", "er"],`
			`["eras", "er"],`
			`["era", "er"],`
			`["erons", "er"],`
			`["erez", "er"],`
			`["eront", "er"],`
			`["erais", "er"],`
			`["erait", "er"],`
			`["erions", "er"],`
			`["eriez", "er"],`
			`["eraient", "er"],`
			`["asse", "er"],`
			`["asses", "er"],`
			`["ât", "er"],`
			`["assions", "er"],`
			`["assiez", "er"],`
			`["assent", "er"],`
			`["ant", "er"]`
			`]`