spaCy/spacy/tests/lang/uk/test_tokenizer_exc.py

import pytest


@pytest.mark.parametrize(
    "text,norms,lemmas",
    [("ім.", ["імені"], ["ім'я"]), ("проф.", ["професор"], ["професор"])],
)
def test_uk_tokenizer_abbrev_exceptions(uk_tokenizer, text, norms, lemmas):
    tokens = uk_tokenizer(text)
    assert len(tokens) == 1
    assert [token.norm_ for token in tokens] == norms
Ukrainian language added. Small fixes in Russian (#3241) * Classes for Ukrainian; small fix in Russian. * Contributor agreement 2019-02-07 23:05:11 +03:00			`import pytest`


Tidy up and fix small bugs and typos 2019-02-08 16:14:49 +03:00			`@pytest.mark.parametrize(`
			`"text,norms,lemmas",`
			`[("ім.", ["імені"], ["ім'я"]), ("проф.", ["професор"], ["професор"])],`
			`)`
Ukrainian language added. Small fixes in Russian (#3241) * Classes for Ukrainian; small fix in Russian. * Contributor agreement 2019-02-07 23:05:11 +03:00			`def test_uk_tokenizer_abbrev_exceptions(uk_tokenizer, text, norms, lemmas):`
			`tokens = uk_tokenizer(text)`
			`assert len(tokens) == 1`
			`assert [token.norm_ for token in tokens] == norms`