spaCy/spacy/lang/lij/punctuation.py

from ..char_classes import ALPHA
from ..punctuation import TOKENIZER_INFIXES

ELISION = " ' ’ ".strip().replace(" ", "").replace("\n", "")


_infixes = TOKENIZER_INFIXES + [
    r"(?<=[{a}][{el}])(?=[{a}])".format(a=ALPHA, el=ELISION)
]

TOKENIZER_INFIXES = _infixes
-												Add Ligurian language

											
										
										
											2020-03-20 07:20:17 +03:00
+								from ..char_classes import ALPHA
-												Configure isort to use the Black profile, recursively isort the `spacy` module (#12721)

* Use isort with Black profile

* isort all the things

* Fix import cycles as a result of import sorting

* Add DOCBIN_ALL_ATTRS type definition

* Add isort to requirements

* Remove isort from build dependencies check

* Typo
											
										
										
											2023-06-14 18:48:41 +03:00
+								from ..punctuation import TOKENIZER_INFIXES
-												Add Ligurian language

											
										
										
											2020-03-20 07:20:17 +03:00
 								ELISION = " ' ’ ".strip().replace(" ", "").replace("\n", "")
 								_infixes = TOKENIZER_INFIXES + [
 								    r"(?<=[{a}][{el}])(?=[{a}])".format(a=ALPHA, el=ELISION)
 								]
 								TOKENIZER_INFIXES = _infixes