Pydantic settings support (#388)

* Add implementation and basic test

* Add full test coverage + bugfix

* Add test coverage for .from_yaml() method

* Update setup.py, tox and dev requirements

* Stop running pydantic tests on Python 3.5 and below

* Remove pydantic from tox Python < 3.6

* Add example and docs

* Update features block

* Add extra test

* Update changelog

README.rst

@@ -59,8 +59,8 @@ Key features of the ``Dependency Injector``:
 - **Overriding**. Can override any provider by another provider on the fly. This helps in testing
   and configuring dev / stage environment to replace API clients with stubs etc. See
   `Provider overriding <https://python-dependency-injector.ets-labs.org/providers/overriding.html>`_.
-- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, environment variables
+- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, ``pydantic`` settings,
-  and dictionaries.
+  environment variables, and dictionaries.
   See `Configuration provider <https://python-dependency-injector.ets-labs.org/providers/configuration.html>`_.
 - **Containers**. Provides declarative and dynamic containers.
   See `Containers <https://python-dependency-injector.ets-labs.org/containers/index.html>`_.

docs/index.rst

@@ -70,8 +70,8 @@ Key features of the ``Dependency Injector``:
 - **Overriding**. Can override any provider by another provider on the fly. This helps in testing
   and configuring dev / stage environment to replace API clients with stubs etc. See
   :ref:`provider-overriding`.
-- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, environment variables
+- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, ``pydantic`` settings,
-  and dictionaries. See :ref:`configuration-provider`.
+  environment variables, and dictionaries. See :ref:`configuration-provider`.
 - **Resources**. Helps with initialization and configuring of logging, event loop, thread
   or process pool, etc. Can be used for per-function execution scope in tandem with wiring.
   See :ref:`resource-provider`.

docs/introduction/key_features.rst

@@ -16,8 +16,8 @@ Key features of the ``Dependency Injector``:
 - **Overriding**. Can override any provider by another provider on the fly. This helps in testing
   and configuring dev / stage environment to replace API clients with stubs etc. See
   :ref:`provider-overriding`.
-- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, environment variables
+- **Configuration**. Reads configuration from ``yaml`` & ``ini`` files, ``pydantic`` settings,
-  and dictionaries. See :ref:`configuration-provider`.
+  environment variables, and dictionaries. See :ref:`configuration-provider`.
 - **Resources**. Helps with initialization and configuring of logging, event loop, thread
   or process pool, etc. Can be used for per-function execution scope in tandem with wiring.
   See :ref:`resource-provider`.

docs/main/changelog.rst

@@ -7,6 +7,10 @@ that were made in every particular version.
 From version 0.7.6 *Dependency Injector* framework strictly 
 follows `Semantic versioning`_
 
+Development version
+-------------------
+- Add ``Configuration.from_pydantic()`` method to load configuration from a ``pydantic`` settings.
+
 4.14.0
 ------
 - Add container providers traversal.

docs/providers/configuration.rst

@@ -5,10 +5,11 @@ Configuration provider
 
 .. meta::
    :keywords: Python,DI,Dependency injection,IoC,Inversion of Control,Configuration,Injection,
-              Option,Ini,Json,Yaml,Dict,Environment Variable,Load,Read,Get
+              Option,Ini,Json,Yaml,Pydantic,Dict,Environment Variable,Load,Read,Get
    :description: Configuration provides configuration options to the other providers. This page
                  demonstrates how to use Configuration provider to inject the dependencies, load
-                 a configuration from an ini or yaml file, dictionary or an environment variable.
+                 a configuration from an ini or yaml file, a dictionary, an environment variable,
+                 or a pydantic settings object.
 
 .. currentmodule:: dependency_injector.providers
 
@@ -89,6 +90,38 @@ You can also specify a YAML loader as an argument:
 
    *Don't forget to mirror the changes in the requirements file.*
 
+Loading from a Pydantic settings
+--------------------------------
+
+``Configuration`` provider can load configuration from a ``pydantic`` settings object using the
+:py:meth:`Configuration.from_pydantic` method:
+
+.. literalinclude:: ../../examples/providers/configuration/configuration_pydantic.py
+   :language: python
+   :lines: 3-
+   :emphasize-lines: 31
+
+To get the data from pydantic settings ``Configuration`` provider calls ``Settings.dict()`` method.
+If you need to pass an argument to this call, use ``.from_pydantic()`` keyword arguments.
+
+.. code-block:: python
+
+   container.config.from_pydantic(Settings(), exclude={'optional'})
+
+.. note::
+
+   ``Dependency Injector`` doesn't install ``pydantic`` by default.
+
+   You can install the ``Dependency Injector`` with an extra dependency::
+
+      pip install dependency-injector[pydantic]
+
+   or install ``pydantic`` directly::
+
+      pip install pydantic
+
+   *Don't forget to mirror the changes in the requirements file.*
+
 Loading from a dictionary
 -------------------------
 
@@ -211,7 +244,7 @@ Methods ``.from_*()`` in strict mode raise an exception if configuration file do
 configuration data is undefined:
 
 .. code-block:: python
-   :emphasize-lines: 10,15,20,25
+   :emphasize-lines: 10,15,20,25,30
 
    class Container(containers.DeclarativeContainer):
 
@@ -231,6 +264,11 @@ configuration data is undefined:
        except FileNotFoundError:
            ...
 
+       try:
+           container.config.from_pydantic(EmptySettings())  # raise exception
+       except ValueError:
+           ...
+
        try:
            container.config.from_env('UNDEFINED_ENV_VAR')  # raise exception
        except ValueError:

examples/providers/configuration/configuration_pydantic.py

@@ -0,0 +1,37 @@
+"""`Configuration` provider values loading example."""
+
+import os
+
+from dependency_injector import containers, providers
+from pydantic import BaseSettings, Field
+
+# Emulate environment variables
+os.environ['AWS_ACCESS_KEY_ID'] = 'KEY'
+os.environ['AWS_SECRET_ACCESS_KEY'] = 'SECRET'
+
+
+class AwsSettings(BaseSettings):
+
+    access_key_id: str = Field(env='aws_access_key_id')
+    secret_access_key: str = Field(env='aws_secret_access_key')
+
+
+class Settings(BaseSettings):
+
+    aws: AwsSettings = AwsSettings()
+    optional: str = Field(default='default_value')
+
+
+class Container(containers.DeclarativeContainer):
+
+    config = providers.Configuration()
+
+
+if __name__ == '__main__':
+    container = Container()
+
+    container.config.from_pydantic(Settings())
+
+    assert container.config.aws.access_key_id() == 'KEY'
+    assert container.config.aws.secret_access_key() == 'SECRET'
+    assert container.config.optional() == 'default_value'

requirements-dev.txt

@@ -10,5 +10,6 @@ mypy
 pyyaml
 httpx
 fastapi
+pydantic
 
 -r requirements-ext.txt

setup.py

@@ -64,6 +64,9 @@ setup(name='dependency-injector',
           'yaml': [
               'pyyaml',
           ],
+          'pydantic': [
+              'pydantic',
+          ],
           'flask': [
               'flask',
           ],

src/dependency_injector/providers.pyi

@@ -26,6 +26,11 @@ try:
 except ImportError:
     yaml = None
 
+try:
+    import pydantic
+except ImportError:
+    pydantic = None
+
 from . import resources
 
 
@@ -162,6 +167,7 @@ class ConfigurationOption(Provider[Any]):
     def update(self, value: Any) -> None: ...
     def from_ini(self, filepath: Union[Path, str], required: bool = False) -> None: ...
     def from_yaml(self, filepath: Union[Path, str], required: bool = False, loader: Optional[Any]=None) -> None: ...
+    def from_pydantic(self, settings: PydanticSettings, required: bool = False, **kwargs: Any) -> None: ...
     def from_dict(self, options: _Dict[str, Any], required: bool = False) -> None: ...
     def from_env(self, name: str, default: Optional[Any] = None, required: bool = False) -> None: ...
 
@@ -183,6 +189,7 @@ class Configuration(Object[Any]):
     def update(self, value: Any) -> None: ...
     def from_ini(self, filepath: Union[Path, str], required: bool = False) -> None: ...
     def from_yaml(self, filepath: Union[Path, str], required: bool = False, loader: Optional[Any]=None) -> None: ...
+    def from_pydantic(self, settings: PydanticSettings, required: bool = False, **kwargs: Any) -> None: ...
     def from_dict(self, options: _Dict[str, Any], required: bool = False) -> None: ...
     def from_env(self, name: str, default: Optional[Any] = None, required: bool = False) -> None: ...
 
@@ -397,3 +404,8 @@ if yaml:
     class YamlLoader(yaml.SafeLoader): ...
 else:
     class YamlLoader: ...
+
+if pydantic:
+    PydanticSettings = pydantic.BaseSettings
+else:
+    PydanticSettings = Any

src/dependency_injector/providers.pyx

@@ -35,6 +35,11 @@ try:
 except ImportError:
     yaml = None
 
+try:
+    import pydantic
+except ImportError:
+    pydantic = None
+
 from .errors import (
     Error,
     NoSuchProviderError,
@@ -1413,7 +1418,6 @@ cdef class ConfigurationOption(Provider):
                 '"pip install dependency-injector[yaml]"'
             )
 
-
         if loader is None:
             loader = YamlLoader
 
@@ -1433,6 +1437,43 @@ cdef class ConfigurationOption(Provider):
             current_config = {}
         self.override(merge_dicts(current_config, config))
 
+    def from_pydantic(self, settings, required=UNDEFINED, **kwargs):
+        """Load configuration from pydantic settings.
+
+        Loaded configuration is merged recursively over existing configuration.
+
+        :param settings: Pydantic settings instances.
+        :type settings: :py:class:`pydantic.BaseSettings`
+
+        :param required: When required is True, raise an exception if settings dict is empty.
+        :type required: bool
+
+        :param kwargs: Keyword arguments forwarded to ``pydantic.BaseSettings.dict()`` call.
+        :type kwargs: Dict[Any, Any]
+
+        :rtype: None
+        """
+        if pydantic is None:
+            raise Error(
+                'Unable to load pydantic configuration - pydantic is not installed. '
+                'Install pydantic or install Dependency Injector with pydantic extras: '
+                '"pip install dependency-injector[pydantic]"'
+            )
+
+        if isinstance(settings, CLASS_TYPES) and issubclass(settings, pydantic.BaseSettings):
+            raise Error(
+                'Got settings class, but expect instance: '
+                'instead "{0}" use "{0}()"'.format(settings.__name__)
+            )
+
+        if not isinstance(settings, pydantic.BaseSettings):
+            raise Error(
+                'Unable to recognize settings instance, expect "pydantic.BaseSettings", '
+                'got {0} instead'.format(settings)
+            )
+
+        self.from_dict(settings.dict(**kwargs), required=required)
+
     def from_dict(self, options, required=UNDEFINED):
         """Load configuration from the dictionary.
 
@@ -1766,6 +1807,43 @@ cdef class Configuration(Object):
             current_config = {}
         self.override(merge_dicts(current_config, config))
 
+    def from_pydantic(self, settings, required=UNDEFINED, **kwargs):
+        """Load configuration from pydantic settings.
+
+        Loaded configuration is merged recursively over existing configuration.
+
+        :param settings: Pydantic settings instances.
+        :type settings: :py:class:`pydantic.BaseSettings`
+
+        :param required: When required is True, raise an exception if settings dict is empty.
+        :type required: bool
+
+        :param kwargs: Keyword arguments forwarded to ``pydantic.BaseSettings.dict()`` call.
+        :type kwargs: Dict[Any, Any]
+
+        :rtype: None
+        """
+        if pydantic is None:
+            raise Error(
+                'Unable to load pydantic configuration - pydantic is not installed. '
+                'Install pydantic or install Dependency Injector with pydantic extras: '
+                '"pip install dependency-injector[pydantic]"'
+            )
+
+        if isinstance(settings, CLASS_TYPES) and issubclass(settings, pydantic.BaseSettings):
+            raise Error(
+                'Got settings class, but expect instance: '
+                'instead "{0}" use "{0}()"'.format(settings.__name__)
+            )
+
+        if not isinstance(settings, pydantic.BaseSettings):
+            raise Error(
+                'Unable to recognize settings instance, expect "pydantic.BaseSettings", '
+                'got {0} instead'.format(settings)
+            )
+
+        self.from_dict(settings.dict(**kwargs), required=required)
+
     def from_dict(self, options, required=UNDEFINED):
         """Load configuration from the dictionary.
 

tests/unit/providers/test_configuration_py2_py3.py

@@ -14,6 +14,11 @@ try:
 except ImportError:
     yaml = None
 
+try:
+    import pydantic
+except ImportError:
+    pydantic = None
+
 
 class ConfigTests(unittest.TestCase):
 
@@ -642,6 +647,27 @@ class ConfigFromYamlTests(unittest.TestCase):
             '"pip install dependency-injector[yaml]"',
         )
 
+    def test_option_no_yaml_installed(self):
+        @contextlib.contextmanager
+        def no_yaml_module():
+            yaml = providers.yaml
+            providers.yaml = None
+
+            yield
+
+            providers.yaml = yaml
+
+        with no_yaml_module():
+            with self.assertRaises(errors.Error) as error:
+                self.config.option.from_yaml(self.config_file_1)
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Unable to load yaml configuration - PyYAML is not installed. '
+            'Install PyYAML or install Dependency Injector with yaml extras: '
+            '"pip install dependency-injector[yaml]"',
+        )
+
 
 class ConfigFromYamlWithEnvInterpolationTests(unittest.TestCase):
 
@@ -723,6 +749,216 @@ class ConfigFromYamlWithEnvInterpolationTests(unittest.TestCase):
         self.assertEqual(self.config.option.section1.value1(), 'test-value')
 
 
+class ConfigFromPydanticTests(unittest.TestCase):
+
+    def setUp(self):
+        self.config = providers.Configuration(name='config')
+
+        class Section11(pydantic.BaseModel):
+            value1 = 1
+
+        class Section12(pydantic.BaseModel):
+            value2 = 2
+
+        class Settings1(pydantic.BaseSettings):
+            section1 = Section11()
+            section2 = Section12()
+
+        self.Settings1 = Settings1
+
+        class Section21(pydantic.BaseModel):
+            value1 = 11
+            value11 = 11
+
+        class Section3(pydantic.BaseModel):
+            value3 = 3
+
+        class Settings2(pydantic.BaseSettings):
+            section1 = Section21()
+            section3 = Section3()
+
+        self.Settings2 = Settings2
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test(self):
+        self.config.from_pydantic(self.Settings1())
+
+        self.assertEqual(self.config(), {'section1': {'value1': 1}, 'section2': {'value2': 2}})
+        self.assertEqual(self.config.section1(), {'value1': 1})
+        self.assertEqual(self.config.section1.value1(), 1)
+        self.assertEqual(self.config.section2(), {'value2': 2})
+        self.assertEqual(self.config.section2.value2(), 2)
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_kwarg(self):
+        self.config.from_pydantic(self.Settings1(), exclude={'section2'})
+
+        self.assertEqual(self.config(), {'section1': {'value1': 1}})
+        self.assertEqual(self.config.section1(), {'value1': 1})
+        self.assertEqual(self.config.section1.value1(), 1)
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_merge(self):
+        self.config.from_pydantic(self.Settings1())
+        self.config.from_pydantic(self.Settings2())
+
+        self.assertEqual(
+            self.config(),
+            {
+                'section1': {
+                    'value1': 11,
+                    'value11': 11,
+                },
+                'section2': {
+                    'value2': 2,
+                },
+                'section3': {
+                    'value3': 3,
+                },
+            },
+        )
+        self.assertEqual(self.config.section1(), {'value1': 11, 'value11': 11})
+        self.assertEqual(self.config.section1.value1(), 11)
+        self.assertEqual(self.config.section1.value11(), 11)
+        self.assertEqual(self.config.section2(), {'value2': 2})
+        self.assertEqual(self.config.section2.value2(), 2)
+        self.assertEqual(self.config.section3(), {'value3': 3})
+        self.assertEqual(self.config.section3.value3(), 3)
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_empty_settings(self):
+        self.config.from_pydantic(pydantic.BaseSettings())
+        self.assertEqual(self.config(), {})
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_empty_settings_strict_mode(self):
+        self.config = providers.Configuration(strict=True)
+        with self.assertRaises(ValueError):
+            self.config.from_pydantic(pydantic.BaseSettings())
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_option_empty_settings(self):
+        self.config.option.from_pydantic(pydantic.BaseSettings())
+        self.assertEqual(self.config.option(), {})
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_option_empty_settings_strict_mode(self):
+        self.config = providers.Configuration(strict=True)
+        with self.assertRaises(ValueError):
+            self.config.option.from_pydantic(pydantic.BaseSettings())
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_required_empty_settings(self):
+        with self.assertRaises(ValueError):
+            self.config.from_pydantic(pydantic.BaseSettings(), required=True)
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_required_option_empty_settings(self):
+        with self.assertRaises(ValueError):
+            self.config.option.from_pydantic(pydantic.BaseSettings(), required=True)
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_not_required_empty_settings_strict_mode(self):
+        self.config = providers.Configuration(strict=True)
+        self.config.from_pydantic(pydantic.BaseSettings(), required=False)
+        self.assertEqual(self.config(), {})
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_not_required_option_empty_settings_strict_mode(self):
+        self.config = providers.Configuration(strict=True)
+        self.config.option.from_pydantic(pydantic.BaseSettings(), required=False)
+        self.assertEqual(self.config.option(), {})
+        self.assertEqual(self.config(), {'option': {}})
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_not_instance_of_settings(self):
+        with self.assertRaises(errors.Error) as error:
+            self.config.from_pydantic({})
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Unable to recognize settings instance, expect "pydantic.BaseSettings", '
+            'got {0} instead'.format({})
+        )
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_option_not_instance_of_settings(self):
+        with self.assertRaises(errors.Error) as error:
+            self.config.option.from_pydantic({})
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Unable to recognize settings instance, expect "pydantic.BaseSettings", '
+            'got {0} instead'.format({})
+        )
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_subclass_instead_of_instance(self):
+        with self.assertRaises(errors.Error) as error:
+            self.config.from_pydantic(self.Settings1)
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Got settings class, but expect instance: '
+            'instead "Settings1" use "Settings1()"'
+        )
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_option_subclass_instead_of_instance(self):
+        with self.assertRaises(errors.Error) as error:
+            self.config.option.from_pydantic(self.Settings1)
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Got settings class, but expect instance: '
+            'instead "Settings1" use "Settings1()"'
+        )
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_no_pydantic_installed(self):
+        @contextlib.contextmanager
+        def no_pydantic_module():
+            pydantic = providers.pydantic
+            providers.pydantic = None
+
+            yield
+
+            providers.pydantic = pydantic
+
+        with no_pydantic_module():
+            with self.assertRaises(errors.Error) as error:
+                self.config.from_pydantic(self.Settings1())
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Unable to load pydantic configuration - pydantic is not installed. '
+            'Install pydantic or install Dependency Injector with pydantic extras: '
+            '"pip install dependency-injector[pydantic]"',
+        )
+
+    @unittest.skipIf(sys.version_info[:2] < (3, 6), 'Pydantic supports Python 3.6+')
+    def test_option_no_pydantic_installed(self):
+        @contextlib.contextmanager
+        def no_pydantic_module():
+            pydantic = providers.pydantic
+            providers.pydantic = None
+
+            yield
+
+            providers.pydantic = pydantic
+
+        with no_pydantic_module():
+            with self.assertRaises(errors.Error) as error:
+                self.config.option.from_pydantic(self.Settings1())
+
+        self.assertEqual(
+            error.exception.args[0],
+            'Unable to load pydantic configuration - pydantic is not installed. '
+            'Install pydantic or install Dependency Injector with pydantic extras: '
+            '"pip install dependency-injector[pydantic]"',
+        )
+
+
 class ConfigFromDict(unittest.TestCase):
 
     def setUp(self):

tox.ini

@@ -11,6 +11,7 @@ deps=
     fastapi
 extras=
     yaml
+    pydantic
     flask
     aiohttp
 commands=