python-dependency-injector/docs/tutorials/aiohttp.rst
Roman Mogylatov d2f4585a8b
Aiohttp tutorial (#273)
* Make a code style change to the giphynav-aiohttp app

* Make minimal punctuation changes for the flask tutorial

* Add parts of http tutorial

* Fix few issues in the flask tutorial

* Make some cosmetic changes to test data

* Fix typo in flask tutorial

* Add more tutorial sections
2020-07-29 22:17:56 -04:00

965 lines
24 KiB
ReStructuredText
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Aiohttp tutorial
================
.. _aiohttp-tutorial:
This tutorials shows how to build an ``aiohttp`` REST API application following the dependency
injection principle.
Start from the scratch or jump to the section:
.. contents::
:local:
:backlinks: none
You can find complete project on the
`Github <https://github.com/ets-labs/python-dependency-injector/tree/master/examples/miniapps/giphynav-aiohttp>`_.
What are we going to build?
---------------------------
.. image:: https://media.giphy.com/media/apvx5lPCPsjN6/source.gif
We will build a REST API application that searches for funny GIFs on the `Giphy <https://giphy.com/>`_.
Let's call it Giphy Navigator.
How does Giphy Navigator work?
- Client sends a request specifying the search query and the number of results.
- Giphy Navigator returns a response in json format.
- The response contains:
- the search query
- the limit number
- the list of gif urls
Example response:
.. code-block:: json
{
"query": "Dependency Injector",
"limit": 10,
"gifs": [
{
"url": "https://giphy.com/gifs/boxes-dependent-swbf2-6Eo7KzABxgJMY"
},
{
"url": "https://giphy.com/gifs/depends-J56qCcOhk6hKE"
},
{
"url": "https://giphy.com/gifs/web-series-ccstudios-bro-dependent-1lhU8KAVwmVVu"
},
{
"url": "https://giphy.com/gifs/TheBoysTV-friends-friend-weneedeachother-XxR9qcIwcf5Jq404Sx"
},
{
"url": "https://giphy.com/gifs/netflix-a-series-of-unfortunate-events-asoue-9rgeQXbwoK53pcxn7f"
},
{
"url": "https://giphy.com/gifs/black-and-white-sad-skins-Hs4YzLs2zJuLu"
},
{
"url": "https://giphy.com/gifs/always-there-for-you-i-am-here-PlayjhCco9jHBYrd9w"
},
{
"url": "https://giphy.com/gifs/stream-famous-dollar-YT2dvOByEwXCdoYiA1"
},
{
"url": "https://giphy.com/gifs/i-love-you-there-for-am-1BhGzgpZXYWwWMAGB1"
},
{
"url": "https://giphy.com/gifs/life-like-twerk-9hlnWxjHqmH28"
}
]
}
The task is naive and that's exactly what we need for the tutorial.
Prepare the environment
-----------------------
Let's create the environment for the project.
First we need to create a project folder and the virtual environment:
.. code-block:: bash
mkdir giphynav-aiohttp-tutorial
cd giphynav-aiohttp-tutorial
python3 -m venv venv
Now let's activate the virtual environment:
.. code-block:: bash
. venv/bin/activate
Environment is ready and now we're going to create the layout of the project.
Project layout
--------------
Create next structure in the current directory. All files should be empty. That's ok for now.
Initial project layout::
./
├── giphynavigator/
│ ├── __init__.py
│ ├── application.py
│ ├── containers.py
│ └── views.py
├── venv/
└── requirements.txt
Install the requirements
------------------------
Now it's time to install the project requirements. We will use next packages:
- ``dependency-injector`` - the dependency injection framework
- ``aiohttp`` - the web framework
- ``aiohttp-devtools`` - the helper library that will provide a development server with live
reloading
- ``pyyaml`` - the YAML files parsing library, used for the reading of the configuration files
- ``pytest-aiohttp``- the helper library for the testing of the ``aiohttp`` application
- ``pytest-cov`` - the helper library for measuring the test coverage
Put next lines into the ``requirements.txt`` file:
.. code-block:: bash
dependency-injector
aiohttp
aiohttp-devtools
pyyaml
pytest-aiohttp
pytest-cov
and run next in the terminal:
.. code-block:: bash
pip install -r requirements.txt
Let's also install the ``httpie``. It is a user-friendly command-line HTTP client for the API era.
We will use it for the manual testing.
Run the command in the terminal:
.. code-block:: bash
pip install httpie
The requirements are setup. Now we will build a minimal application.
Minimal application
-------------------
In this section we will build a minimal application. It will have an endpoint that we can call.
The endpoint will answer in the right format and will have no data.
Edit ``views.py``:
.. code-block:: python
"""Views module."""
from aiohttp import web
async def index(request: web.Request) -> web.Response:
query = request.query.get('query', 'Dependency Injector')
limit = int(request.query.get('limit', 10))
gifs = []
return web.json_response(
{
'query': query,
'limit': limit,
'gifs': gifs,
},
)
Now let's create the main part of our application - the container. Container will keep all of the
application components and their dependencies. First two providers we need to add are
the ``aiohttp`` application provider and the view provider.
Put next into the ``containers.py``:
.. code-block:: python
"""Application containers module."""
from dependency_injector import containers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
index_view = aiohttp.View(views.index)
At the last we need to create the ``aiohttp`` application factory. It is traditionally called
``create_app()``. It will create the container. Then it will use the container to create
the ``aiohttp`` application. Last step is to configure the routing - we will assign
``index_view`` from the container to handle the requests to the root ``/`` of our REST API server.
Put next into the ``application.py``:
.. code-block:: python
"""Application module."""
from aiohttp import web
from .containers import ApplicationContainer
def create_app():
"""Create and return Flask application."""
container = ApplicationContainer()
app: web.Application = container.app()
app.container = container
app.add_routes([
web.get('/', container.index_view.as_view()),
])
return app
.. note::
Container is the first object in the application.
The container is used to create all other objects.
Now we're ready to run our application
Do next in the terminal:
.. code-block:: bash
adev runserver giphynavigator/application.py --livereload
The output should be something like:
.. code-block:: bash
[18:52:59] Starting aux server at http://localhost:8001 ◆
[18:52:59] Starting dev server at http://localhost:8000 ●
Let's use ``httpie`` to check that it works:
.. code-block:: bash
http http://127.0.0.1:8000/
You should see:
.. code-block:: json
HTTP/1.1 200 OK
Content-Length: 844
Content-Type: application/json; charset=utf-8
Date: Wed, 29 Jul 2020 21:01:50 GMT
Server: Python/3.8 aiohttp/3.6.2
{
"gifs": [],
"limit": 10,
"query": "Dependency Injector"
}
Minimal application is ready. Let's connect our application with the Giphy API.
Giphy API client
----------------
In this section we will integrate our application with the Giphy API.
We will create our own API client using ``aiohttp`` client.
Create ``giphy.py`` module in the ``giphynavigator`` package:
.. code-block:: bash
:emphasize-lines: 6
./
├── giphynavigator/
│ ├── __init__.py
│ ├── application.py
│ ├── containers.py
│ ├── giphy.py
│ └── views.py
├── venv/
└── requirements.txt
and put next into it:
.. code-block:: python
"""Giphy client module."""
from aiohttp import ClientSession, ClientTimeout
class GiphyClient:
API_URL = 'http://api.giphy.com/v1'
def __init__(self, api_key, timeout):
self._api_key = api_key
self._timeout = ClientTimeout(timeout)
async def search(self, query, limit):
"""Make search API call and return result."""
if not query:
return []
url = f'{self.API_URL}/gifs/search'
params = {
'q': query,
'api_key': self._api_key,
'limit': limit,
}
async with ClientSession(timeout=self._timeout) as session:
async with session.get(url, params=params) as response:
if response.status != 200:
response.raise_for_status()
return await response.json()
Now we need to add ``GiphyClient`` into the container. The ``GiphyClient`` has two dependencies
that have to be injected: the API key and the request timeout. We will need to use two more
providers from the ``dependency_injector.providers`` module:
- ``Factory`` provider that will create the ``GiphyClient`` client.
- ``Configuration`` provider that will provide the API key and the request timeout.
Edit ``containers.py``:
.. code-block:: python
:emphasize-lines: 3,7,15,17-21
"""Application containers module."""
from dependency_injector import containers, providers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import giphy, views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
config = providers.Configuration()
giphy_client = providers.Factory(
giphy.GiphyClient,
api_key=config.giphy.api_key,
timeout=config.giphy.request_timeout,
)
index_view = aiohttp.View(views.index)
.. note::
We have used the configuration value before it was defined. That's the principle how the
``Configuration`` provider works.
Use first, define later.
Now let's add the configuration file.
We will use YAML.
Create an empty file ``config.yml`` in the root root of the project:
.. code-block:: bash
:emphasize-lines: 9
./
├── giphynavigator/
│ ├── __init__.py
│ ├── application.py
│ ├── containers.py
│ ├── giphy.py
│ └── views.py
├── venv/
├── config.yml
└── requirements.txt
and put next into it:
.. code-block:: yaml
giphy:
request_timeout: 10
We will use an environment variable ``GIPHY_API_KEY`` to provide the API key.
Now we need to edit ``create_app()`` to make two things when application starts:
- Load the configuration file the ``config.yml``.
- Load the API key from the ``GIPHY_API_KEY`` environment variable.
Edit ``application.py``:
.. code-block:: python
:emphasize-lines: 11-12
"""Application module."""
from aiohttp import web
from .containers import ApplicationContainer
def create_app():
"""Create and return Flask application."""
container = ApplicationContainer()
container.config.from_yaml('config.yml')
container.config.giphy.api_key.from_env('GIPHY_API_KEY')
app: web.Application = container.app()
app.container = container
app.add_routes([
web.get('/', container.index_view.as_view()),
])
return app
Now we need to create an API key and set it to the environment variable.
As for now, dont worry, just take this one:
.. code-block:: bash
export GIPHY_API_KEY=wBJ2wZG7SRqfrU9nPgPiWvORmloDyuL0
.. note::
To create your own Giphy API key follow this
`guide <https://support.giphy.com/hc/en-us/articles/360020283431-Request-A-GIPHY-API-Key>`_.
The Giphy API client and the configuration setup is done. Let's proceed to the search service.
Search service
--------------
Now it's time to add the ``SearchService``. It will:
- Perform the search.
- Format result data.
``SearchService`` will use ``GiphyClient``.
Create ``services.py`` module in the ``giphynavigator`` package:
.. code-block:: bash
:emphasize-lines: 7
./
├── giphynavigator/
│ ├── __init__.py
│ ├── application.py
│ ├── containers.py
│ ├── giphy.py
│ ├── services.py
│ └── views.py
├── venv/
└── requirements.txt
and put next into it:
.. code-block:: python
"""Services module."""
from .giphy import GiphyClient
class SearchService:
def __init__(self, giphy_client: GiphyClient):
self._giphy_client = giphy_client
async def search(self, query, limit):
"""Search for gifs and return formatted data."""
if not query:
return []
result = await self._giphy_client.search(query, limit)
return [{'url': gif['url']} for gif in result['data']]
The ``SearchService`` has a dependency on the ``GiphyClient``. This dependency will be injected.
Let's add ``SearchService`` to the container.
Edit ``containers.py``:
.. code-block:: python
:emphasize-lines: 7,23-26
"""Application containers module."""
from dependency_injector import containers, providers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import giphy, services, views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
config = providers.Configuration()
giphy_client = providers.Factory(
giphy.GiphyClient,
api_key=config.giphy.api_key,
timeout=config.giphy.request_timeout,
)
search_service = providers.Factory(
services.SearchService,
giphy_client=giphy_client,
)
index_view = aiohttp.View(views.index)
The search service is ready. In the next section we're going to make it work.
Make the search work
--------------------
Now we are ready to make the search work. Let's use the ``SearchService`` in the ``index`` view.
Edit ``views.py``:
.. code-block:: python
:emphasize-lines: 5,8-11,15
"""Views module."""
from aiohttp import web
from .services import SearchService
async def index(
request: web.Request,
search_service: SearchService,
) -> web.Response:
query = request.query.get('query', 'Dependency Injector')
limit = int(request.query.get('limit', 10))
gifs = await search_service.search(query, limit)
return web.json_response(
{
'query': query,
'limit': limit,
'gifs': gifs,
},
)
Now let's inject the ``SearchService`` dependency into the ``index`` view.
Edit ``containers.py``:
.. code-block:: python
:emphasize-lines: 28-31
"""Application containers module."""
from dependency_injector import containers, providers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import giphy, services, views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
config = providers.Configuration()
giphy_client = providers.Factory(
giphy.GiphyClient,
api_key=config.giphy.api_key,
timeout=config.giphy.request_timeout,
)
search_service = providers.Factory(
services.SearchService,
giphy_client=giphy_client,
)
index_view = aiohttp.View(
views.index,
search_service=search_service,
)
Make sure the app is running or use:
.. code-block:: bash
adev runserver giphynavigator/application.py --livereload
and make a request to the API in the terminal:
.. code-block:: bash
http http://localhost:8000/ query=="wow,it works" limit==5
You should see:
.. code-block:: json
HTTP/1.1 200 OK
Content-Length: 850
Content-Type: application/json; charset=utf-8
Date: Wed, 29 Jul 2020 22:22:55 GMT
Server: Python/3.8 aiohttp/3.6.2
{
"gifs": [
{
"url": "https://giphy.com/gifs/discoverychannel-nugget-gold-rush-rick-ness-KGGPIlnC4hr4u2s3pY"
},
{
"url": "https://giphy.com/gifs/primevideoin-ll1hyBS2IrUPLE0E71"
},
{
"url": "https://giphy.com/gifs/jackman-works-jackmanworks-l4pTgQoCrmXq8Txlu"
},
{
"url": "https://giphy.com/gifs/cat-massage-at-work-l46CzMaOlJXAFuO3u"
},
{
"url": "https://giphy.com/gifs/everwhatproductions-fun-christmas-3oxHQCI8tKXoeW4IBq"
},
],
"limit": 10,
"query": "wow,it works"
}
.. image:: https://media.giphy.com/media/3oxHQCI8tKXoeW4IBq/source.gif
The search works!
Make some refactoring
---------------------
Our ``index`` view has two hardcoded config values:
- Default search query
- Default results limit
Let's make some refactoring. We will move these values to the config.
Edit ``views.py``:
.. code-block:: python
:emphasize-lines: 11-12,14-15
"""Views module."""
from aiohttp import web
from .services import SearchService
async def index(
request: web.Request,
search_service: SearchService,
default_query: str,
default_limit: int,
) -> web.Response:
query = request.query.get('query', default_query)
limit = int(request.query.get('limit', default_limit))
gifs = await search_service.search(query, limit)
return web.json_response(
{
'query': query,
'limit': limit,
'gifs': gifs,
},
)
Now we need to inject these values. Let's update the container.
Edit ``containers.py``:
.. code-block:: python
:emphasize-lines: 31-32
"""Application containers module."""
from dependency_injector import containers, providers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import giphy, services, views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
config = providers.Configuration()
giphy_client = providers.Factory(
giphy.GiphyClient,
api_key=config.giphy.api_key,
timeout=config.giphy.request_timeout,
)
search_service = providers.Factory(
services.SearchService,
giphy_client=giphy_client,
)
index_view = aiohttp.View(
views.index,
search_service=search_service,
default_query=config.search.default_query,
default_limit=config.search.default_limit,
)
Finally let's update the config.
Edit ``config.yml``:
.. code-block:: yaml
:emphasize-lines: 3-5
giphy:
request_timeout: 10
search:
default_query: "Dependency Injector"
default_limit: 10
The refactoring is done. We've made it cleaner - hardcoded values are now moved to the config.
In the next section we will add some tests.
Tests
-----
It would be nice to add some tests. Let's do it.
We will use `pytest <https://docs.pytest.org/en/stable/>`_ and
`coverage <https://coverage.readthedocs.io/>`_.
Create ``tests.py`` module in the ``giphynavigator`` package:
.. code-block:: bash
:emphasize-lines: 8
./
├── giphynavigator/
│ ├── __init__.py
│ ├── application.py
│ ├── containers.py
│ ├── giphy.py
│ ├── services.py
│ ├── tests.py
│ └── views.py
├── venv/
└── requirements.txt
and put next into it:
.. code-block:: python
:emphasize-lines: 30,57,71
"""Tests module."""
from unittest import mock
import pytest
from giphynavigator.application import create_app
from giphynavigator.giphy import GiphyClient
@pytest.fixture
def app():
return create_app()
@pytest.fixture
def client(app, aiohttp_client, loop):
return loop.run_until_complete(aiohttp_client(app))
async def test_index(client, app):
giphy_client_mock = mock.AsyncMock(spec=GiphyClient)
giphy_client_mock.search.return_value = {
'data': [
{'url': 'https://giphy.com/gif1.gif'},
{'url': 'https://giphy.com/gif2.gif'},
],
}
with app.container.giphy_client.override(giphy_client_mock):
response = await client.get(
'/',
params={
'query': 'test',
'limit': 10,
},
)
assert response.status == 200
data = await response.json()
assert data == {
'query': 'test',
'limit': 10,
'gifs': [
{'url': 'https://giphy.com/gif1.gif'},
{'url': 'https://giphy.com/gif2.gif'},
],
}
async def test_index_no_data(client, app):
giphy_client_mock = mock.AsyncMock(spec=GiphyClient)
giphy_client_mock.search.return_value = {
'data': [],
}
with app.container.giphy_client.override(giphy_client_mock):
response = await client.get('/')
assert response.status == 200
data = await response.json()
assert data['gifs'] == []
async def test_index_default_params(client, app):
giphy_client_mock = mock.AsyncMock(spec=GiphyClient)
giphy_client_mock.search.return_value = {
'data': [],
}
with app.container.giphy_client.override(giphy_client_mock):
response = await client.get('/')
assert response.status == 200
data = await response.json()
assert data['query'] == app.container.config.search.default_query()
assert data['limit'] == app.container.config.search.default_limit()
Now let's run it and check the coverage:
.. code-block:: bash
py.test giphynavigator/tests.py --cov=giphynavigator
You should see:
.. code-block:: bash
platform darwin -- Python 3.8.3, pytest-5.4.3, py-1.9.0, pluggy-0.13.1
plugins: cov-2.10.0, aiohttp-0.3.0, asyncio-0.14.0
collected 3 items
giphynavigator/tests.py ... [100%]
---------- coverage: platform darwin, python 3.8.3-final-0 -----------
Name Stmts Miss Cover
---------------------------------------------------
giphynavigator/__init__.py 0 0 100%
giphynavigator/__main__.py 5 5 0%
giphynavigator/application.py 10 0 100%
giphynavigator/containers.py 10 0 100%
giphynavigator/giphy.py 16 11 31%
giphynavigator/services.py 9 1 89%
giphynavigator/tests.py 35 0 100%
giphynavigator/views.py 7 0 100%
---------------------------------------------------
TOTAL 92 17 82%
.. note::
Take a look at the highlights in the ``tests.py``.
It emphasizes the overriding of the ``GiphyClient``. The real API call are mocked.
Conclusion
----------
In this tutorial we've build an ``aiohttp`` REST API application following the dependency
injection principle.
We've used ``Dependency Injector`` as a dependency injection framework.
The benefit you get with the ``Dependency Injector`` is the container. It starts to payoff
when you need to understand or change your application structure. It's easy with the container,
cause you have everything in one place:
.. code-block:: python
"""Application containers module."""
from dependency_injector import containers, providers
from dependency_injector.ext import aiohttp
from aiohttp import web
from . import giphy, services, views
class ApplicationContainer(containers.DeclarativeContainer):
"""Application container."""
app = aiohttp.Application(web.Application)
config = providers.Configuration()
giphy_client = providers.Factory(
giphy.GiphyClient,
api_key=config.giphy.api_key,
timeout=config.giphy.request_timeout,
)
search_service = providers.Factory(
services.SearchService,
giphy_client=giphy_client,
)
index_view = aiohttp.View(
views.index,
search_service=search_service,
default_query=config.search.default_query,
default_limit=config.search.default_limit,
)
What's next?
- Look at the other :ref:`tutorials`.
- Know more about the :ref:`providers`.
- Go to the :ref:`contents`.
.. disqus::