diff --git a/README.rst b/README.rst index 30579d70..2c3160af 100644 --- a/README.rst +++ b/README.rst @@ -37,6 +37,7 @@ Features * Pre configured Celery_ (optional) * Integration with Maildump_ for local email testing (optional) * Integration with Sentry_ for error logging (optional) +* Docker support using docker-compose_ for dev and prod .. _Hitch: https://github.com/hitchtest/hitchtest .. _Bootstrap: https://github.com/twbs/bootstrap @@ -51,6 +52,7 @@ Features .. _Celery: http://www.celeryproject.org/ .. _Maildump: https://github.com/ThiefMaster/maildump .. _Sentry: https://getsentry.com +.. _docker-compose: https://www.github.com/docker/compose Constraints @@ -166,6 +168,50 @@ To get live reloading to work you'll probably need to install an `appropriate br It's time to write the code!!! +Getting up and running using docker +---------------------------------- + +The steps below will get you up and running with a local development environment. We assume you have the following installed: + +* docker +* docker-compose + +Open a terminal at the project root and run the following for local development:: + + $ docker-compose -f dev.yml up + +You can also set the environment variable ``COMPOSE_FILE`` pointing to ``dev.yml`` like this:: + + $ export COMPOSE_FILE=dev.yml + +And then run:: + + $ docker-compose up + + +To migrate your app and to create a superuser, run:: + + $ docker-compose run django python manage.py migrate + + $ docker-compose run django python manage.py createsuperuser + + +If you are using `boot2docker` to develop on OS X or Windows, you need to create a `/data` partition inside your boot2docker +vm to make all changes persistent. If you don't do that your `/data` directory will get wiped out on every reboot. + +To create a persistent folder, log into the `boot2docker` vm by running:: + + $ bootdocker ssh + +And then:: + + $ sudo su + $ echo 'ln -sfn /mnt/sda1/data /data' >> /var/lib/boot2docker/bootlocal.sh + +In case you are wondering why you can't use a host volume to keep the files on your mac: As of `boot2docker` 1.7 you'll +run into permission problems with mounted host volumes if the container creates his own user and `chown`s the directories +on the volume. Postgres is doing that, so we need this quick fix to ensure that all development data persists. + For Readers of Two Scoops of Django 1.8 -------------------------------------------- diff --git a/{{cookiecutter.repo_name}}/Dockerfile b/{{cookiecutter.repo_name}}/Dockerfile new file mode 100644 index 00000000..547a92b3 --- /dev/null +++ b/{{cookiecutter.repo_name}}/Dockerfile @@ -0,0 +1,23 @@ +FROM python:2.7 +ENV PYTHONUNBUFFERED 1 + +# Requirements have to be pulled and installed here, otherwise caching won't work +ADD ./requirements /requirements +ADD ./requirements.txt /requirements.txt + +RUN pip install -r /requirements.txt +RUN pip install -r /requirements/local.txt + +RUN groupadd -r django && useradd -r -g django django +ADD . /app +RUN chown -R django /app + +ADD ./compose/django/gunicorn.sh /gunicorn.sh +ADD ./compose/django/entrypoint.sh /entrypoint.sh + +RUN chmod +x /entrypoint.sh && chown django /entrypoint.sh +RUN chmod +x /gunicorn.sh && chown django /gunicorn.sh + +WORKDIR /app + +ENTRYPOINT ["/entrypoint.sh"] \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/README.rst b/{{cookiecutter.repo_name}}/README.rst index 81f5d58e..20030c44 100644 --- a/{{cookiecutter.repo_name}}/README.rst +++ b/{{cookiecutter.repo_name}}/README.rst @@ -200,7 +200,7 @@ The testing framework runs Django, Celery (if enabled), Postgres, HitchSMTP (a m Deployment ---------- -It is possible to deploy to Heroku or to your own server by using Dokku, an open source Heroku clone. +It is possible to deploy to Heroku, to your own server by using Dokku, an open source Heroku clone or using docker-compose. Heroku ^^^^^^ @@ -277,3 +277,116 @@ You can then deploy by running the following commands. ssh -t dokku@yourservername.com dokku run {{cookiecutter.repo_name}} python manage.py createsuperuser When deploying via Dokku make sure you backup your database in some fashion as it is NOT done automatically. + +Docker +^^^^^^ + +**Warning** + +Docker is evolving extremely fast, but it has still some rough edges here and there. Compose is currently (as of version 1.4) +not considered production ready. That means you won't be able to scale to multiple servers and you won't be able to run +zero downtime deployments out of the box. Consider all this as experimental until you understand all the implications +to run docker (with compose) on production. + +**Run your app with docker-compose** + +Prerequisites: + +* docker (tested with 1.8) +* docker-compose (tested with 0.4) + +Before you start, check out the `docker-compose.yml` file in the root of this project. This is where each component +of this application gets its configuration from. It consists of a `postgres` service that runs the database, `redis` +for caching, `nginx` as reverse proxy and last but not least the `django` application run by gunicorn. +{% if cookiecutter.use_celery == 'y' -%} +Since this application also runs Celery, there are two more services with a service called `celeryworker` that runs the +celery worker process and `celerybeat` that runs the celery beat process. +{% endif %} + + +All of these servicese except `redis` rely on environment variables set by you. There is an `env.example` file in the +root directory of this project as a starting point. Add your own variables to the file and rename it to `.env`. This +file won't be tracked by git by default so you'll have to make sure to use some other mechanism to copy your secret if +you are relying solely on git. + + +By default, the application is configured to listen on all interfaces on port 80. If you want to change that, open the +`docker-compose.yml` file and replace `0.0.0.0` with your own ip. If you are using `nginx-proxy`_ to run multiple +application stacks on one host, remove the port setting entirely and add `VIRTUAL_HOST={{cookiecutter.domain_name}}` to your env file. +This pass all incoming requests on `nginx-proxy` to the nginx service your application is using. + +.. _nginx-proxy: https://github.com/jwilder/nginx-proxy + +Postgres is saving its database files to `/data/{{cookiecutter.repo_name}}/postgres` by default. Change that if you wan't +something else and make sure to make backups since this is not done automatically. + +To get started, pull your code from source control (don't forget the `.env` file) and change to your projects root +directory. + +You'll need to build the stack first. To do that, run:: + + docker-compose build + +Once this is ready, you can run it with:: + + docker-compose up + + +To run a migration, open up a second terminal and run:: + + docker-compose run django python manage.py migrate + +To create a superuser, run:: + + docker-compose run django python manage.py createsuperuser + + +If you need a shell, run:: + + docker-compose run django python manage.py shell_plus + + +Once you are ready with your initial setup, you wan't to make sure that your application is run by a process manager to +survive reboots and auto restarts in case of an error. You can use the process manager you are most familiar with. All +it needs to do is to run `docker-compose up` in your projects root directory. + +If you are using `supervisor`, you can use this file as a starting point:: + + [program:{{cookiecutter.repo_name}}] + command=docker-compose up + directory=/path/to/{{cookiecutter.repo_name}} + redirect_stderr=true + autostart=true + autorestart=true + priority=10 + + +Place it in `/etc/supervisor/conf.d/{{cookiecutter.repo_name}}.conf` and run:: + + supervisorctl reread + supervisorctl start {{cookiecutter.repo_name}} + + + +To get the status, run:: + + supervisorctl status + +If you have errors, you can always check your stack with `docker-compose`. Switch to your projects root directory and run:: + + docker-compose ps + + +to get an output of all running containers. + +To check your logs, run:: + + docker-compose logs + +If you want to scale your application, run:: + + docker-compose scale django=4 + docker-compose scale celeryworker=2 + + +**Don't run the scale command on postgres or celerybeat** \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/compose/django/entrypoint.sh b/{{cookiecutter.repo_name}}/compose/django/entrypoint.sh new file mode 100644 index 00000000..8c07a641 --- /dev/null +++ b/{{cookiecutter.repo_name}}/compose/django/entrypoint.sh @@ -0,0 +1,18 @@ +#!/bin/bash +set -e +# This entrypoint is used to play nicely with the current cookiecutter configuration. +# Since docker-compose relies heavily on environment variables itself for configuration, we'd have to define multiple +# environment variables just to support cookiecutter out of the box. That makes no sense, so this little entrypoint +# does all this for us. +export DJANGO_CACHE_URL=redis://redis:6379/0 + +# the official postgres image uses 'postgres' as default user if not set explictly. +if [ -z "$POSTGRES_ENV_POSTGRES_USER" ]; then + export POSTGRES_ENV_POSTGRES_USER=postgres +fi + +export DATABASE_URL=postgres://$POSTGRES_ENV_POSTGRES_USER:$POSTGRES_ENV_POSTGRES_PASSWORD@postgres:5432/$POSTGRES_ENV_POSTGRES_USER +{% if cookiecutter.use_celery == 'y' %} +export CELERY_BROKER_URL=$DJANGO_CACHE_URL +{% endif %} +exec "$@" \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/compose/django/gunicorn.sh b/{{cookiecutter.repo_name}}/compose/django/gunicorn.sh new file mode 100644 index 00000000..014f173e --- /dev/null +++ b/{{cookiecutter.repo_name}}/compose/django/gunicorn.sh @@ -0,0 +1,3 @@ +#!/bin/sh +python /app/manage.py collectstatic --noinput +/usr/local/bin/gunicorn config.wsgi -w 4 -b 0.0.0.0:5000 --chdir=/app \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/compose/nginx/Dockerfile b/{{cookiecutter.repo_name}}/compose/nginx/Dockerfile new file mode 100644 index 00000000..19639576 --- /dev/null +++ b/{{cookiecutter.repo_name}}/compose/nginx/Dockerfile @@ -0,0 +1,2 @@ +FROM nginx:latest +ADD nginx.conf /etc/nginx/nginx.conf \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/compose/nginx/nginx.conf b/{{cookiecutter.repo_name}}/compose/nginx/nginx.conf new file mode 100644 index 00000000..720b22e5 --- /dev/null +++ b/{{cookiecutter.repo_name}}/compose/nginx/nginx.conf @@ -0,0 +1,53 @@ +user nginx; +worker_processes 1; + +error_log /var/log/nginx/error.log warn; +pid /var/run/nginx.pid; + + +events { + worker_connections 1024; +} + + +http { + include /etc/nginx/mime.types; + default_type application/octet-stream; + + log_format main '$remote_addr - $remote_user [$time_local] "$request" ' + '$status $body_bytes_sent "$http_referer" ' + '"$http_user_agent" "$http_x_forwarded_for"'; + + access_log /var/log/nginx/access.log main; + + sendfile on; + #tcp_nopush on; + + keepalive_timeout 65; + + #gzip on; + + upstream app { + server django:5000; + } + + server { + listen 80; + charset utf-8; + + + location / { + # checks for static file, if not found proxy to app + try_files $uri @proxy_to_app; + } + + location @proxy_to_app { + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header Host $http_host; + proxy_redirect off; + + proxy_pass http://app; + } + + } +} \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/dev.yml b/{{cookiecutter.repo_name}}/dev.yml new file mode 100644 index 00000000..98146ffe --- /dev/null +++ b/{{cookiecutter.repo_name}}/dev.yml @@ -0,0 +1,16 @@ +postgres: + image: postgres + volumes: + # If you are using boot2docker, postgres data has to live in the VM for now until #581 is fixed + # for more info see here: https://github.com/boot2docker/boot2docker/issues/581 + - /data/{{cookiecutter.repo_name}}/postgres:/var/lib/postgresql/data + +django: + build: . + command: python /app/manage.py runserver_plus 0.0.0.0:8000 + volumes: + - .:/app + ports: + - "8000:8000" + links: + - postgres \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/docker-compose.yml b/{{cookiecutter.repo_name}}/docker-compose.yml new file mode 100644 index 00000000..01f5d68b --- /dev/null +++ b/{{cookiecutter.repo_name}}/docker-compose.yml @@ -0,0 +1,43 @@ +postgres: + image: postgres:9.4 + volumes: + - /data/{{cookiecutter.repo_name}}/postgres:/var/lib/postgresql/data + env_file: .env + +django: + build: . + user: django + links: + - postgres + - redis + command: /gunicorn.sh + env_file: .env + +nginx: + build: ./compose/nginx + links: + - django + ports: + - "0.0.0.0:80:80" + +redis: + image: redis:3.0 +{% if cookiecutter.use_celery == 'y' %} +celeryworker: + build: . + user: django + env_file: .env + links: + - postgres + - redis + command: celery -A {{cookiecutter.repo_name}}.taskapp worker -l INFO + +celerybeat: + build: . + user: django + env_file: .env + links: + - postgres + - redis + command: celery -A {{cookiecutter.repo_name}}.taskapp beat -l INFO +{% endif %} \ No newline at end of file diff --git a/{{cookiecutter.repo_name}}/env.example b/{{cookiecutter.repo_name}}/env.example new file mode 100644 index 00000000..b4375324 --- /dev/null +++ b/{{cookiecutter.repo_name}}/env.example @@ -0,0 +1,12 @@ +POSTGRES_PASSWORD=mysecretpass +POSTGRES_USER=postgresuser + +DJANGO_SETTINGS_MODULE=config.settings.production +DJANGO_SECRET_KEY= +DJANGO_AWS_ACCESS_KEY_ID= +DJANGO_AWS_SECRET_ACCESS_KEY= +DJANGO_AWS_STORAGE_BUCKET_NAME= +DJANGO_MAILGUN_API_KEY= +DJANGO_MAILGUN_SERVER_NAME= +DJANGO_SERVER_EMAIL= +DJANGO_SECURE_SSL_REDIRECT=False \ No newline at end of file