Merge pull request #20 from The-Daishogun/master

ADD some info to docs
This commit is contained in:
M1ha Shvn 2021-03-15 12:45:04 +05:00 committed by GitHub
commit 4f7289ce9a
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

View File

@ -8,10 +8,11 @@ At the begging I expect, that you already have:
## Configuration ## Configuration
Add required parameters to [Django settings.py](https://docs.djangoproject.com/en/3.0/topics/settings/): Add required parameters to [Django settings.py](https://docs.djangoproject.com/en/3.0/topics/settings/):
1. [CLICKHOUSE_DATABASES](configuration.md#clickhouse_databases) 1. Add `'django_clickhouse'` to `INSTALLED_APPS`
2. [Intermediate storage](storages.md) configuration. For instance, [RedisStorage](storages.md#redisstorage) 2. [CLICKHOUSE_DATABASES](configuration.md#clickhouse_databases)
3. It's recommended to change [CLICKHOUSE_CELERY_QUEUE](configuration.md#clickhouse_celery_queue) 3. [Intermediate storage](storages.md) configuration. For instance, [RedisStorage](storages.md#redisstorage)
4. Add sync task to [celerybeat schedule](http://docs.celeryproject.org/en/v2.3.3/userguide/periodic-tasks.html). 4. It's recommended to change [CLICKHOUSE_CELERY_QUEUE](configuration.md#clickhouse_celery_queue)
5. Add sync task to [celerybeat schedule](http://docs.celeryproject.org/en/v2.3.3/userguide/periodic-tasks.html).
Note, that executing planner every 2 seconds doesn't mean sync is executed every 2 seconds. Note, that executing planner every 2 seconds doesn't mean sync is executed every 2 seconds.
Sync time depends on model sync_delay attribute value and [CLICKHOUSE_SYNC_DELAY](configuration.md#clickhouse_sync_delay) configuration parameter. Sync time depends on model sync_delay attribute value and [CLICKHOUSE_SYNC_DELAY](configuration.md#clickhouse_sync_delay) configuration parameter.
You can read more in [sync section](synchronization.md). You can read more in [sync section](synchronization.md).
@ -19,35 +20,73 @@ Add required parameters to [Django settings.py](https://docs.djangoproject.com/e
You can also change other [configuration parameters](configuration.md) depending on your project. You can also change other [configuration parameters](configuration.md) depending on your project.
#### Example #### Example
```python * if you already have a celery workflow:
# django-clickhouse library setup ```python
CLICKHOUSE_DATABASES = { # django-clickhouse library setup
CLICKHOUSE_DATABASES = {
# Connection name to refer in using(...) method # Connection name to refer in using(...) method
'default': { 'default': {
'db_name': 'test', 'db_name': 'test',
'username': 'default', 'username': 'default',
'password': '' 'password': ''
} }
} }
CLICKHOUSE_REDIS_CONFIG = { CLICKHOUSE_REDIS_CONFIG = {
'host': '127.0.0.1', 'host': '127.0.0.1',
'port': 6379, 'port': 6379,
'db': 8, 'db': 8,
'socket_timeout': 10 'socket_timeout': 10
} }
CLICKHOUSE_CELERY_QUEUE = 'clickhouse' CLICKHOUSE_CELERY_QUEUE = 'clickhouse'
# If you have no any celerybeat tasks, define a new dictionary # If you have no any celerybeat tasks, define a new dictionary
# More info: http://docs.celeryproject.org/en/v2.3.3/userguide/periodic-tasks.html # More info: http://docs.celeryproject.org/en/v2.3.3/userguide/periodic-tasks.html
from datetime import timedelta from datetime import timedelta
CELERYBEAT_SCHEDULE = { CELERYBEAT_SCHEDULE = {
'clickhouse_auto_sync': { 'clickhouse_auto_sync': {
'task': 'django_clickhouse.tasks.clickhouse_auto_sync', 'task': 'django_clickhouse.tasks.clickhouse_auto_sync',
'schedule': timedelta(seconds=2), # Every 2 seconds 'schedule': timedelta(seconds=2), # Every 2 seconds
'options': {'expires': 1, 'queue': CLICKHOUSE_CELERY_QUEUE} 'options': {'expires': 1, 'queue': CLICKHOUSE_CELERY_QUEUE}
} }
} }
``` ```
* if you don't have a celery workflow:
create a `celery.py` file in `mysite/mysite/celery.py`:
```python
import os
from datetime import timedelta
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "mysite.settings")
app = Celery("podafarini")
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object("django.conf:settings", namespace="CELERY")
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
app.conf.beat_schedule = {
"clickhouse_auto_sync": {
"task": "django_clickhouse.tasks.clickhouse_auto_sync",
"schedule": timedelta(seconds=2), # Every 2 seconds
"options": {"expires": 1, "queue": "clickhouse"},
}
}
```
`mysite/mysite/__init__.py`:
```python
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery import app as celery_app
__all__ = ("celery_app",)
```
## Adopting django model ## Adopting django model
Read [ClickHouseSyncModel](models.md#djangomodel) section. Read [ClickHouseSyncModel](models.md#djangomodel) section.
@ -76,7 +115,8 @@ from my_app.models import User
class ClickHouseUser(ClickHouseModel): class ClickHouseUser(ClickHouseModel):
django_model = User django_model = User
# Uncomment the line below if you want your models to sync between databases
# need_sync = True
id = fields.UInt32Field() id = fields.UInt32Field()
first_name = fields.StringField() first_name = fields.StringField()
birthday = fields.DateField() birthday = fields.DateField()
@ -91,11 +131,11 @@ class ClickHouseUser(ClickHouseModel):
3. Create `0001_initial.py` file inside the created package. Result structure should be: 3. Create `0001_initial.py` file inside the created package. Result structure should be:
``` ```
my_app my_app
>> clickhouse_migrations | clickhouse_migrations
>>>> __init__.py |-- __init__.py
>>>> 0001_initial.py |-- 0001_initial.py
>> clickhouse_models.py | clickhouse_models.py
>> models.py | models.py
``` ```
4. Add content to file `0001_initial.py`: 4. Add content to file `0001_initial.py`: