Infinidat/infi.clickhouse_orm
1
1
Fork 0
mirror of https://github.com/Infinidat/infi.clickhouse_orm.git synced 2024-11-28 19:53:44 +03:00
Code Issues Packages Projects Releases Wiki Activity
458 Commits 36 Branches 42 Tags 1.4 MiB
4f0624d35c
Commit Graph

230 Commits

Author SHA1 Message Date
Понаревский Владимир
793726adb2 Fix after review 2018-05-08 16:41:50 +03:00
Понаревский Владимир
a5c93dc176 Add disabler for the timezone getting 2018-05-08 13:19:45 +03:00
M1ha
eb15dd65ec Added custom partitioning to all MergeTree family engines 2018-04-24 15:19:05 +05:00
Itai Shirav
20e609f952 Update docs 2018-04-21 15:23:00 +03:00
Itai Shirav
7cc5c50240 Merge branch 'issue-66' of https://github.com/carrotquest/infi.clickhouse_orm into carrotquest-issue-66 2018-04-21 13:50:09 +03:00
Itai Shirav
66eda2214c Make tests pass 2018-04-21 13:27:42 +03:00
Itai Shirav
b6229125a3 Merge branch 'custom_partitioning' of https://github.com/carrotquest/infi.clickhouse_orm into carrotquest-custom_partitioning 
# Conflicts:
#	src/infi/clickhouse_orm/engines.py
#	src/infi/clickhouse_orm/models.py
#	tests/test_database.py
#	tests/test_engines.py
 2018-04-21 13:04:19 +03:00
Itai Shirav
6fa6786a9c Update docs 2018-04-21 12:10:30 +03:00
Ivan Ladelshchikov
a5f2fa4d76 add Distributed engine 2018-04-21 11:48:32 +03:00
Ivan Ladelshchikov
3268019216 _fields and _writable_fields are OrderedDicts 2018-04-21 11:48:32 +03:00
Ivan Ladelshchikov
57112f9de6 add ServerError exception 2018-04-21 11:48:32 +03:00
M1ha
0927136ffd 1) Added a test on https://github.com/Infinidat/infi.clickhouse_orm/issues/66 
2) Fixed issue
 2018-04-20 12:38:36 +05:00
M1ha
7fb0589692 1. Added support of custom partitioning (https://clickhouse.yandex/docs/en/table_engines/custom_partitioning_key/) 
2. Added attribute server_version to Database class
3. Changed Engine.create_table_sql(), Engine.drop_table_sql(), Model.create_table_sql(), Model.drop_table_sql()  parameter to db from db_name
 2018-04-12 14:21:46 +05:00
Itai Shirav
cd8d82c226 Update docs 2018-04-07 15:20:33 +03:00
desile
c98edc4f77 Added doc for between operator 2017-12-03 21:54:58 +03:00
pv.larkin
d553aaf9eb Add between operator 2017-11-29 14:52:52 +03:00
pv.larkin
0342dc863b Add OR and AND operations for Q objects 2017-11-29 14:27:54 +03:00
Itai Shirav
fa3f96e464 Merge branch 'migrate_materialized_fields' of https://github.com/carrotquest/infi.clickhouse_orm into carrotquest-migrate_materialized_fields 
# Conflicts:
#	tests/sample_migrations/0012.py
#	tests/test_migrations.py (reverted from commit 288db6a28f56b9ae5a4fa1c0ead111f679886488)
 2017-10-30 15:49:25 +02:00
Itai Shirav
b758333f42 Merge branch 'dollar_issue_fix' of https://github.com/carrotquest/infi.clickhouse_orm into carrotquest-dollar_issue_fix 2017-10-30 15:30:31 +02:00
Itai Shirav
b93c145cde Merge branch 'tsionyx-develop' into develop 2017-10-30 15:28:37 +02:00
Itai Shirav
314090fa56 Merge branch 'raw_migrations' of https://github.com/carrotquest/infi.clickhouse_orm into carrotquest-raw_migrations 2017-10-30 15:09:23 +02:00
Itai Shirav
dcbf371077 Merge branch 'develop' of https://github.com/anci/infi.clickhouse_orm into anci-develop 2017-10-30 15:07:00 +02:00
Itai Shirav
48c08b869e Merge branch 'carrotquest-develop' into develop 2017-10-30 15:00:17 +02:00
Ivan Ladelshchikov
50e63fb1f8 show field name in error when invalid value assigned 2017-10-10 15:30:37 +04:00
M1ha
5ea20a11a9 Added tests and resolved https://github.com/Infinidat/infi.clickhouse_orm/issues/47 2017-10-10 12:24:17 +05:00
M1ha
b952f93e78 Merge branch 'develop' of https://github.com/Infinidat/infi.clickhouse_orm into migrate_materialized_fields 2017-10-10 11:24:35 +05:00
M1ha
911af41ec1 Fixed https://github.com/Infinidat/infi.clickhouse_orm/issues/56 issue 2017-10-04 17:53:23 +05:00
Ivan Ladelshchikov
38472012db allow ISO 8601 compliant values in DateTimeField 2017-10-03 13:26:00 +04:00
M1ha
51a814732b Merge branch 'develop' of https://github.com/Infinidat/infi.clickhouse_orm into raw_migrations 
# Conflicts:
#	docs/schema_migrations.md
#	tests/sample_migrations/0010.py
#	tests/sample_migrations/0011.py
 2017-10-02 15:49:59 +05:00
M1ha
b28e11c69d Added RunPython and RunSQL migration types 2017-09-20 10:08:07 +05:00
anci
322b4a0741 add field names list explicitly to Database.insert method 2017-09-18 21:50:40 +03:00
M1ha
0391482ec7 Fixed 2 bugs in Merge engine: 
1) If database name contained some signes (+, -) create merge table failed.
2) When merge table was created and set_database was not previously called, "currentDatabase()" was also used, which could lead to errors on multiple databases.
 2017-09-18 15:53:15 +05:00
Itai Shirav
8304ddca5c Update docs 2017-09-13 12:17:26 +03:00
Itai Shirav
59564f8c70 Add distinct method to querysets 2017-09-13 12:15:48 +03:00
Itai Shirav
7bbcae574a Add AlterTableWithBuffer migration operation 2017-09-13 12:15:48 +03:00
M1ha
430872b958 Added readonly parameter to all Field subclasses 2017-09-11 10:17:23 +05:00
M1ha
b7cf611595 Fixed error text 2017-09-08 09:15:30 +05:00
M1ha
de9f64cd3a Added Merge engine 
1) Divided readonly and system flags of Field model. Readonly flag only restricts insert operations, while system flag restricts also create and drop table operations

2) Added Merge engine and tests for it
3) Added docs for Merge engine
4) Added opportunity to make Field readonly. This is useful for "virtual" columns (https://clickhouse.yandex/docs/en/single/index.html#virtual-columns)
 2017-09-07 17:44:27 +05:00
M1ha Shvn
0829936940 Merge branch 'develop' into develop 2017-08-22 13:33:01 +05:00
M1ha
508c959d2c Fixed bug with getting SystemPart info on actual ClickHouse version (1.1.54245) 2017-08-22 13:27:40 +05:00
Marsel
88b1cea98c Fix TypeError in Py3 2017-08-20 10:37:11 +03:00
Marsel
c4e60ca2a8 Include unicode_literals 2017-08-20 10:37:11 +03:00
Marsel
70d34b097f Fix python3 compatibility 2017-08-20 10:37:11 +03:00
Itai Shirav
67e0442645 - Nullable arrays not supported in latest ClickHouse version 
- system.parts table no longer includes "replicated" column in latest ClickHouse version
 2017-08-20 09:30:40 +03:00
Itai Shirav
639867bb32 - Added QuerySet.paginate() 
- Support for basic aggregation in querysets
 2017-08-14 12:14:52 +03:00
Itai Shirav
cb6c329d32 Migrations: when creating a table for a BufferModel, create the underlying table too if necessary 2017-06-26 11:09:57 +03:00
Itai Shirav
1966896850 Support queryset slicing 2017-06-24 12:28:42 +03:00
Itai Shirav
2c0ef08ee1 Minor refactoring of extra_null_values 2017-06-23 11:10:49 +03:00
Dzianis Sivets
9b7e7a179e NullableField of ArrayField 2017-06-16 20:38:01 +03:00
Dzianis Sivets
1b0ea035b8 Nullable fields support 2017-06-16 20:37:37 +03:00
Itai Shirav
d02d6b14eb - Added ne and not_in queryset operators 
- Querysets no longer have a default order when `order_by` is not called
- Added `autocreate` flag to database initializer
- Fix for SELECT FROM JOIN (#37)
 2017-06-15 12:19:17 +03:00
Ivan Ladelshchikov
fcb8196d3d fix unicode params for Py2 2017-06-06 20:16:56 +05:00
Itai Shirav
6301ab468e fix queryset problem with non-ascii chars 2017-05-05 15:39:01 +03:00
Itai Shirav
00ca503b7c Generate a class reference document 2017-05-05 15:31:08 +03:00
Itai Shirav
3f8434fe50 rename QuerySet.query to as_sql 2017-05-05 15:21:55 +03:00
Itai Shirav
4625a7e00f Generate a class reference document 2017-05-03 08:36:47 +03:00
Itai Shirav
be474b3aed Add simple engines: TinyLog, Log, Memory 2017-04-28 18:36:40 +03:00
Itai Shirav
abbe334875 queryset tests 2017-04-26 15:46:34 +03:00
Itai Shirav
dbea017d60 Add support for FixedString fields 2017-04-25 16:03:52 +03:00
Itai Shirav
2b8c0b6c38 Do not send readonly=1 when connection is already in readonly mode (this causes a database error) 2017-04-25 08:39:24 +03:00
Itai Shirav
3487f3b241 initial work on query builder 2017-04-24 13:13:31 +03:00
M1ha
533e6e187a Fixed "TypeError: a bytes-like object is required, not 'str'" on \n insertions 2017-04-06 11:12:59 +05:00
M1ha
76718426bf Fixed bug with StringIO import, incompatible with python 3 2017-04-06 10:54:26 +05:00
Itai Shirav
dacf88adbf Performance improvement: build a _writable_fields list once instead of calculating it every time it's needed 2017-04-05 18:20:01 +03:00
Itai Shirav
c5a9b16eac Performance improvement: skip utils.escape for numeric fields 2017-04-05 18:19:12 +03:00
Itai Shirav
21907966b2 Reduce memory footprint of Database.insert() 2017-04-05 17:42:42 +03:00
Itai Shirav
ec99044fab Greatly improve performance when inserting large strings (credit to M1hacka for identifying the problem) 2017-04-05 17:09:56 +03:00
Itai Shirav
04340884be Add tests for engine types 2017-04-04 15:21:08 +03:00
Itai Shirav
526b6d53b9 Merge branch 'develop' of https://github.com/leenr/infi.clickhouse_orm into leenr-develop 2017-04-04 14:49:13 +03:00
Itai Shirav
e1ef96f05b Add a test for "with totals" 2017-04-04 14:41:23 +03:00
Itai Shirav
b27756d44b Merge branch 'with-totals-fix' of https://github.com/pilosus/infi.clickhouse_orm into pilosus-with-totals-fix 2017-04-04 12:47:18 +03:00
Itai Shirav
015a4512e7 Fix datetime tests 2017-04-04 12:41:18 +03:00
nikepan
b2a5482a65 blank datetime fix beauty 2017-03-27 17:07:10 +03:00
nikepan
e5222ffcb0 blank datetime fix improve 2017-03-27 17:03:13 +03:00
nikepan
9baa863d1f blank datetime better fix 2017-03-27 16:51:34 +03:00
nikepan
1460d4c85e blank datetime fix 2017-03-27 16:30:38 +03:00
Vitaly Samigullin
84681ec979 GROUP BY WITH TOTALS workaround to skip blank line separator making parsing 2017-03-12 16:49:54 +03:00
leenr
b4a1d14a12 Make ver_col in ReplacingMergeTree optional 2017-03-04 15:43:38 +03:00
leenr
1b8c4dbec1 Add support for ReplacingMergeTree (https://clickhouse.yandex/reference_en.html#ReplacingMergeTree) 2017-03-04 15:29:19 +03:00
Itai Shirav
1d573ded96 Merge branch 'develop' of https://github.com/emakarov/infi.clickhouse_orm into emakarov-develop 
# Conflicts:
#	src/infi/clickhouse_orm/models.py
#	tests/test_database.py
 2017-02-19 09:12:00 +02:00
M1ha
6f975a801c 1) Added get_database method to Model 
2) Added some assertions in tests for adding _database attribute in selects and inserts
3) database.insert() method sets _database
 2017-02-19 09:04:32 +02:00
M1ha
f3e75cfae3 1) Removed database params for working with SystemPart operations 
2) Added _database attribute to each model, got through select
 2017-02-19 09:04:32 +02:00
M1ha
14f2ab78b5 Fixed: 
1) Added partition working to readme
2) replaced insertable_only parameter with include_readonly
3) Added empty string alias and materialized field control
 2017-02-19 09:04:32 +02:00
M1ha
27217102da Fixed bugs, connected to merge 2017-02-19 09:04:05 +02:00
M1ha
5f2195f87f 1) Replaced is_insertable() field mehtod with readonly property (unification with model and tables) 
2) Method SystemPart.all() was replaced with get()
3) Added conditions parameter to SystemPart.get() and SystemPart.get_active() methods.
 2017-02-19 09:02:56 +02:00
M1ha
58b7a9aeac 1) Added to_dict model method 
2) Fixed bug in test_freeze cleaning, if backups don't contain all directory names (e. g. 1, 2, 3, 6, 7 - count=5, created_backup=8, not 6)
 2017-02-19 09:01:33 +02:00
M1ha
db61efd4cc 1) Added readonly models 
2) Added SystemPart models in order to execute partition operations
 2017-02-19 09:01:33 +02:00
emakarov
77b33c0ed4 test added. engine sql query fix to support special characters 2017-02-08 23:21:48 +03:00
Itai Shirav
dfd4d09e70 DateField.to_python() - handle timezone-aware datetime values correctly 2017-02-08 15:19:39 +02:00
emakarov
86a3fec143 cleaner code for Buffer engine and Buffer Model class 2017-02-07 21:32:51 +03:00
emakarov
e2e02ab029 Conflict resolved 2017-02-07 21:19:23 +03:00
emakarov
d19787cb9f Fix for create_table_sql for Buffer table 2017-02-07 20:41:26 +03:00
emakarov
c9697de56c Buffer engine initial commit 2017-02-07 19:19:50 +03:00
Itai Shirav
5b03e66048 Send readonly=1 when database is created in readonly mode 2017-02-07 16:36:41 +02:00
Itai Shirav
f29d737f29 Always keep datetime fields in UTC internally, and convert server timezone to UTC when parsing query results. 2017-02-07 15:25:16 +02:00
Itai Shirav
6c4640bb24 TRIVIAL code style 2017-02-07 12:20:47 +02:00
M1ha
2509b5b2e3 Rewritten Alias and Materialized fields to field parameters like default. 2017-01-27 10:46:12 +05:00
M1ha
ca341ea997 Added MaterializedField and AliasField 2017-01-26 15:43:41 +05:00
Itai Shirav
ce8c77684e Allow passing -1 as the page number (returning the last page) 2017-01-25 15:10:14 +02:00
Itai Shirav
9f2619a807 Accept datetime values for date fields (by Zloool) 2017-01-25 14:48:07 +02:00
Itai Shirav
cce85beeef Move database creation into a separate method 2017-01-25 14:40:03 +02:00
tswr
c492d419cc Fixing Database class to allow readonly mode 2016-11-02 14:28:00 +03:00
Itai Shirav
b2cf8f4245 Fixes by tsionyx 2016-09-29 11:24:22 +03:00
Itai Shirav
685e3dffe9 Bug fix - parse_array fails on int arrays 2016-09-15 12:03:41 +03:00
Itai Shirav
66f8e8a4ae Performance improvements when inserting 
escape - check first if there are any special chars before replacing
send lines in batches
Use list comprehension in to_tsv
 2016-09-15 11:32:07 +03:00
Itai Shirav
13bd956fc6 Add support for array fields 2016-09-01 15:25:48 +03:00
Itai Shirav
8fc3a31d4b Add support for enum fields 2016-08-31 15:43:06 +03:00
Itai Shirav
071bcd283d Add Python 3 support 2016-08-01 10:28:10 +03:00
Ivan Ladelshchikov
e7d0668418 use self variables to fix undefined behaviour 2016-07-28 07:43:10 +03:00
Itai Shirav
ffa99f3f10 cache ad hoc models 2016-07-11 17:11:56 +03:00
Itai Shirav
f29b3ea696 Add $table and $db placeholders 2016-07-11 16:17:49 +03:00
Itai Shirav
52e9d30c5c stream only SELECTs, to prevent "connection reset by peer" errors 2016-07-10 16:40:47 +03:00
Itai Shirav
87ee685c8b migrations support 2016-07-05 16:20:18 +03:00
Itai Shirav
b4f4af8c75 support pagination 2016-07-03 15:39:08 +03:00
Itai Shirav
a3bd212133 Simplify conversion of int and float fields 2016-06-30 16:17:53 +03:00
Itai Shirav
4efdcda97b Fix bug in date parsing 2016-06-30 16:12:02 +03:00
Itai Shirav
9489115d6e Fix bug in datetime parsing 2016-06-30 12:11:47 +03:00
Itai Shirav
c6c9f13e51 Support dashes and other special characters in database names and table names 2016-06-30 11:36:54 +03:00
Itai Shirav
6e786d75e9 Support model class inheritance 2016-06-29 14:52:55 +03:00
Itai Shirav
5b033a1e1b fix int ranges 2016-06-26 17:41:17 +03:00
Itai Shirav
64f8cde1c0 documentation 2016-06-26 16:52:25 +03:00
Itai Shirav
92ea9d413e support ad-hoc models 2016-06-26 15:11:16 +03:00
Itai Shirav
9262f0eae6 add database.select and field range checks 2016-06-23 19:05:44 +03:00
Itai Shirav
92e8b8259f better validation of keyword arguments in model constructor 2016-06-23 18:34:52 +03:00
Itai Shirav
b08f1b3688 model field conversion on assignment 2016-06-23 18:24:20 +03:00
Itai Shirav
25e85adc0d add database.insert and database.count 2016-06-23 17:35:17 +03:00
Itai Shirav
571ea8b6e5 rename project 2016-06-23 15:46:11 +03:00
Itai Shirav
b1174f09ea Initial commit of database class 2016-06-23 15:10:49 +03:00
Itai Shirav
4da45b0be5 Getting started 2016-06-23 14:11:20 +03:00
Itai Shirav
2f99d58a61 added all project files 2016-06-23 14:05:07 +03:00