Commit Graph

204 Commits

Author SHA1 Message Date
Daniele Varrazzo
be8e1a2632 Making my linter happy 2019-06-07 18:20:36 +01:00
Changaco
842e383c0c fix NamedTupleCursor._cached_make_nt 2019-06-04 14:30:30 +02:00
Alexander Kukushkin
6cff5a3e08 Smart replication feedback
This commit makes psycopg2 responsible for sending the status update
(feedback) messages to the server regardless of whether a synchronous or
asynchronous connection is used.

Feedback is sent every *status_update* (default value is 10) seconds,
which could be configured by passing a corresponding parameter to the
`start_replication()` or `start_replication_expert()` methods.
The actual feedback message is sent by the
`pq_read_replication_message()` when the *status_update* timeout is
reached.

The default behavior of the `send_feedback()` method is changed.
It doesn't send a feedback message on every call anymore but just
updates internal structures. There is still a way to *force* sending
a message if *force* or *reply* parameters are set.

The new approach has certain advantages:
1. The client can simply call the `send_feedback()` for every
   processed message and the library will take care of not overwhelming
   the server. Actually, in the synchronous mode it is even mandatory
   to confirm every processed message.
2. The library tracks internally the pointer of the last received
   message which is not keepalive. If the client confirmed the last
   message and after that server sends only keepalives with increasing
   *wal_end*, the library can safely move forward *flush* position to
   the *wal_end* and later automatically report it to the server.

Reporting of the *wal_end* received from keepalive messages is very
important. Not doing so casing:
1. Excessive disk usage, because the replication slot prevents from
   WAL being cleaned up.
2. The smart and fast shutdown of the server could last indefinitely
   because walsender waits until the client report *flush* position
   equal to the *wal_end*.

This implementation is only extending the existing API and therefore
should not break any of the existing code.
2019-05-06 10:27:44 +02:00
Daniele Varrazzo
4058f363d6 More robust RealDictRow population
In the presence of repeated columns the mapping would have remained into
into the dictionary. Now it is removed.

Fix #884
2019-04-06 21:17:28 +01:00
Daniele Varrazzo
cc815e8e8d RealDictRow inherits from OrderedDict
Now its state is unmodified, so apart from special-casing creation
and initial population can work unmodified, and all the desired
properties just work (modifiability, picklability...)

Close #886.
2019-04-06 19:43:31 +01:00
Daniele Varrazzo
21d16b6f67 Fixed RealDictCursor.pop()
Addresses #886, but there might be something else broken there.
2019-04-06 19:43:09 +01:00
Daniele Varrazzo
dfb301b42b Use PY2, PY3 for conditional code instead of sys.version_info 2019-03-16 19:54:40 +00:00
Jon Dufresne
3f890f8bbe Use True/False instead of 1/0 to represent bool values
Slightly more modern, readable, and Pythonic.
2019-03-13 11:13:36 +00:00
Jon Dufresne
03bb44dd2c Convert while 1: statements to while True:
A slightly more readable and modern syntax.
2019-03-13 11:13:05 +00:00
Daniele Varrazzo
599432552a Copyright year updated
ag -l Copyright | xargs sed -i \
        "s/\(.*copyright (C) [0-9]\+\)\(-[0-9]\+\)\?\(.*\)/\1-$(date +%Y)\3/I"
2019-02-17 01:36:36 +00:00
Daniele Varrazzo
35ec7ad9c1 Use a proper LRU cache for namedtuples
Previous one didn't refresh by last use. Use the stdlib version for py3
and one of our own for py2.

Max size set to 512, which should be fine for everyone (tweaking is
still possible by monkeypatching, as the tests do, but I don't want to
make an interface of it).
2019-02-02 19:29:20 +00:00
Daniele Varrazzo
805527fcd6 Added caching of types generated by NamedTupleCursor
see #838
2019-02-02 14:22:25 +00:00
Daniele Varrazzo
bde9fc6dea Docs wordsmith for execute_values() fetch param 2018-12-27 14:42:58 +01:00
Daniele Varrazzo
6fdac46137 Some harmless adjustments in execute_values() code
No optional result, and create the list only if requred.
2018-12-27 14:39:13 +01:00
Ivan Kotelnikov
b40ac15efc rename param name to fetch 2018-11-24 20:51:18 +02:00
Ivan Kotelnikov
9a8d7d5a6a add RETURNING clause support in execute_values function 2018-11-21 16:38:13 +02:00
Daniele Varrazzo
8fb0f694f7 Don't barf on Composite passed to execute_values()
Close #794
2018-10-23 01:57:48 +01:00
Daniele Varrazzo
05f9e231a0 Full flake8 3.5 cleanup 2018-10-23 00:39:14 +01:00
Daniele Varrazzo
44bd2927c5 Use the connection.info properties instead of the legacy methods 2018-10-13 03:28:42 +01:00
Jon Dufresne
b07e34e0b8 Prefer https:// URLs when available 2018-09-22 19:02:33 -07:00
Daniel Hahler
6af55ee51c Improve doc for extras.wait_select 2018-09-05 16:45:33 +01:00
Daniele Varrazzo
0bb7d0db48 DictCursor and RealDictCursor rows maintain columns order
Close #177.
2018-05-21 03:14:08 +01:00
Daniele Varrazzo
7bdaf0affd Restored methods iter*() on dict cursors rows 2018-05-21 02:51:37 +01:00
Daniele Varrazzo
e8a831dda2 Merge branch 'master' into drop-2to3 2018-05-20 23:56:29 +01:00
Daniele Varrazzo
bc84b6233e Allow non-ascii chars in namedtuple fields
They can be valid chars in Python 3. Or maybe not? In which case Python
will throw an exception, but that's fine.

Fix regression introduced fixing #211
2018-05-18 12:15:50 +01:00
Daniele Varrazzo
ddb87b7727 Convert fields names into valid Python identifiers in NamedTupleCursor
Close #211.
2018-01-29 03:30:46 +00:00
Jon Dufresne
8ad2098b74 Drop 2to3 build step; make all code compatible with all Pythons
Make all library code compatible with both Python 2 and Python 3. Helps
move to modern Python idioms. Can now write for Python 3 (with
workarounds for Python 2) instead of the other way around.

In the future, when it is eventually time to drop Python 2, the library
will be in a better position to remove workarounds

Added a very small comparability module compat.py where required. It
includes definitions for:

- text_type -- A type. str on Python 3. unicode on Python 2.
- string_types -- A tuple. Contains only str on Python 3. Contains str &
                  unicode on Python 2.
2017-12-11 20:26:58 -08:00
Jon Dufresne
f35465231f Drop the Python 2 style interface from DictRow
Now standardizes on the Python 3 interface for all uses. Makes behavior
of DictRow between Pythons more consistent and predictable.
2017-12-11 20:02:16 -08:00
Jon Dufresne
3a6a8e96fb User super() throughout DictRow class
Avoid calling parent method directly.
2017-12-11 19:01:38 -08:00
Jon Dufresne
19b1efd629 Drop long deprecated function register_tstz_w_secs()
Deprecated in commit b263fbf274 on
2010-01-13. The deprecation warning was first released in version 2.2.2.

The function used to register an alternate type caster for TIMESTAMP
WITH TIME ZONE to deal with historical time zones with seconds in the
UTC offset. These are now correctly handled by the default type caster,
so currently the function doesn't do anything.
2017-12-10 07:57:11 -08:00
Jon Dufresne
f5703dc3e5 Use builtin function next() throughout project
Available since Python 2.6. Use of .next() is deprecated and not
supported in Python 3. Forward compatible with modern Python.

https://docs.python.org/2/library/functions.html#next
2017-12-01 19:57:37 -08:00
Hugo
283de27098 Remove redundant parentheses 2017-11-28 16:06:52 +00:00
Hugo
08b479bc10 __slots__ should be a tuple 2017-11-28 16:05:59 +00:00
Daniele Varrazzo
914ccbacb5 Merge branch 'python3_mintimeloggingconnection' 2017-11-28 03:23:42 +00:00
Jon Dufresne
05c28cce78 Remove workarounds for namedtuple on Python <= 2.5
namedtuple is available on all Python versions supported by psycopg2. It
was first introduced in Python 2.6. Can remove all workarounds and
special documentation.
2017-11-26 13:55:30 -08:00
Jon Dufresne
390e43fcb1 Use modern except syntax throughout project
The syntax "except Exception, exc:" is deprecated. All Python versions
supported by psycopg2 support the newer, modern syntax. Forward
compatible with future Python versions.
2017-11-20 20:00:35 -08:00
Kevin Campbell
b804c092ac Patch for issue #609 for MinTimeLoggingConnection
On Python3 MinTimeLoggingConnection raises an exception as it tries to
mix strings and bytes
2017-11-03 17:30:06 +08:00
Daniele Varrazzo
3ec06b88f0 Construct proper human sentences to describe execute_values()
With bonus typos dropped.
2017-08-31 16:34:04 +01:00
Daniele Varrazzo
3b48918bef Note that the fast executemany functions don't respect rowcount
See issue #540
2017-03-28 10:37:04 +01:00
Daniele Varrazzo
7210287e76 Stop a docstring generating a warning
Valid reST, slightly less valid Python.
2017-02-07 12:29:05 +00:00
Daniele Varrazzo
95226baa9b Further minimal performance tweaks to execute_values 2017-02-02 17:44:25 +00:00
Daniele Varrazzo
dc1b4fff90 Avoid an useless encode/decode roundtrip in execute_values()
Tests moved into a separate module.
2017-02-02 17:29:17 +00:00
Daniele Varrazzo
d2fdc5ca9f Better docs for fast executemany functions.
Issue #502.
2017-02-02 16:02:33 +00:00
Daniele Varrazzo
9bac37baf7 Fixed execute_values with unicode
Also added unicode tests.
2017-02-01 02:47:59 +00:00
Daniele Varrazzo
2e2dcd536b Fixed fast execute functions with Python 3 2017-02-01 02:36:54 +00:00
Daniele Varrazzo
a95fd3df1a Added execute_batch and execute_values functions 2017-02-01 01:59:47 +00:00
NotSqrt
9ffb61214c Fix DeprecationWarning: generator '__iter__' raised StopIteration
Closes #498
2017-01-04 09:45:53 +01:00
Daniele Varrazzo
21f38a4c07 Don't try to write bytes in the LoggingConnection file
Close #483
2016-12-29 22:42:25 +01:00
Tim Graham
35b4a01b6d Fix "invalid escape sequence" warning in Python 3.6
http://bugs.python.org/issue27364
2016-12-25 20:43:29 +01:00
Daniele Varrazzo
643ba70bad Added ipaddress objects conversion
Close #387
2016-10-11 04:54:41 +01:00