This commit makes psycopg2 responsible for sending the status update
(feedback) messages to the server regardless of whether a synchronous or
asynchronous connection is used.
Feedback is sent every *status_update* (default value is 10) seconds,
which could be configured by passing a corresponding parameter to the
`start_replication()` or `start_replication_expert()` methods.
The actual feedback message is sent by the
`pq_read_replication_message()` when the *status_update* timeout is
reached.
The default behavior of the `send_feedback()` method is changed.
It doesn't send a feedback message on every call anymore but just
updates internal structures. There is still a way to *force* sending
a message if *force* or *reply* parameters are set.
The new approach has certain advantages:
1. The client can simply call the `send_feedback()` for every
processed message and the library will take care of not overwhelming
the server. Actually, in the synchronous mode it is even mandatory
to confirm every processed message.
2. The library tracks internally the pointer of the last received
message which is not keepalive. If the client confirmed the last
message and after that server sends only keepalives with increasing
*wal_end*, the library can safely move forward *flush* position to
the *wal_end* and later automatically report it to the server.
Reporting of the *wal_end* received from keepalive messages is very
important. Not doing so casing:
1. Excessive disk usage, because the replication slot prevents from
WAL being cleaned up.
2. The smart and fast shutdown of the server could last indefinitely
because walsender waits until the client report *flush* position
equal to the *wal_end*.
This implementation is only extending the existing API and therefore
should not break any of the existing code.
Now its state is unmodified, so apart from special-casing creation
and initial population can work unmodified, and all the desired
properties just work (modifiability, picklability...)
Close#886.
I don't know why it returns 0 instead of the right value. At least it
doesn't segfault, so don't skip the test altogether.
The test is unrelated to this branch: will cherry-pick elsewhere (if I
remember it...)
It won't work on windows if it's in the script: failing with errors
such as:
AttributeError: 'module' object has no attribute 'process'
or:
Can't get attribute 'process' on <module '__main__' (built-in)>
The new function keeps together PQconsumeInput() with PQisBusy(), in
order to handle the condition in which not all the results of a sequence
of statements arrive in the same roundtrip.
Added pointer to a PGresult to the connection to keep the state across
async communication: it can probably be used to simplify other code
paths where a result is brought forward manually.
Close#802Close#855Close#856
Allows removing many duplicate imports and better follows PEP8
guidelines:
https://www.python.org/dev/peps/pep-0008/#imports
> Imports are always put at the top of the file, just after any module
> comments and docstrings, and before module globals and constants.
Fixes flake8 warning:
./tests/test_connection.py:390:16: F821 undefined name 'e'
./tests/test_connection.py:391:61: F821 undefined name 'e'
./tests/test_connection.py:408:16: F821 undefined name 'e'
./tests/test_connection.py:409:61: F821 undefined name 'e'
In the event of an unexpected error, let the exception bubble up the
stack for a more informative test failure message.
Rather than deleting, the class, use the skip feature. Provides a more
informative message during test output.
Never skip DatetimeTests as all supported Python environments have the
datetime module builtin.
Previous one didn't refresh by last use. Use the stdlib version for py3
and one of our own for py2.
Max size set to 512, which should be fine for everyone (tweaking is
still possible by monkeypatching, as the tests do, but I don't want to
make an interface of it).
ctypes is available and works on all supported Pythons. It has been
available since Python 2.5. The tests were written when Python 2.4 was
still supported.
Fixes adaptation of int/long subclasses whose str() is not the number,
such IntEnum
Close#591
Note that I thought it would have needed a new adapter, so I considered
it a new feature. But it is more a shortcoming of the int adapter
failing to do something reasonable (poor Liskov, always mistreated) so I
may actually backport it if there is a new 2.7 release.
...somehow. Postgres doesn't support them and converts them into a
simple empty array. However this is not really our concern: the syntax
we return is valid.
Close#788
Added tests to check bad types, which discovered the above problem: on
type error we would have decref'd on exit something that was only
borrowed (because we wouldn't have performed matching increfs).
Fixed several shortcomings highlighted in #576 and not fixed as
requested.
Also fixed broken behaviour of ignoring the algorithm if the connection
is missing.
