Oid is defined as unsigned 32. On some Python implementations (probably
the ones where maxint = 2 ** 31) this can cause int overflow for large
values (see #961). On my 64 box it doesn't seem the case.
Oid handling was sloppy here and there (messages, casts...): trying to
use uint everywhere, and added a couple of helper macros to treat Oid
consistently.
Close#961.
This commit makes psycopg2 responsible for sending the status update
(feedback) messages to the server regardless of whether a synchronous or
asynchronous connection is used.
Feedback is sent every *status_update* (default value is 10) seconds,
which could be configured by passing a corresponding parameter to the
`start_replication()` or `start_replication_expert()` methods.
The actual feedback message is sent by the
`pq_read_replication_message()` when the *status_update* timeout is
reached.
The default behavior of the `send_feedback()` method is changed.
It doesn't send a feedback message on every call anymore but just
updates internal structures. There is still a way to *force* sending
a message if *force* or *reply* parameters are set.
The new approach has certain advantages:
1. The client can simply call the `send_feedback()` for every
processed message and the library will take care of not overwhelming
the server. Actually, in the synchronous mode it is even mandatory
to confirm every processed message.
2. The library tracks internally the pointer of the last received
message which is not keepalive. If the client confirmed the last
message and after that server sends only keepalives with increasing
*wal_end*, the library can safely move forward *flush* position to
the *wal_end* and later automatically report it to the server.
Reporting of the *wal_end* received from keepalive messages is very
important. Not doing so casing:
1. Excessive disk usage, because the replication slot prevents from
WAL being cleaned up.
2. The smart and fast shutdown of the server could last indefinitely
because walsender waits until the client report *flush* position
equal to the *wal_end*.
This implementation is only extending the existing API and therefore
should not break any of the existing code.
On Py3 subprocessing will actually GC the objects and the FD is open,
resulting in connections closed in different processes.
The behaviour is verified in py 3.4 to 3.7 at least,
There's not so much need for a strict convention for static functions.
Leaving some 'psyco_' prefix when the internal function and the
python-exposed function clashed.
The new function keeps together PQconsumeInput() with PQisBusy(), in
order to handle the condition in which not all the results of a sequence
of statements arrive in the same roundtrip.
Added pointer to a PGresult to the connection to keep the state across
async communication: it can probably be used to simplify other code
paths where a result is brought forward manually.
Close#802Close#855Close#856
Have advance_write calling flush itself, advance_read calling is_busy
itself, rather than calling them in the caller and passing the result.
Why we were doing the former on write I don't know. On read the paths
between async and green function was different but they got unified in
the previous commit.
The locking version was used for sync connections, the non-locking one
for green ones. However it only calls non-blocking functions, so it
doesn't really matter releasing the gil. So have only the non-locking
one.
Note that the name are sort of swapped: pq_is_busy() does now what
pq_is_busy_locked() used to do.
The use_pydatetime configuration option has been removed. Now, always
default to Python's builtin datetime. mxDatetime support continues to be
available as an alternative to Python's builtin datetime.
Pass around the module instead of its dict (getting the latter is fast
if needed), mark function raising with negative results, check all errors,
consistent names...
Dropped code duplications, more regular increc/decref pattern.
Check the return value of formatting named cursor: would have segfaulted
in case of error.
Fixes adaptation of int/long subclasses whose str() is not the number,
such IntEnum
Close#591
Note that I thought it would have needed a new adapter, so I considered
it a new feature. But it is more a shortcoming of the int adapter
failing to do something reasonable (poor Liskov, always mistreated) so I
may actually backport it if there is a new 2.7 release.
...somehow. Postgres doesn't support them and converts them into a
simple empty array. However this is not really our concern: the syntax
we return is valid.
Close#788
I'm still fought whether docs should be in the C module or in the .rst.
I'd prefer the first because DRY, but writing multiline strings in C
really sucks.
