Commit Graph

96 Commits

Author SHA1 Message Date
Chris Wilson
45599b2716 Add warning about send_feedback and cursor choice 2021-05-20 16:33:00 +02:00
Alexey Bashtanov
96f0f63de2 dictcursor-docs-improvement 2020-06-18 15:40:45 +01:00
Jann Kleen
4097b4f2a0 Fix typo 'againts' 2019-10-19 17:58:37 +01:00
Daniele Varrazzo
850c585501 Added UltraJSON example
Close #972
2019-10-19 17:47:01 +01:00
Alexander Kukushkin
5eec11f232 Improve docs 2019-05-06 15:42:37 +02:00
Alexander Kukushkin
f827e49f55 Change the default value of keepalive_interval parameter to None
The previous default value was 10 seconds, what might cause silent
overwrite of the *status_interval* specified in the `start_replication()`
2019-05-06 15:26:21 +02:00
Alexander Kukushkin
6cff5a3e08 Smart replication feedback
This commit makes psycopg2 responsible for sending the status update
(feedback) messages to the server regardless of whether a synchronous or
asynchronous connection is used.

Feedback is sent every *status_update* (default value is 10) seconds,
which could be configured by passing a corresponding parameter to the
`start_replication()` or `start_replication_expert()` methods.
The actual feedback message is sent by the
`pq_read_replication_message()` when the *status_update* timeout is
reached.

The default behavior of the `send_feedback()` method is changed.
It doesn't send a feedback message on every call anymore but just
updates internal structures. There is still a way to *force* sending
a message if *force* or *reply* parameters are set.

The new approach has certain advantages:
1. The client can simply call the `send_feedback()` for every
   processed message and the library will take care of not overwhelming
   the server. Actually, in the synchronous mode it is even mandatory
   to confirm every processed message.
2. The library tracks internally the pointer of the last received
   message which is not keepalive. If the client confirmed the last
   message and after that server sends only keepalives with increasing
   *wal_end*, the library can safely move forward *flush* position to
   the *wal_end* and later automatically report it to the server.

Reporting of the *wal_end* received from keepalive messages is very
important. Not doing so casing:
1. Excessive disk usage, because the replication slot prevents from
   WAL being cleaned up.
2. The smart and fast shutdown of the server could last indefinitely
   because walsender waits until the client report *flush* position
   equal to the *wal_end*.

This implementation is only extending the existing API and therefore
should not break any of the existing code.
2019-05-06 10:27:44 +02:00
Daniele Varrazzo
544e157649 Fixed replication cursor docs warning indentation level 2019-04-01 10:54:01 +01:00
Daniele Varrazzo
ed74189acd Replication objects docs header level bumped up
Now it appears in the ToCs.
2019-04-01 10:40:44 +01:00
Daniele Varrazzo
813ca30953 Fixed notice about new wal_end attribute being on the cursor 2019-04-01 10:40:06 +01:00
Daniele Varrazzo
b8bf6d9917 Added news entry about ReplicationMessage.wal_end 2019-03-30 21:23:20 +00:00
grunskis-bonial
ff91ad5186 Address code review feedback 2019-03-30 21:23:13 +00:00
Martins Grunskis
f946042a79 Store WAL end pointer in the replication cursor 2019-03-30 21:23:11 +00:00
Daniele Varrazzo
3ae9dfd545 Better link from docs to "json and jsonb" data types 2019-03-05 17:31:45 +00:00
Daniele Varrazzo
bde9fc6dea Docs wordsmith for execute_values() fetch param 2018-12-27 14:42:58 +01:00
Ivan Kotelnikov
2e591e27f2 add versionchanged to docs 2018-11-24 20:51:18 +02:00
Jon Dufresne
b07e34e0b8 Prefer https:// URLs when available 2018-09-22 19:02:33 -07:00
Jon Dufresne
19b1efd629 Drop long deprecated function register_tstz_w_secs()
Deprecated in commit b263fbf274 on
2010-01-13. The deprecation warning was first released in version 2.2.2.

The function used to register an alternate type caster for TIMESTAMP
WITH TIME ZONE to deal with historical time zones with seconds in the
UTC offset. These are now correctly handled by the default type caster,
so currently the function doesn't do anything.
2017-12-10 07:57:11 -08:00
Jon Dufresne
c1d3948be9 Update documentation to reflect JSON import behavior
The docs don't need to describe what will happen on Python versions
before 2.6 as they are unsupported by psycopg2.

Should have been included in commit
d58844e548, but was missed.
2017-11-28 06:38:44 -08:00
Jon Dufresne
05c28cce78 Remove workarounds for namedtuple on Python <= 2.5
namedtuple is available on all Python versions supported by psycopg2. It
was first introduced in Python 2.6. Can remove all workarounds and
special documentation.
2017-11-26 13:55:30 -08:00
Daniele Varrazzo
9614e7241b Further docs cleanup
Recent Sphinx versions seem overly aggressive in autodetecting python,
or I just didn't notice the errors, so be explicit in what language to
use with code examples.
2017-11-06 18:34:23 +00:00
Daniele Varrazzo
1a97445471 Accept Composable in start_replication_expert()
Close #554
2017-06-17 03:30:44 +01:00
Sebastian Bank
30af82ef2f Document that executemany() is not logged with LoggingConnection 2017-06-15 23:27:28 +01:00
Daniele Varrazzo
165449c724 Added doc link to replication commands 2017-05-10 01:55:01 +01:00
Jon Nelson
c661695b28 correct minor grammatical issue with deprecation warning 2017-03-03 13:54:22 +00:00
Daniele Varrazzo
27652ed3b0 Added notes about using execute_batch with prepared statements 2017-02-02 02:40:03 +00:00
Daniele Varrazzo
a95fd3df1a Added execute_batch and execute_values functions 2017-02-01 01:59:47 +00:00
Daniele Varrazzo
643ba70bad Added ipaddress objects conversion
Close #387
2016-10-11 04:54:41 +01:00
Daniele Varrazzo
706ad2f177 Conver network array types into array of strings by default 2016-10-11 02:31:45 +01:00
Daniele Varrazzo
86198c1c21 inet adapters deprecated
Close #343
2016-10-11 02:17:56 +01:00
Daniele Varrazzo
86434548a7 Replication docs massaging, mostly formatting 2016-08-07 02:23:02 +01:00
Daniele Varrazzo
9a4f8f915f Reshuffling and indexing of replication docs 2016-08-07 02:07:16 +01:00
Oleksandr Shulgin
8b79bf43ac Drop ReplicationCursor.flush_feedback(), rectify pq_*_replication_*() interface. 2015-10-23 18:30:18 +02:00
Oleksandr Shulgin
dd6bcbd04f Improve async replication example. 2015-10-23 17:51:03 +02:00
Oleksandr Shulgin
e69dafbecc Move the decode parameter to start_replication().
It makes more sense this way, because otherwise it must be passed to every call
of `read_message()`.
2015-10-23 11:31:55 +02:00
Oleksandr Shulgin
23abe4f501 Add quick start to the replication doc, minor doc fixes. 2015-10-20 12:36:13 +02:00
Oleksandr Shulgin
0bb81fc848 Properly subclass ReplicationCursor on C level. 2015-10-19 20:00:39 +02:00
Oleksandr Shulgin
4ab7cf0157 Replace stop_replication with requirement for an exception. 2015-10-19 15:42:42 +02:00
Oleksandr Shulgin
8e518d4954 Merge branch 'master' into feature/replication-protocol 2015-10-15 12:27:43 +02:00
Oleksandr Shulgin
e3097ec956 Fix select/timeout indication in async replication example 2015-10-14 17:42:53 +02:00
Oleksandr Shulgin
a0b42a12ff Update stop_repl, require replication consumer to be a callable. 2015-10-14 15:15:07 +02:00
Oleksandr Shulgin
6ad299945f Remove IDENTIFY_SYSTEM wrapper method (it can't work with async anyway). 2015-10-13 18:05:33 +02:00
Oleksandr Shulgin
ea2b87eade Fix create_replication_slot doc signature 2015-10-13 11:01:13 +02:00
Oleksandr Shulgin
0233620c26 Rework replication connection/cursor classes 2015-10-01 19:33:27 +02:00
Oleksandr Shulgin
95ee218c6d Update replication connection/cursor interface and docs. 2015-10-01 19:33:20 +02:00
Oleksandr Shulgin
937a7a9024 Cleanup start replication wrt. slot type a bit. 2015-10-01 19:33:16 +02:00
Daniele Varrazzo
f635547ec6 The wait_select callback can cancel a query using Ctrl-C
Fixes #333.
2015-10-01 15:26:13 +01:00
Oleksandr Shulgin
9386653d72 Update docs on ReplicationCursor 2015-07-01 14:08:32 +02:00
Oleksandr Shulgin
318706f28c Update docs for Replication protocol 2015-06-30 16:17:31 +02:00
Oleksandr Shulgin
e32e1b834e Add support for streaming replication protocol
Introduce ReplicationConnection and ReplicationCursor classes, that
incapsulate initiation of special type of PostgreSQL connection and
handling of special replication commands only available in this special
connection mode.

The handling of stream of replication data from the server is modelled
largely after the existing support for "COPY table TO file" command and
pg_recvlogical tool supplied with PostgreSQL (though, it can also be
used for physical replication.)
2015-06-01 11:45:04 +02:00