2010-02-26 03:17:52 +03:00
|
|
|
`psycopg2.extras` -- Miscellaneous goodies for Psycopg 2
|
2010-02-09 07:58:28 +03:00
|
|
|
=============================================================
|
|
|
|
|
|
|
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
|
|
|
|
|
|
|
.. module:: psycopg2.extras
|
|
|
|
|
2010-02-13 19:06:39 +03:00
|
|
|
.. testsetup::
|
|
|
|
|
|
|
|
import psycopg2.extras
|
2010-02-14 23:32:35 +03:00
|
|
|
from psycopg2.extras import Inet
|
2010-02-13 19:06:39 +03:00
|
|
|
|
|
|
|
create_test_table()
|
|
|
|
|
2010-02-09 07:58:28 +03:00
|
|
|
This module is a generic place used to hold little helper functions and
|
|
|
|
classes until a better place in the distribution is found.
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2013-04-07 05:30:12 +04:00
|
|
|
.. _cursor-subclasses:
|
2010-11-06 04:39:43 +03:00
|
|
|
|
|
|
|
Connection and cursor subclasses
|
|
|
|
--------------------------------
|
|
|
|
|
|
|
|
A few objects that change the way the results are returned by the cursor or
|
2013-04-07 05:30:12 +04:00
|
|
|
modify the object behavior in some other way. Typically `!cursor` subclasses
|
|
|
|
are passed as *cursor_factory* argument to `~psycopg2.connect()` so that the
|
|
|
|
connection's `~connection.cursor()` method will generate objects of this
|
|
|
|
class. Alternatively a `!cursor` subclass can be used one-off by passing it
|
|
|
|
as the *cursor_factory* argument to the `!cursor()` method.
|
|
|
|
|
|
|
|
If you want to use a `!connection` subclass you can pass it as the
|
|
|
|
*connection_factory* argument of the `!connect()` function.
|
|
|
|
|
|
|
|
|
|
|
|
.. index::
|
|
|
|
pair: Cursor; Dictionary
|
|
|
|
|
|
|
|
.. _dict-cursor:
|
|
|
|
|
2010-11-06 04:39:43 +03:00
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
Dictionary-like cursor
|
2010-11-06 04:39:43 +03:00
|
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2013-04-26 12:21:56 +04:00
|
|
|
The dict cursors allow to access to the retrieved records using an interface
|
2010-11-06 04:39:43 +03:00
|
|
|
similar to the Python dictionaries instead of the tuples.
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2010-02-13 19:06:39 +03:00
|
|
|
>>> dict_cur = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
|
|
|
|
>>> dict_cur.execute("INSERT INTO test (num, data) VALUES(%s, %s)",
|
|
|
|
... (100, "abc'def"))
|
|
|
|
>>> dict_cur.execute("SELECT * FROM test")
|
|
|
|
>>> rec = dict_cur.fetchone()
|
2010-02-13 05:10:51 +03:00
|
|
|
>>> rec['id']
|
|
|
|
1
|
|
|
|
>>> rec['num']
|
|
|
|
100
|
|
|
|
>>> rec['data']
|
|
|
|
"abc'def"
|
|
|
|
|
2010-02-14 08:02:09 +03:00
|
|
|
The records still support indexing as the original tuple:
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
>>> rec[2]
|
|
|
|
"abc'def"
|
|
|
|
|
|
|
|
|
|
|
|
.. autoclass:: DictCursor
|
|
|
|
|
|
|
|
.. autoclass:: DictConnection
|
|
|
|
|
2013-04-07 05:30:12 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
Not very useful since Psycopg 2.5: you can use `psycopg2.connect`\
|
|
|
|
``(dsn, cursor_factory=DictCursor)`` instead of `!DictConnection`.
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
.. autoclass:: DictRow
|
|
|
|
|
|
|
|
|
|
|
|
Real dictionary cursor
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
.. autoclass:: RealDictCursor
|
|
|
|
|
|
|
|
.. autoclass:: RealDictConnection
|
|
|
|
|
2013-04-07 05:30:12 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
Not very useful since Psycopg 2.5: you can use `psycopg2.connect`\
|
|
|
|
``(dsn, cursor_factory=RealDictCursor)`` instead of
|
|
|
|
`!RealDictConnection`.
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
.. autoclass:: RealDictRow
|
|
|
|
|
|
|
|
|
|
|
|
|
2010-11-06 04:39:43 +03:00
|
|
|
.. index::
|
|
|
|
pair: Cursor; namedtuple
|
|
|
|
|
|
|
|
`namedtuple` cursor
|
|
|
|
^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
|
|
|
.. versionadded:: 2.3
|
|
|
|
|
2011-02-19 19:16:28 +03:00
|
|
|
These objects require :py:func:`collections.namedtuple` to be found, so it is
|
2010-11-06 04:39:43 +03:00
|
|
|
available out-of-the-box only from Python 2.6. Anyway, the namedtuple
|
|
|
|
implementation is compatible with previous Python versions, so all you
|
2010-11-06 07:40:47 +03:00
|
|
|
have to do is to `download it`__ and make it available where we
|
2010-11-06 04:39:43 +03:00
|
|
|
expect it to be... ::
|
|
|
|
|
|
|
|
from somewhere import namedtuple
|
|
|
|
import collections
|
|
|
|
collections.namedtuple = namedtuple
|
|
|
|
from psycopg.extras import NamedTupleConnection
|
|
|
|
# ...
|
|
|
|
|
|
|
|
.. __: http://code.activestate.com/recipes/500261-named-tuples/
|
|
|
|
|
|
|
|
.. autoclass:: NamedTupleCursor
|
|
|
|
|
|
|
|
.. autoclass:: NamedTupleConnection
|
|
|
|
|
2013-04-07 05:30:12 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
Not very useful since Psycopg 2.5: you can use `psycopg2.connect`\
|
|
|
|
``(dsn, cursor_factory=NamedTupleCursor)`` instead of
|
|
|
|
`!NamedTupleConnection`.
|
|
|
|
|
2010-11-06 04:39:43 +03:00
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
.. index::
|
|
|
|
pair: Cursor; Logging
|
|
|
|
|
|
|
|
Logging cursor
|
2010-11-06 04:39:43 +03:00
|
|
|
^^^^^^^^^^^^^^
|
2010-02-13 05:10:51 +03:00
|
|
|
|
|
|
|
.. autoclass:: LoggingConnection
|
|
|
|
:members: initialize,filter
|
|
|
|
|
|
|
|
.. autoclass:: LoggingCursor
|
|
|
|
|
|
|
|
|
|
|
|
.. autoclass:: MinTimeLoggingConnection
|
|
|
|
:members: initialize,filter
|
|
|
|
|
|
|
|
.. autoclass:: MinTimeLoggingCursor
|
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
|
|
|
|
|
2016-08-07 04:07:16 +03:00
|
|
|
.. _replication-objects:
|
2015-10-20 13:36:13 +03:00
|
|
|
|
|
|
|
Replication connection and cursor classes
|
2016-08-07 04:07:16 +03:00
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
2016-08-07 04:23:02 +03:00
|
|
|
See :ref:`replication-support` for an introduction to the topic.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
|
|
|
|
|
|
|
The following replication types are defined:
|
|
|
|
|
|
|
|
.. data:: REPLICATION_LOGICAL
|
|
|
|
.. data:: REPLICATION_PHYSICAL
|
|
|
|
|
|
|
|
|
2016-08-07 04:23:02 +03:00
|
|
|
.. index::
|
|
|
|
pair: Connection; replication
|
|
|
|
|
2015-10-01 16:34:51 +03:00
|
|
|
.. autoclass:: LogicalReplicationConnection
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
This connection factory class can be used to open a special type of
|
|
|
|
connection that is used for logical replication.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Example::
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
from psycopg2.extras import LogicalReplicationConnection
|
|
|
|
log_conn = psycopg2.connect(dsn, connection_factory=LogicalReplicationConnection)
|
|
|
|
log_cur = log_conn.cursor()
|
2015-10-01 16:34:51 +03:00
|
|
|
|
|
|
|
|
|
|
|
.. autoclass:: PhysicalReplicationConnection
|
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
This connection factory class can be used to open a special type of
|
|
|
|
connection that is used for physical replication.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Example::
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
from psycopg2.extras import PhysicalReplicationConnection
|
|
|
|
phys_conn = psycopg2.connect(dsn, connection_factory=PhysicalReplicationConnection)
|
|
|
|
phys_cur = phys_conn.cursor()
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Both `LogicalReplicationConnection` and `PhysicalReplicationConnection` use
|
2015-10-20 13:36:13 +03:00
|
|
|
`ReplicationCursor` for actual communication with the server.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2016-08-07 04:07:16 +03:00
|
|
|
.. index::
|
|
|
|
pair: Message; replication
|
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
The individual messages in the replication stream are represented by
|
|
|
|
`ReplicationMessage` objects (both logical and physical type):
|
2015-10-01 20:28:00 +03:00
|
|
|
|
|
|
|
.. autoclass:: ReplicationMessage
|
|
|
|
|
|
|
|
.. attribute:: payload
|
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
The actual data received from the server.
|
|
|
|
|
|
|
|
An instance of either `bytes()` or `unicode()`, depending on the value
|
2016-08-07 04:23:02 +03:00
|
|
|
of `decode` option passed to `~ReplicationCursor.start_replication()`
|
|
|
|
on the connection. See `~ReplicationCursor.read_message()` for
|
2015-10-23 12:31:55 +03:00
|
|
|
details.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
|
|
|
.. attribute:: data_size
|
|
|
|
|
|
|
|
The raw size of the message payload (before possible unicode
|
|
|
|
conversion).
|
|
|
|
|
|
|
|
.. attribute:: data_start
|
|
|
|
|
|
|
|
LSN position of the start of the message.
|
|
|
|
|
|
|
|
.. attribute:: wal_end
|
|
|
|
|
|
|
|
LSN position of the current end of WAL on the server.
|
|
|
|
|
|
|
|
.. attribute:: send_time
|
|
|
|
|
|
|
|
A `~datetime` object representing the server timestamp at the moment
|
|
|
|
when the message was sent.
|
|
|
|
|
|
|
|
.. attribute:: cursor
|
|
|
|
|
|
|
|
A reference to the corresponding `ReplicationCursor` object.
|
|
|
|
|
|
|
|
|
2016-08-07 04:07:16 +03:00
|
|
|
.. index::
|
|
|
|
pair: Cursor; replication
|
|
|
|
|
2015-06-01 12:35:05 +03:00
|
|
|
.. autoclass:: ReplicationCursor
|
|
|
|
|
2015-10-13 12:01:13 +03:00
|
|
|
.. method:: create_replication_slot(slot_name, slot_type=None, output_plugin=None)
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Create streaming replication slot.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
:param slot_name: name of the replication slot to be created
|
|
|
|
:param slot_type: type of replication: should be either
|
|
|
|
`REPLICATION_LOGICAL` or `REPLICATION_PHYSICAL`
|
|
|
|
:param output_plugin: name of the logical decoding output plugin to be
|
|
|
|
used by the slot; required for logical
|
|
|
|
replication connections, disallowed for physical
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Example::
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
log_cur.create_replication_slot("logical1", "test_decoding")
|
|
|
|
phys_cur.create_replication_slot("physical1")
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
# either logical or physical replication connection
|
|
|
|
cur.create_replication_slot("slot1", slot_type=REPLICATION_LOGICAL)
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
When creating a slot on a logical replication connection, a logical
|
|
|
|
replication slot is created by default. Logical replication requires
|
|
|
|
name of the logical decoding output plugin to be specified.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
When creating a slot on a physical replication connection, a physical
|
|
|
|
replication slot is created by default. No output plugin parameter is
|
|
|
|
required or allowed when creating a physical replication slot.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
In either case the type of slot being created can be specified
|
2015-10-01 20:28:00 +03:00
|
|
|
explicitly using *slot_type* parameter.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Replication slots are a feature of PostgreSQL server starting with
|
|
|
|
version 9.4.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
|
|
|
.. method:: drop_replication_slot(slot_name)
|
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Drop streaming replication slot.
|
|
|
|
|
|
|
|
:param slot_name: name of the replication slot to drop
|
|
|
|
|
|
|
|
Example::
|
|
|
|
|
|
|
|
# either logical or physical replication connection
|
|
|
|
cur.drop_replication_slot("slot1")
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Replication slots are a feature of PostgreSQL server starting with
|
|
|
|
version 9.4.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
.. method:: start_replication(slot_name=None, slot_type=None, start_lsn=0, timeline=0, options=None, decode=False)
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
Start replication on the connection.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
:param slot_name: name of the replication slot to use; required for
|
|
|
|
logical replication, physical replication can work
|
|
|
|
with or without a slot
|
|
|
|
:param slot_type: type of replication: should be either
|
|
|
|
`REPLICATION_LOGICAL` or `REPLICATION_PHYSICAL`
|
|
|
|
:param start_lsn: the optional LSN position to start replicating from,
|
|
|
|
can be an integer or a string of hexadecimal digits
|
|
|
|
in the form ``XXX/XXX``
|
|
|
|
:param timeline: WAL history timeline to start streaming from (optional,
|
|
|
|
can only be used with physical replication)
|
|
|
|
:param options: a dictionary of options to pass to logical replication
|
|
|
|
slot (not allowed with physical replication)
|
2015-10-23 12:31:55 +03:00
|
|
|
:param decode: a flag indicating that unicode conversion should be
|
|
|
|
performed on messages received from the server
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
If a *slot_name* is specified, the slot must exist on the server and
|
|
|
|
its type must match the replication type used.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
If not specified using *slot_type* parameter, the type of replication
|
|
|
|
is defined by the type of replication connection. Logical replication
|
|
|
|
is only allowed on logical replication connection, but physical
|
|
|
|
replication can be used with both types of connection.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
On the other hand, physical replication doesn't require a named
|
2015-10-20 13:36:13 +03:00
|
|
|
replication slot to be used, only logical replication does. In any
|
|
|
|
case logical replication and replication slots are a feature of
|
|
|
|
PostgreSQL server starting with version 9.4. Physical replication can
|
|
|
|
be used starting with 9.0.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
If *start_lsn* is specified, the requested stream will start from that
|
2015-10-20 13:36:13 +03:00
|
|
|
LSN. The default is `!None` which passes the LSN ``0/0`` causing
|
|
|
|
replay to begin at the last point for which the server got flush
|
|
|
|
confirmation from the client, or the oldest available point for a new
|
|
|
|
slot.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
The server might produce an error if a WAL file for the given LSN has
|
2015-10-20 13:36:13 +03:00
|
|
|
already been recycled or it may silently start streaming from a later
|
2015-10-01 20:28:00 +03:00
|
|
|
position: the client can verify the actual position using information
|
2015-10-20 13:36:13 +03:00
|
|
|
provided by the `ReplicationMessage` attributes. The exact server
|
2015-10-01 20:28:00 +03:00
|
|
|
behavior depends on the type of replication and use of slots.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
The *timeline* parameter can only be specified with physical
|
|
|
|
replication and only starting with server version 9.3.
|
2015-10-01 16:34:51 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
A dictionary of *options* may be passed to the logical decoding plugin
|
|
|
|
on a logical replication slot. The set of supported options depends
|
|
|
|
on the output plugin that was used to create the slot. Must be
|
|
|
|
`!None` for physical replication.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
If *decode* is set to `!True` the messages received from the server
|
|
|
|
would be converted according to the connection `~connection.encoding`.
|
|
|
|
*This parameter should not be set with physical replication or with
|
|
|
|
logical replication plugins that produce binary output.*
|
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
This function constructs a ``START_REPLICATION`` command and calls
|
|
|
|
`start_replication_expert()` internally.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
After starting the replication, to actually consume the incoming
|
2015-10-20 13:36:13 +03:00
|
|
|
server messages use `consume_stream()` or implement a loop around
|
|
|
|
`read_message()` in case of :ref:`asynchronous connection
|
|
|
|
<async-support>`.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
.. method:: start_replication_expert(command, decode=False)
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
Start replication on the connection using provided
|
|
|
|
``START_REPLICATION`` command. See `start_replication()` for
|
|
|
|
description of *decode* parameter.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
.. method:: consume_stream(consume, keepalive_interval=10)
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2016-08-07 04:23:02 +03:00
|
|
|
:param consume: a callable object with signature :samp:`consume({msg})`
|
2015-10-01 20:28:00 +03:00
|
|
|
:param keepalive_interval: interval (in seconds) to send keepalive
|
|
|
|
messages to the server
|
|
|
|
|
|
|
|
This method can only be used with synchronous connection. For
|
2015-10-19 21:00:39 +03:00
|
|
|
asynchronous connections see `read_message()`.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
Before using this method to consume the stream call
|
2015-10-01 20:28:00 +03:00
|
|
|
`start_replication()` first.
|
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
This method enters an endless loop reading messages from the server
|
2015-10-23 12:31:55 +03:00
|
|
|
and passing them to ``consume()`` one at a time, then waiting for more
|
|
|
|
messages from the server. In order to make this method break out of
|
|
|
|
the loop and return, ``consume()`` can throw a `StopReplication`
|
|
|
|
exception. Any unhandled exception will make it break out of the loop
|
|
|
|
as well.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
The *msg* object passed to ``consume()`` is an instance of
|
|
|
|
`ReplicationMessage` class. See `read_message()` for details about
|
|
|
|
message decoding.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
This method also sends keepalive messages to the server in case there
|
2015-10-01 20:28:00 +03:00
|
|
|
were no new data from the server for the duration of
|
|
|
|
*keepalive_interval* (in seconds). The value of this parameter must
|
2015-10-20 13:36:13 +03:00
|
|
|
be set to at least 1 second, but it can have a fractional part.
|
|
|
|
|
|
|
|
After processing certain amount of messages the client should send a
|
|
|
|
confirmation message to the server. This should be done by calling
|
|
|
|
`send_feedback()` method on the corresponding replication cursor. A
|
|
|
|
reference to the cursor is provided in the `ReplicationMessage` as an
|
|
|
|
attribute.
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2015-10-14 16:15:07 +03:00
|
|
|
The following example is a sketch implementation of ``consume()``
|
|
|
|
callable for logical replication::
|
2015-10-01 20:28:00 +03:00
|
|
|
|
|
|
|
class LogicalStreamConsumer(object):
|
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
...
|
|
|
|
|
2015-10-14 16:15:07 +03:00
|
|
|
def __call__(self, msg):
|
2015-10-20 13:36:13 +03:00
|
|
|
self.process_message(msg.payload)
|
2015-10-01 20:28:00 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
if self.should_send_feedback(msg):
|
2015-10-19 21:00:39 +03:00
|
|
|
msg.cursor.send_feedback(flush_lsn=msg.data_start)
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
consumer = LogicalStreamConsumer()
|
2015-10-23 12:31:55 +03:00
|
|
|
cur.consume_stream(consumer)
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-01 20:28:00 +03:00
|
|
|
.. warning::
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
When using replication with slots, failure to constantly consume
|
|
|
|
*and* report success to the server appropriately can eventually
|
|
|
|
lead to "disk full" condition on the server, because the server
|
|
|
|
retains all the WAL segments that might be needed to stream the
|
|
|
|
changes via all of the currently open replication slots.
|
2015-06-01 12:35:05 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
On the other hand, it is not recommended to send confirmation
|
|
|
|
after *every* processed message, since that will put an
|
|
|
|
unnecessary load on network and the server. A possible strategy
|
|
|
|
is to confirm after every COMMIT message.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-19 21:00:39 +03:00
|
|
|
.. method:: send_feedback(write_lsn=0, flush_lsn=0, apply_lsn=0, reply=False)
|
2015-06-30 17:17:31 +03:00
|
|
|
|
|
|
|
:param write_lsn: a LSN position up to which the client has written the data locally
|
2015-10-20 13:36:13 +03:00
|
|
|
:param flush_lsn: a LSN position up to which the client has processed the
|
2015-06-30 17:17:31 +03:00
|
|
|
data reliably (the server is allowed to discard all
|
|
|
|
and every data that predates this LSN)
|
|
|
|
:param apply_lsn: a LSN position up to which the warm standby server
|
|
|
|
has applied the changes (physical replication
|
|
|
|
master-slave protocol only)
|
|
|
|
:param reply: request the server to send back a keepalive message immediately
|
|
|
|
|
|
|
|
Use this method to report to the server that all messages up to a
|
2015-10-20 13:36:13 +03:00
|
|
|
certain LSN position have been processed on the client and may be
|
2015-10-01 20:28:00 +03:00
|
|
|
discarded on the server.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-07-01 15:08:32 +03:00
|
|
|
This method can also be called with all default parameters' values to
|
2015-10-01 20:28:00 +03:00
|
|
|
just send a keepalive message to the server.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
Low-level replication cursor methods for :ref:`asynchronous connection
|
|
|
|
<async-support>` operation.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
With the synchronous connection a call to `consume_stream()` handles all
|
2015-10-19 21:00:39 +03:00
|
|
|
the complexity of handling the incoming messages and sending keepalive
|
|
|
|
replies, but at times it might be beneficial to use low-level interface
|
2015-10-20 13:36:13 +03:00
|
|
|
for better control, in particular to `~select` on multiple sockets. The
|
|
|
|
following methods are provided for asynchronous operation:
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
.. method:: read_message()
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-10-23 12:31:55 +03:00
|
|
|
Try to read the next message from the server without blocking and
|
|
|
|
return an instance of `ReplicationMessage` or `!None`, in case there
|
|
|
|
are no more data messages from the server at the moment.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
|
|
|
This method should be used in a loop with asynchronous connections
|
2015-10-23 12:31:55 +03:00
|
|
|
(after calling `start_replication()` once). For synchronous
|
|
|
|
connections see `consume_stream()`.
|
|
|
|
|
2016-08-07 04:23:02 +03:00
|
|
|
The returned message's `~ReplicationMessage.payload` is an instance of
|
|
|
|
`!unicode` decoded according to connection `~connection.encoding`
|
|
|
|
*iff* *decode* was set to `!True` in the initial call to
|
2015-10-23 12:31:55 +03:00
|
|
|
`start_replication()` on this connection, otherwise it is an instance
|
2016-08-07 04:23:02 +03:00
|
|
|
of `!bytes` with no decoding.
|
2015-07-01 15:08:32 +03:00
|
|
|
|
|
|
|
It is expected that the calling code will call this method repeatedly
|
2015-10-20 13:36:13 +03:00
|
|
|
in order to consume all of the messages that might have been buffered
|
|
|
|
until `!None` is returned. After receiving `!None` from this method
|
|
|
|
the caller should use `~select.select()` or `~select.poll()` on the
|
|
|
|
corresponding connection to block the process until there is more data
|
|
|
|
from the server.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
|
|
|
The server can send keepalive messages to the client periodically.
|
|
|
|
Such messages are silently consumed by this method and are never
|
|
|
|
reported to the caller.
|
|
|
|
|
|
|
|
.. method:: fileno()
|
|
|
|
|
2015-07-01 15:08:32 +03:00
|
|
|
Call the corresponding connection's `~connection.fileno()` method and
|
|
|
|
return the result.
|
2015-06-30 17:17:31 +03:00
|
|
|
|
|
|
|
This is a convenience method which allows replication cursor to be
|
|
|
|
used directly in `~select.select()` or `~select.poll()` calls.
|
|
|
|
|
2015-10-19 21:00:39 +03:00
|
|
|
.. attribute:: io_timestamp
|
2015-06-30 17:17:31 +03:00
|
|
|
|
|
|
|
A `~datetime` object representing the timestamp at the moment of last
|
|
|
|
communication with the server (a data or keepalive message in either
|
|
|
|
direction).
|
|
|
|
|
|
|
|
An actual example of asynchronous operation might look like this::
|
|
|
|
|
2015-10-23 18:51:03 +03:00
|
|
|
from select import select
|
|
|
|
from datetime import datetime
|
|
|
|
|
2015-10-20 13:36:13 +03:00
|
|
|
def consume(msg):
|
|
|
|
...
|
|
|
|
|
|
|
|
keepalive_interval = 10.0
|
|
|
|
while True:
|
|
|
|
msg = cur.read_message()
|
|
|
|
if msg:
|
|
|
|
consume(msg)
|
|
|
|
else:
|
|
|
|
now = datetime.now()
|
|
|
|
timeout = keepalive_interval - (now - cur.io_timestamp).total_seconds()
|
2015-10-23 18:51:03 +03:00
|
|
|
try:
|
|
|
|
sel = select([cur], [], [], max(0, timeout))
|
|
|
|
if not any(sel):
|
|
|
|
cur.send_feedback() # timed out, send keepalive message
|
|
|
|
except InterruptedError:
|
|
|
|
pass # recalculate timeout and continue
|
2015-06-30 17:17:31 +03:00
|
|
|
|
2015-06-01 12:35:05 +03:00
|
|
|
.. index::
|
|
|
|
pair: Cursor; Replication
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2015-10-19 16:42:42 +03:00
|
|
|
.. autoclass:: StopReplication
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2010-11-06 04:39:43 +03:00
|
|
|
.. index::
|
|
|
|
single: Data types; Additional
|
|
|
|
|
|
|
|
Additional data types
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
|
2012-09-19 07:12:20 +04:00
|
|
|
.. index::
|
|
|
|
pair: JSON; Data types
|
|
|
|
pair: JSON; Adaptation
|
|
|
|
|
2013-03-18 23:22:31 +04:00
|
|
|
.. _adapt-json:
|
|
|
|
|
2012-09-24 14:51:35 +04:00
|
|
|
JSON_ adaptation
|
|
|
|
^^^^^^^^^^^^^^^^
|
2012-09-19 07:12:20 +04:00
|
|
|
|
2012-12-02 16:04:24 +04:00
|
|
|
.. versionadded:: 2.5
|
2014-08-13 04:32:19 +04:00
|
|
|
.. versionchanged:: 2.5.4
|
|
|
|
added |jsonb| support. In previous versions |jsonb| values are returned
|
|
|
|
as strings. See :ref:`the FAQ <faq-jsonb-adapt>` for a workaround.
|
2012-09-19 07:12:20 +04:00
|
|
|
|
2014-08-13 04:32:19 +04:00
|
|
|
Psycopg can adapt Python objects to and from the PostgreSQL |pgjson|_ and
|
|
|
|
|jsonb| types. With PostgreSQL 9.2 and following versions adaptation is
|
|
|
|
available out-of-the-box. To use JSON data with previous database versions
|
|
|
|
(either with the `9.1 json extension`__, but even if you want to convert text
|
|
|
|
fields to JSON) you can use the `register_json()` function.
|
2012-09-24 14:51:35 +04:00
|
|
|
|
|
|
|
.. __: http://people.planetpostgresql.org/andrew/index.php?/archives/255-JSON-for-PG-9.2-...-and-now-for-9.1!.html
|
|
|
|
|
2014-08-13 04:32:19 +04:00
|
|
|
The Python library used by default to convert Python objects to JSON and to
|
|
|
|
parse data from the database depends on the language version: with Python 2.6
|
|
|
|
and following the :py:mod:`json` module from the standard library is used;
|
|
|
|
with previous versions the `simplejson`_ module is used if available. Note
|
|
|
|
that the last `!simplejson` version supporting Python 2.4 is the 2.0.9.
|
2012-09-19 07:12:20 +04:00
|
|
|
|
|
|
|
.. _JSON: http://www.json.org/
|
|
|
|
.. |pgjson| replace:: :sql:`json`
|
2014-08-13 04:32:19 +04:00
|
|
|
.. |jsonb| replace:: :sql:`jsonb`
|
2012-09-19 07:12:20 +04:00
|
|
|
.. _pgjson: http://www.postgresql.org/docs/current/static/datatype-json.html
|
|
|
|
.. _simplejson: http://pypi.python.org/pypi/simplejson/
|
|
|
|
|
2012-09-24 14:51:35 +04:00
|
|
|
In order to pass a Python object to the database as query argument you can use
|
|
|
|
the `Json` adapter::
|
|
|
|
|
|
|
|
curs.execute("insert into mytable (jsondata) values (%s)",
|
|
|
|
[Json({'a': 100})])
|
|
|
|
|
2014-08-13 04:32:19 +04:00
|
|
|
Reading from the database, |pgjson| and |jsonb| values will be automatically
|
|
|
|
converted to Python objects.
|
2012-09-24 14:51:35 +04:00
|
|
|
|
2014-01-14 22:30:41 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
If you are using the PostgreSQL :sql:`json` data type but you want to read
|
|
|
|
it as string in Python instead of having it parsed, your can either cast
|
|
|
|
the column to :sql:`text` in the query (it is an efficient operation, that
|
|
|
|
doesn't involve a copy)::
|
|
|
|
|
|
|
|
cur.execute("select jsondata::text from mytable")
|
|
|
|
|
|
|
|
or you can register a no-op `!loads()` function with
|
|
|
|
`register_default_json()`::
|
|
|
|
|
|
|
|
psycopg2.extras.register_default_json(loads=lambda x: x)
|
|
|
|
|
2012-09-24 14:51:35 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
You can use `~psycopg2.extensions.register_adapter()` to adapt any Python
|
|
|
|
dictionary to JSON, either registering `Json` or any subclass or factory
|
|
|
|
creating a compatible adapter::
|
|
|
|
|
|
|
|
psycopg2.extensions.register_adapter(dict, psycopg2.extras.Json)
|
|
|
|
|
|
|
|
This setting is global though, so it is not compatible with similar
|
|
|
|
adapters such as the one registered by `register_hstore()`. Any other
|
|
|
|
object supported by JSON can be registered the same way, but this will
|
|
|
|
clobber the default adaptation rule, so be careful to unwanted side
|
|
|
|
effects.
|
|
|
|
|
|
|
|
If you want to customize the adaptation from Python to PostgreSQL you can
|
2014-01-14 22:30:41 +04:00
|
|
|
either provide a custom `!dumps()` function to `Json`::
|
2012-09-24 14:51:35 +04:00
|
|
|
|
|
|
|
curs.execute("insert into mytable (jsondata) values (%s)",
|
|
|
|
[Json({'a': 100}, dumps=simplejson.dumps)])
|
|
|
|
|
|
|
|
or you can subclass it overriding the `~Json.dumps()` method::
|
|
|
|
|
|
|
|
class MyJson(Json):
|
|
|
|
def dumps(self, obj):
|
|
|
|
return simplejson.dumps(obj)
|
|
|
|
|
|
|
|
curs.execute("insert into mytable (jsondata) values (%s)",
|
|
|
|
[MyJson({'a': 100})])
|
|
|
|
|
|
|
|
Customizing the conversion from PostgreSQL to Python can be done passing a
|
2014-08-13 04:32:19 +04:00
|
|
|
custom `!loads()` function to `register_json()`. For the builtin data types
|
|
|
|
(|pgjson| from PostgreSQL 9.2, |jsonb| from PostgreSQL 9.4) use
|
|
|
|
`register_default_json()` and `register_default_jsonb()`. For example, if you
|
|
|
|
want to convert the float values from :sql:`json` into
|
|
|
|
:py:class:`~decimal.Decimal` you can use::
|
2012-09-24 14:51:35 +04:00
|
|
|
|
|
|
|
loads = lambda x: json.loads(x, parse_float=Decimal)
|
|
|
|
psycopg2.extras.register_json(conn, loads=loads)
|
|
|
|
|
|
|
|
|
|
|
|
|
2012-09-19 07:12:20 +04:00
|
|
|
.. autoclass:: Json
|
2012-09-24 14:51:35 +04:00
|
|
|
|
|
|
|
.. automethod:: dumps
|
2012-09-19 07:12:20 +04:00
|
|
|
|
2012-09-19 07:26:35 +04:00
|
|
|
.. autofunction:: register_json
|
|
|
|
|
2014-08-13 03:43:33 +04:00
|
|
|
.. versionchanged:: 2.5.4
|
|
|
|
added the *name* parameter to enable :sql:`jsonb` support.
|
|
|
|
|
2012-09-19 18:49:00 +04:00
|
|
|
.. autofunction:: register_default_json
|
|
|
|
|
2014-08-13 04:32:19 +04:00
|
|
|
.. autofunction:: register_default_jsonb
|
|
|
|
|
|
|
|
.. versionadded:: 2.5.4
|
|
|
|
|
2012-09-19 07:12:20 +04:00
|
|
|
|
|
|
|
|
2010-09-27 05:25:09 +04:00
|
|
|
.. index::
|
|
|
|
pair: hstore; Data types
|
|
|
|
pair: dict; Adaptation
|
|
|
|
|
2013-03-18 23:22:31 +04:00
|
|
|
.. _adapt-hstore:
|
|
|
|
|
2010-09-27 05:25:09 +04:00
|
|
|
Hstore data type
|
2010-11-06 04:39:43 +03:00
|
|
|
^^^^^^^^^^^^^^^^
|
2010-09-27 05:25:09 +04:00
|
|
|
|
2010-11-05 15:38:49 +03:00
|
|
|
.. versionadded:: 2.3
|
2010-09-27 05:25:09 +04:00
|
|
|
|
2010-09-28 03:33:34 +04:00
|
|
|
The |hstore|_ data type is a key-value store embedded in PostgreSQL. It has
|
|
|
|
been available for several server versions but with the release 9.0 it has
|
2013-03-06 13:36:32 +04:00
|
|
|
been greatly improved in capacity and usefulness with the addition of many
|
2010-09-28 03:33:34 +04:00
|
|
|
functions. It supports GiST or GIN indexes allowing search by keys or
|
|
|
|
key/value pairs as well as regular BTree indexes for equality, uniqueness etc.
|
2010-09-27 05:25:09 +04:00
|
|
|
|
2011-02-19 19:16:28 +03:00
|
|
|
Psycopg can convert Python `!dict` objects to and from |hstore| structures.
|
|
|
|
Only dictionaries with string/unicode keys and values are supported. `!None`
|
2011-02-25 03:19:49 +03:00
|
|
|
is also allowed as value but not as a key. Psycopg uses a more efficient |hstore|
|
2010-09-28 03:33:34 +04:00
|
|
|
representation when dealing with PostgreSQL 9.0 but previous server versions
|
2011-02-25 03:19:49 +03:00
|
|
|
are supported as well. By default the adapter/typecaster are disabled: they
|
2010-09-28 03:33:34 +04:00
|
|
|
can be enabled using the `register_hstore()` function.
|
|
|
|
|
|
|
|
.. autofunction:: register_hstore
|
2010-09-27 05:25:09 +04:00
|
|
|
|
2012-09-22 05:10:58 +04:00
|
|
|
.. versionchanged:: 2.4
|
|
|
|
added the *oid* parameter. If not specified, the typecaster is
|
|
|
|
installed also if |hstore| is not installed in the :sql:`public`
|
|
|
|
schema.
|
|
|
|
|
|
|
|
.. versionchanged:: 2.4.3
|
|
|
|
added support for |hstore| array.
|
|
|
|
|
|
|
|
|
2010-09-27 05:25:09 +04:00
|
|
|
.. |hstore| replace:: :sql:`hstore`
|
2012-02-28 20:28:07 +04:00
|
|
|
.. _hstore: http://www.postgresql.org/docs/current/static/hstore.html
|
2010-09-27 05:25:09 +04:00
|
|
|
|
|
|
|
|
|
|
|
|
2011-01-02 02:34:13 +03:00
|
|
|
.. index::
|
|
|
|
pair: Composite types; Data types
|
|
|
|
pair: tuple; Adaptation
|
|
|
|
pair: namedtuple; Adaptation
|
|
|
|
|
2013-03-18 23:22:31 +04:00
|
|
|
.. _adapt-composite:
|
|
|
|
|
2011-01-02 02:34:13 +03:00
|
|
|
Composite types casting
|
|
|
|
^^^^^^^^^^^^^^^^^^^^^^^
|
|
|
|
|
2011-02-15 14:00:08 +03:00
|
|
|
.. versionadded:: 2.4
|
2011-01-02 02:34:13 +03:00
|
|
|
|
|
|
|
Using `register_composite()` it is possible to cast a PostgreSQL composite
|
2011-12-16 00:11:17 +04:00
|
|
|
type (either created with the |CREATE TYPE|_ command or implicitly defined
|
|
|
|
after a table row type) into a Python named tuple, or into a regular tuple if
|
|
|
|
:py:func:`collections.namedtuple` is not found.
|
2011-01-02 02:34:13 +03:00
|
|
|
|
|
|
|
.. |CREATE TYPE| replace:: :sql:`CREATE TYPE`
|
2012-02-28 20:28:07 +04:00
|
|
|
.. _CREATE TYPE: http://www.postgresql.org/docs/current/static/sql-createtype.html
|
2011-01-02 02:34:13 +03:00
|
|
|
|
|
|
|
.. doctest::
|
|
|
|
|
|
|
|
>>> cur.execute("CREATE TYPE card AS (value int, suit text);")
|
|
|
|
>>> psycopg2.extras.register_composite('card', cur)
|
|
|
|
<psycopg2.extras.CompositeCaster object at 0x...>
|
|
|
|
|
|
|
|
>>> cur.execute("select (8, 'hearts')::card")
|
|
|
|
>>> cur.fetchone()[0]
|
|
|
|
card(value=8, suit='hearts')
|
|
|
|
|
2012-09-22 05:01:04 +04:00
|
|
|
Nested composite types are handled as expected, provided that the type of the
|
|
|
|
composite components are registered as well.
|
2011-01-02 02:34:13 +03:00
|
|
|
|
|
|
|
.. doctest::
|
|
|
|
|
|
|
|
>>> cur.execute("CREATE TYPE card_back AS (face card, back text);")
|
|
|
|
>>> psycopg2.extras.register_composite('card_back', cur)
|
|
|
|
<psycopg2.extras.CompositeCaster object at 0x...>
|
|
|
|
|
|
|
|
>>> cur.execute("select ((8, 'hearts'), 'blue')::card_back")
|
|
|
|
>>> cur.fetchone()[0]
|
|
|
|
card_back(face=card(value=8, suit='hearts'), back='blue')
|
|
|
|
|
|
|
|
Adaptation from Python tuples to composite types is automatic instead and
|
|
|
|
requires no adapter registration.
|
|
|
|
|
2012-09-22 05:01:04 +04:00
|
|
|
|
|
|
|
.. _custom-composite:
|
|
|
|
|
|
|
|
.. Note::
|
|
|
|
|
|
|
|
If you want to convert PostgreSQL composite types into something different
|
|
|
|
than a `!namedtuple` you can subclass the `CompositeCaster` overriding
|
|
|
|
`~CompositeCaster.make()`. For example, if you want to convert your type
|
|
|
|
into a Python dictionary you can use::
|
|
|
|
|
|
|
|
>>> class DictComposite(psycopg2.extras.CompositeCaster):
|
|
|
|
... def make(self, values):
|
|
|
|
... return dict(zip(self.attnames, values))
|
|
|
|
|
|
|
|
>>> psycopg2.extras.register_composite('card', cur,
|
|
|
|
... factory=DictComposite)
|
|
|
|
|
|
|
|
>>> cur.execute("select (8, 'hearts')::card")
|
|
|
|
>>> cur.fetchone()[0]
|
|
|
|
{'suit': 'hearts', 'value': 8}
|
|
|
|
|
|
|
|
|
2011-01-02 02:34:13 +03:00
|
|
|
.. autofunction:: register_composite
|
|
|
|
|
2012-09-22 05:08:21 +04:00
|
|
|
.. versionchanged:: 2.4.3
|
|
|
|
added support for array of composite types
|
2012-12-02 16:04:24 +04:00
|
|
|
.. versionchanged:: 2.5
|
2012-09-22 05:08:21 +04:00
|
|
|
added the *factory* parameter
|
|
|
|
|
|
|
|
|
2011-01-02 02:34:13 +03:00
|
|
|
.. autoclass:: CompositeCaster
|
|
|
|
|
2012-09-22 05:01:04 +04:00
|
|
|
.. automethod:: make
|
|
|
|
|
2012-12-02 16:04:24 +04:00
|
|
|
.. versionadded:: 2.5
|
2012-09-22 05:01:04 +04:00
|
|
|
|
|
|
|
Object attributes:
|
|
|
|
|
|
|
|
.. attribute:: name
|
|
|
|
|
|
|
|
The name of the PostgreSQL type.
|
|
|
|
|
2012-09-22 18:10:40 +04:00
|
|
|
.. attribute:: schema
|
|
|
|
|
|
|
|
The schema where the type is defined.
|
|
|
|
|
2012-12-02 16:04:24 +04:00
|
|
|
.. versionadded:: 2.5
|
2012-09-22 18:10:40 +04:00
|
|
|
|
2012-09-22 05:01:04 +04:00
|
|
|
.. attribute:: oid
|
|
|
|
|
|
|
|
The oid of the PostgreSQL type.
|
|
|
|
|
|
|
|
.. attribute:: array_oid
|
|
|
|
|
|
|
|
The oid of the PostgreSQL array type, if available.
|
|
|
|
|
|
|
|
.. attribute:: type
|
|
|
|
|
|
|
|
The type of the Python objects returned. If :py:func:`collections.namedtuple()`
|
|
|
|
is available, it is a named tuple with attributes equal to the type
|
|
|
|
components. Otherwise it is just the `!tuple` object.
|
|
|
|
|
|
|
|
.. attribute:: attnames
|
|
|
|
|
|
|
|
List of component names of the type to be casted.
|
|
|
|
|
|
|
|
.. attribute:: atttypes
|
|
|
|
|
|
|
|
List of component type oids of the type to be casted.
|
2011-01-02 02:34:13 +03:00
|
|
|
|
|
|
|
|
2012-09-24 03:49:44 +04:00
|
|
|
.. index::
|
|
|
|
pair: range; Data types
|
|
|
|
|
2013-03-18 23:22:31 +04:00
|
|
|
.. _adapt-range:
|
|
|
|
|
2012-09-24 03:49:44 +04:00
|
|
|
Range data types
|
|
|
|
^^^^^^^^^^^^^^^^
|
|
|
|
|
2012-12-02 16:04:24 +04:00
|
|
|
.. versionadded:: 2.5
|
2012-09-24 03:49:44 +04:00
|
|
|
|
|
|
|
Psycopg offers a `Range` Python type and supports adaptation between them and
|
|
|
|
PostgreSQL |range|_ types. Builtin |range| types are supported out-of-the-box;
|
|
|
|
user-defined |range| types can be adapted using `register_range()`.
|
|
|
|
|
|
|
|
.. |range| replace:: :sql:`range`
|
|
|
|
.. _range: http://www.postgresql.org/docs/current/static/rangetypes.html
|
|
|
|
|
|
|
|
.. autoclass:: Range
|
|
|
|
|
|
|
|
This Python type is only used to pass and retrieve range values to and
|
|
|
|
from PostgreSQL and doesn't attempt to replicate the PostgreSQL range
|
|
|
|
features: it doesn't perform normalization and doesn't implement all the
|
|
|
|
operators__ supported by the database.
|
|
|
|
|
|
|
|
.. __: http://www.postgresql.org/docs/current/static/functions-range.html#RANGE-OPERATORS-TABLE
|
|
|
|
|
|
|
|
`!Range` objects are immutable, hashable, and support the ``in`` operator
|
|
|
|
(checking if an element is within the range). They can be tested for
|
2014-02-23 01:56:46 +04:00
|
|
|
equivalence. Empty ranges evaluate to `!False` in boolean context,
|
|
|
|
nonempty evaluate to `!True`.
|
2014-02-19 01:24:59 +04:00
|
|
|
|
|
|
|
.. versionchanged:: 2.5.3
|
|
|
|
|
2014-02-23 01:56:46 +04:00
|
|
|
`!Range` objects can be sorted although, as on the server-side, this
|
|
|
|
ordering is not particularly meangingful. It is only meant to be used
|
|
|
|
by programs assuming objects using `!Range` as primary key can be
|
|
|
|
sorted on them. In previous versions comparing `!Range`\s raises
|
|
|
|
`!TypeError`.
|
|
|
|
|
2012-09-24 03:49:44 +04:00
|
|
|
Although it is possible to instantiate `!Range` objects, the class doesn't
|
|
|
|
have an adapter registered, so you cannot normally pass these instances as
|
|
|
|
query arguments. To use range objects as query arguments you can either
|
|
|
|
use one of the provided subclasses, such as `NumericRange` or create a
|
|
|
|
custom subclass using `register_range()`.
|
|
|
|
|
|
|
|
Object attributes:
|
|
|
|
|
|
|
|
.. autoattribute:: isempty
|
|
|
|
.. autoattribute:: lower
|
|
|
|
.. autoattribute:: upper
|
|
|
|
.. autoattribute:: lower_inc
|
|
|
|
.. autoattribute:: upper_inc
|
|
|
|
.. autoattribute:: lower_inf
|
|
|
|
.. autoattribute:: upper_inf
|
|
|
|
|
|
|
|
|
|
|
|
The following `Range` subclasses map builtin PostgreSQL |range| types to
|
|
|
|
Python objects: they have an adapter registered so their instances can be
|
|
|
|
passed as query arguments. |range| values read from database queries are
|
|
|
|
automatically casted into instances of these classes.
|
|
|
|
|
|
|
|
.. autoclass:: NumericRange
|
|
|
|
.. autoclass:: DateRange
|
|
|
|
.. autoclass:: DateTimeRange
|
|
|
|
.. autoclass:: DateTimeTZRange
|
|
|
|
|
2013-06-18 18:00:30 +04:00
|
|
|
.. note::
|
|
|
|
|
|
|
|
Python lacks a representation for :sql:`infinity` date so Psycopg converts
|
|
|
|
the value to `date.max` and such. When written into the database these
|
|
|
|
dates will assume their literal value (e.g. :sql:`9999-12-31` instead of
|
|
|
|
:sql:`infinity`). Check :ref:`infinite-dates-handling` for an example of
|
|
|
|
an alternative adapter to map `date.max` to :sql:`infinity`. An
|
|
|
|
alternative dates adapter will be used automatically by the `DateRange`
|
|
|
|
adapter and so on.
|
|
|
|
|
|
|
|
|
2012-09-24 03:49:44 +04:00
|
|
|
Custom |range| types (created with |CREATE TYPE|_ :sql:`... AS RANGE`) can be
|
|
|
|
adapted to a custom `Range` subclass:
|
|
|
|
|
|
|
|
.. autofunction:: register_range
|
|
|
|
|
|
|
|
.. autoclass:: RangeCaster
|
|
|
|
|
|
|
|
Object attributes:
|
|
|
|
|
|
|
|
.. attribute:: range
|
|
|
|
|
|
|
|
The `!Range` subclass adapted.
|
|
|
|
|
|
|
|
.. attribute:: adapter
|
|
|
|
|
|
|
|
The `~psycopg2.extensions.ISQLQuote` responsible to adapt `!range`.
|
|
|
|
|
|
|
|
.. attribute:: typecaster
|
|
|
|
|
|
|
|
The object responsible for casting.
|
|
|
|
|
|
|
|
.. attribute:: array_typecaster
|
|
|
|
|
2012-09-24 15:13:46 +04:00
|
|
|
The object responsible to cast arrays, if available, else `!None`.
|
2012-09-24 03:49:44 +04:00
|
|
|
|
|
|
|
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
.. index::
|
|
|
|
pair: UUID; Data types
|
|
|
|
|
2013-03-18 23:22:31 +04:00
|
|
|
.. _adapt-uuid:
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
UUID data type
|
2010-11-06 04:39:43 +03:00
|
|
|
^^^^^^^^^^^^^^
|
2010-02-13 05:10:51 +03:00
|
|
|
|
|
|
|
.. versionadded:: 2.0.9
|
|
|
|
.. versionchanged:: 2.0.13 added UUID array support.
|
|
|
|
|
2010-02-14 08:02:09 +03:00
|
|
|
.. doctest::
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
>>> psycopg2.extras.register_uuid()
|
2010-02-13 19:06:39 +03:00
|
|
|
<psycopg2._psycopg.type object at 0x...>
|
2010-02-14 08:02:09 +03:00
|
|
|
|
2010-02-13 19:06:39 +03:00
|
|
|
>>> # Python UUID can be used in SQL queries
|
|
|
|
>>> import uuid
|
2010-02-14 08:02:09 +03:00
|
|
|
>>> my_uuid = uuid.UUID('{12345678-1234-5678-1234-567812345678}')
|
|
|
|
>>> psycopg2.extensions.adapt(my_uuid).getquoted()
|
|
|
|
"'12345678-1234-5678-1234-567812345678'::uuid"
|
|
|
|
|
2010-02-13 19:06:39 +03:00
|
|
|
>>> # PostgreSQL UUID are transformed into Python UUID objects.
|
2010-02-13 05:10:51 +03:00
|
|
|
>>> cur.execute("SELECT 'a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11'::uuid")
|
|
|
|
>>> cur.fetchone()[0]
|
|
|
|
UUID('a0eebc99-9c0b-4ef8-bb6d-6bb9bd380a11')
|
|
|
|
|
|
|
|
|
|
|
|
.. autofunction:: register_uuid
|
|
|
|
|
|
|
|
.. autoclass:: UUID_adapter
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. index::
|
|
|
|
pair: INET; Data types
|
|
|
|
|
2010-02-14 23:32:35 +03:00
|
|
|
:sql:`inet` data type
|
2010-11-06 04:39:43 +03:00
|
|
|
^^^^^^^^^^^^^^^^^^^^^^
|
2010-02-14 23:32:35 +03:00
|
|
|
|
|
|
|
.. versionadded:: 2.0.9
|
2012-02-24 03:56:55 +04:00
|
|
|
.. versionchanged:: 2.4.5 added inet array support.
|
2010-02-14 23:32:35 +03:00
|
|
|
|
|
|
|
.. doctest::
|
2010-02-13 05:10:51 +03:00
|
|
|
|
|
|
|
>>> psycopg2.extras.register_inet()
|
2010-02-13 19:06:39 +03:00
|
|
|
<psycopg2._psycopg.type object at 0x...>
|
2010-02-14 23:32:35 +03:00
|
|
|
|
|
|
|
>>> cur.mogrify("SELECT %s", (Inet('127.0.0.1/32'),))
|
|
|
|
"SELECT E'127.0.0.1/32'::inet"
|
|
|
|
|
|
|
|
>>> cur.execute("SELECT '192.168.0.1/24'::inet")
|
2010-02-13 05:10:51 +03:00
|
|
|
>>> cur.fetchone()[0].addr
|
2010-02-14 23:32:35 +03:00
|
|
|
'192.168.0.1/24'
|
|
|
|
|
2010-02-13 05:10:51 +03:00
|
|
|
|
2012-02-24 03:56:55 +04:00
|
|
|
.. autofunction:: register_inet
|
2010-02-13 05:10:51 +03:00
|
|
|
|
|
|
|
.. autoclass:: Inet
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
.. index::
|
|
|
|
single: Time zones; Fractional
|
|
|
|
|
|
|
|
Fractional time zones
|
|
|
|
---------------------
|
|
|
|
|
|
|
|
.. autofunction:: register_tstz_w_secs
|
|
|
|
|
2010-02-13 08:49:34 +03:00
|
|
|
.. versionadded:: 2.0.9
|
|
|
|
|
2010-11-01 02:57:40 +03:00
|
|
|
.. versionchanged:: 2.2.2
|
2010-05-20 05:10:33 +04:00
|
|
|
function is no-op: see :ref:`tz-handling`.
|
|
|
|
|
2010-04-04 06:10:18 +04:00
|
|
|
.. index::
|
|
|
|
pair: Example; Coroutine;
|
|
|
|
|
2010-11-06 04:39:43 +03:00
|
|
|
|
|
|
|
|
2010-04-04 06:10:18 +04:00
|
|
|
Coroutine support
|
|
|
|
-----------------
|
|
|
|
|
|
|
|
.. autofunction:: wait_select(conn)
|
|
|
|
|
2015-10-01 17:26:13 +03:00
|
|
|
.. versionchanged:: 2.6.2
|
|
|
|
allow to cancel a query using :kbd:`Ctrl-C`, see
|
|
|
|
:ref:`the FAQ <faq-interrupt-query>` for an example.
|