mirror of
https://github.com/psycopg/psycopg2.git
synced 2024-11-22 08:56:34 +03:00
First iteration with the Sphinx module documentation.
This commit is contained in:
parent
442d8693cd
commit
961b4eedc5
1
.gitignore
vendored
1
.gitignore
vendored
|
@ -6,3 +6,4 @@ MANIFEST
|
||||||
*.sw[po]
|
*.sw[po]
|
||||||
dist/*
|
dist/*
|
||||||
build/*
|
build/*
|
||||||
|
doc/_build/*
|
||||||
|
|
91
doc/Makefile
Normal file
91
doc/Makefile
Normal file
|
@ -0,0 +1,91 @@
|
||||||
|
# Makefile for Sphinx documentation
|
||||||
|
#
|
||||||
|
|
||||||
|
# You can set these variables from the command line.
|
||||||
|
SPHINXOPTS =
|
||||||
|
SPHINXBUILD = sphinx-build
|
||||||
|
PAPER =
|
||||||
|
BUILDDIR = _build
|
||||||
|
|
||||||
|
# Internal variables.
|
||||||
|
PAPEROPT_a4 = -D latex_paper_size=a4
|
||||||
|
PAPEROPT_letter = -D latex_paper_size=letter
|
||||||
|
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
|
||||||
|
|
||||||
|
.PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
|
||||||
|
|
||||||
|
doc: html
|
||||||
|
|
||||||
|
help:
|
||||||
|
@echo "Please use \`make <target>' where <target> is one of"
|
||||||
|
@echo " html to make standalone HTML files"
|
||||||
|
@echo " dirhtml to make HTML files named index.html in directories"
|
||||||
|
@echo " pickle to make pickle files"
|
||||||
|
@echo " json to make JSON files"
|
||||||
|
@echo " htmlhelp to make HTML files and a HTML help project"
|
||||||
|
@echo " qthelp to make HTML files and a qthelp project"
|
||||||
|
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
|
||||||
|
@echo " changes to make an overview of all changed/added/deprecated items"
|
||||||
|
@echo " linkcheck to check all external links for integrity"
|
||||||
|
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
|
||||||
|
|
||||||
|
clean:
|
||||||
|
-rm -rf $(BUILDDIR)/*
|
||||||
|
|
||||||
|
html:
|
||||||
|
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
|
||||||
|
@echo
|
||||||
|
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
|
||||||
|
|
||||||
|
dirhtml:
|
||||||
|
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
|
||||||
|
@echo
|
||||||
|
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
|
||||||
|
|
||||||
|
pickle:
|
||||||
|
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can process the pickle files."
|
||||||
|
|
||||||
|
json:
|
||||||
|
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can process the JSON files."
|
||||||
|
|
||||||
|
htmlhelp:
|
||||||
|
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can run HTML Help Workshop with the" \
|
||||||
|
".hhp project file in $(BUILDDIR)/htmlhelp."
|
||||||
|
|
||||||
|
qthelp:
|
||||||
|
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
|
||||||
|
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
|
||||||
|
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/psycopg.qhcp"
|
||||||
|
@echo "To view the help file:"
|
||||||
|
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/psycopg.qhc"
|
||||||
|
|
||||||
|
latex:
|
||||||
|
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
|
||||||
|
@echo
|
||||||
|
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
|
||||||
|
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
|
||||||
|
"run these through (pdf)latex."
|
||||||
|
|
||||||
|
changes:
|
||||||
|
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
|
||||||
|
@echo
|
||||||
|
@echo "The overview file is in $(BUILDDIR)/changes."
|
||||||
|
|
||||||
|
linkcheck:
|
||||||
|
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
|
||||||
|
@echo
|
||||||
|
@echo "Link check complete; look for any errors in the above output " \
|
||||||
|
"or in $(BUILDDIR)/linkcheck/output.txt."
|
||||||
|
|
||||||
|
doctest:
|
||||||
|
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
|
||||||
|
@echo "Testing of doctests in the sources finished, look at the " \
|
||||||
|
"results in $(BUILDDIR)/doctest/output.txt."
|
243
doc/advanced.rst
Normal file
243
doc/advanced.rst
Normal file
|
@ -0,0 +1,243 @@
|
||||||
|
More advanced topics
|
||||||
|
====================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
double: Subclassing; Cursor
|
||||||
|
double: Subclassing; Connection
|
||||||
|
|
||||||
|
.. _subclassing-connection:
|
||||||
|
.. _subclassing-cursor:
|
||||||
|
|
||||||
|
Connection and cursor factories
|
||||||
|
-------------------------------
|
||||||
|
|
||||||
|
Psycopg exposes two new-style classes that can be sub-classed and expanded to
|
||||||
|
adapt them to the needs of the programmer: :class:`psycopg2.extensions.cursor`
|
||||||
|
and :class:`psycopg2.extensions.connection`. The :class:`connection` class is
|
||||||
|
usually sub-classed only to provide an easy way to create customized cursors
|
||||||
|
but other uses are possible. :class:`cursor` is much more interesting, because
|
||||||
|
it is the class where query building, execution and result type-casting into
|
||||||
|
Python variables happens.
|
||||||
|
|
||||||
|
An example of cursor subclass performing logging is::
|
||||||
|
|
||||||
|
import psycopg2
|
||||||
|
import psycopg2.extensions
|
||||||
|
import logging
|
||||||
|
|
||||||
|
class LoggingCursor(psycopg2.extensions.cursor):
|
||||||
|
def execute(self, sql, args=None):
|
||||||
|
logger = logging.getLogger('sql_debug')
|
||||||
|
logger.info(self.mogrify(sql, args))
|
||||||
|
|
||||||
|
try:
|
||||||
|
psycopg2.extensions.cursor.execute(self, sql, args)
|
||||||
|
except Exception, exc:
|
||||||
|
logger.error("%s: %s" % (exc.__class__.__name__, exc))
|
||||||
|
raise
|
||||||
|
|
||||||
|
conn = psycopg2.connect(DSN)
|
||||||
|
curs = conn.cursor(cursor_factory=LoggingCursor)
|
||||||
|
curs.execute("INSERT INTO mytable VALUES (%s, %s, %s);",
|
||||||
|
(10, 20, 30))
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Objects; Creating new adapters
|
||||||
|
single: Adaptation; Creating new adapters
|
||||||
|
single: Data types; Creating new adapters
|
||||||
|
|
||||||
|
.. _adapting-new-types:
|
||||||
|
|
||||||
|
Adapting new Python types to SQL syntax
|
||||||
|
---------------------------------------
|
||||||
|
|
||||||
|
Any Python class or type can be adapted to an SQL string. Adaptation mechanism
|
||||||
|
is similar to the Object Adaptation proposed in the :pep:`246` and is exposed
|
||||||
|
by the :func:`psycopg2.extensions.adapt()` function.
|
||||||
|
|
||||||
|
The :meth:`cursor.execute()` method adapts its arguments to the
|
||||||
|
:class:`psycopg2.extensions.ISQLQuote` protocol. Objects that conform to this
|
||||||
|
protocol expose a ``getquoted()`` method returning the SQL representation of
|
||||||
|
the object as a string.
|
||||||
|
|
||||||
|
The easiest way to adapt an object to an SQL string is to register an adapter
|
||||||
|
function via the :func:`psycopg2.extensions.register_adapter()` function. The
|
||||||
|
adapter function must take the value to be adapted as argument and return a
|
||||||
|
conform object. A convenient object is the :func:`psycopg2.extensions.AsIs`
|
||||||
|
wrapper, whose ``getquoted()`` result is simply the ``str()``\ ingification of
|
||||||
|
the wrapped object.
|
||||||
|
|
||||||
|
Example: mapping of a ``Point`` class into the ``point`` PostgreSQL geometric
|
||||||
|
type::
|
||||||
|
|
||||||
|
from psycopg2.extensions import adapt, register_adapter, AsIs
|
||||||
|
|
||||||
|
class Point(object):
|
||||||
|
def __init__(self, x, y):
|
||||||
|
self.x = x
|
||||||
|
self.y = y
|
||||||
|
|
||||||
|
def adapt_point(point):
|
||||||
|
return AsIs("'(%s, %s)'" % (adapt(point.x), adapt(point.y)))
|
||||||
|
|
||||||
|
register_adapter(Point, adapt_point)
|
||||||
|
|
||||||
|
curs.execute("INSERT INTO atable (apoint) VALUES (%s)",
|
||||||
|
(Point(1.23, 4.56),))
|
||||||
|
|
||||||
|
The above function call results in the SQL command::
|
||||||
|
|
||||||
|
INSERT INTO atable (apoint) VALUES ((1.23, 4.56));
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index:: Type casting
|
||||||
|
|
||||||
|
.. _type-casting-from-sql-to-python:
|
||||||
|
|
||||||
|
Type casting of SQL types into Python values
|
||||||
|
--------------------------------------------
|
||||||
|
|
||||||
|
PostgreSQL objects read from the database can be adapted to Python objects
|
||||||
|
through an user-defined adapting function. An adapter function takes two
|
||||||
|
argments: the object string representation as returned by PostgreSQL and the
|
||||||
|
cursor currently being read, and should return a new Python object. For
|
||||||
|
example, the following function parses a PostgreSQL ``point`` into the
|
||||||
|
previously defined ``Point`` class::
|
||||||
|
|
||||||
|
def cast_point(value, curs):
|
||||||
|
if value is not None:
|
||||||
|
# Convert from (f1, f2) syntax using a regular expression.
|
||||||
|
m = re.match("\((.*),(.*)\)", value)
|
||||||
|
if m:
|
||||||
|
return Point(float(m.group(1)), float(m.group(2)))
|
||||||
|
|
||||||
|
To create a mapping from the PostgreSQL type (either standard or user-defined),
|
||||||
|
its ``oid`` must be known. It can be retrieved either by the second column of
|
||||||
|
the cursor description::
|
||||||
|
|
||||||
|
curs.execute("SELECT NULL::point")
|
||||||
|
point_oid = curs.description[0][1] # usually returns 600
|
||||||
|
|
||||||
|
or by querying the system catalogs for the type name and namespace (the
|
||||||
|
namespace for system objects is ``pg_catalog``)::
|
||||||
|
|
||||||
|
curs.execute("""
|
||||||
|
SELECT pg_type.oid
|
||||||
|
FROM pg_type JOIN pg_namespace
|
||||||
|
ON typnamespace = pg_namespace.oid
|
||||||
|
WHERE typname = %(typename)s
|
||||||
|
AND nspname = %(namespace)s""",
|
||||||
|
{'typename': 'point', 'namespace': 'pg_catalog'})
|
||||||
|
|
||||||
|
point_oid = curs.fetchone()[0]
|
||||||
|
|
||||||
|
After you know the object ``oid``, you must can and register the new type::
|
||||||
|
|
||||||
|
POINT = psycopg2.extensions.new_type((point_oid,), "POINT", cast_point)
|
||||||
|
psycopg2.extensions.register_type(POINT)
|
||||||
|
|
||||||
|
The :func:`psycopg2.extensions.new_type()` function binds the object oids
|
||||||
|
(more than one can be specified) to the adapter function.
|
||||||
|
:func:`psycopg2.extensions.register_type()` completes the spell. Conversion
|
||||||
|
is automatically performed when a column whose type is a registered ``oid`` is
|
||||||
|
read::
|
||||||
|
|
||||||
|
>>> curs.execute("SELECT '(10.2,20.3)'::point")
|
||||||
|
>>> point = curs.fetchone()[0]
|
||||||
|
>>> print type(point), point.x, point.y
|
||||||
|
<class '__main__.Point'> 10.2 20.3
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
double: Asynchronous; Query
|
||||||
|
|
||||||
|
.. _asynchronous-queries:
|
||||||
|
|
||||||
|
Asynchronous queries
|
||||||
|
--------------------
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
Async quaeries are not enabled for 2.0
|
||||||
|
|
||||||
|
.. todo::
|
||||||
|
|
||||||
|
I think this is false now: async queries seem working right now...
|
||||||
|
|
||||||
|
Program code can initiate an asynchronous query by passing an ``async=1`` flag
|
||||||
|
to the :meth:`cursor.execute` or :meth:`cursor.callproc` methods. A very
|
||||||
|
simple example, from the connection to the query::
|
||||||
|
|
||||||
|
conn = psycopg.connect(database='test')
|
||||||
|
curs = conn.cursor()
|
||||||
|
curs.execute("SELECT * from test WHERE fielda > %s", (1971,), async=1)
|
||||||
|
|
||||||
|
From then on any query on other cursors derived from the same connection is
|
||||||
|
doomed to fail (and raise an exception) until the original cursor (the one
|
||||||
|
executing the query) complete the asynchronous operation. This can happen in
|
||||||
|
a number of different ways:
|
||||||
|
|
||||||
|
1) one of the :obj:`.fetch*()` methods is called, effectively blocking untill
|
||||||
|
data has been sent from the backend to the client, terminating the query.
|
||||||
|
|
||||||
|
2) :meth:`connection.cancel` is called. This method tries to abort the
|
||||||
|
current query and will block until the query is aborted or fully executed.
|
||||||
|
The return value is ``True`` if the query was successfully aborted or
|
||||||
|
``False`` if it was executed. Query result are discarded in both cases.
|
||||||
|
|
||||||
|
.. todo::
|
||||||
|
|
||||||
|
Can't see any ``connection.cancel`` method.
|
||||||
|
|
||||||
|
3) :meth:`cursor.execute` is called again on the same cursor
|
||||||
|
(:obj:`.execute()` on a different cursor will simply raise an exception).
|
||||||
|
This waits for the complete execution of the current query, discard any
|
||||||
|
data and execute the new one.
|
||||||
|
|
||||||
|
Note that calling :obj:`.execute()` two times in a row will not abort the
|
||||||
|
former query and will temporarily go to synchronous mode until the first of
|
||||||
|
the two queries is executed.
|
||||||
|
|
||||||
|
Cursors now have some extra methods that make them usefull during
|
||||||
|
asynchronous queries:
|
||||||
|
|
||||||
|
:meth:`cursor.fileno`
|
||||||
|
Returns the file descriptor associated with the current connection and
|
||||||
|
make possible to use a cursor in a context where a file object would be
|
||||||
|
expected (like in a :func:`select()` call).
|
||||||
|
|
||||||
|
:meth:`cursor.isready`
|
||||||
|
Returns ``False`` if the backend is still processing the query or ``True``
|
||||||
|
if data is ready to be fetched (by one of the :obj:`.fetch*()` methods).
|
||||||
|
|
||||||
|
A code snippet that shows how to use the cursor object in a :func:`select()`
|
||||||
|
call::
|
||||||
|
|
||||||
|
import psycopg
|
||||||
|
import select
|
||||||
|
|
||||||
|
conn = psycopg.connect(database='test')
|
||||||
|
curs = conn.cursor()
|
||||||
|
curs.execute("SELECT * from test WHERE fielda > %s", (1971,), async=1)
|
||||||
|
|
||||||
|
# wait for input with a maximum timeout of 5 seconds
|
||||||
|
query_ended = False
|
||||||
|
while not query_ended:
|
||||||
|
rread, rwrite, rspec = select([curs, another_file], [], [], 5)
|
||||||
|
|
||||||
|
if curs.isready():
|
||||||
|
query_ended = True
|
||||||
|
|
||||||
|
# manage input from other sources like other_file, etc.
|
||||||
|
|
||||||
|
print "Query Results:"
|
||||||
|
for row in curs:
|
||||||
|
print row
|
||||||
|
|
||||||
|
|
|
@ -1,67 +0,0 @@
|
||||||
psycopg asynchronous API
|
|
||||||
************************
|
|
||||||
|
|
||||||
** Important: async quaeries are not enabled for 2.0 **
|
|
||||||
|
|
||||||
Program code can initiate an asynchronous query by passing an 'async=1' flag
|
|
||||||
to the .execute() method. A very simple example, from the connection to the
|
|
||||||
query:
|
|
||||||
|
|
||||||
conn = psycopg.connect(database='test')
|
|
||||||
curs = conn.cursor()
|
|
||||||
curs.execute("SEECT * from test WHERE fielda > %s", (1971,), async=1)
|
|
||||||
|
|
||||||
From then on any query on other cursors derived from the same connection is
|
|
||||||
doomed to fail (and raise an exception) until the original cursor (the one
|
|
||||||
executing the query) complete the asynchronous operation. This can happen in
|
|
||||||
a number of different ways:
|
|
||||||
|
|
||||||
1) one of the .fetchXXX() methods is called, effectively blocking untill
|
|
||||||
data has been sent from the backend to the client, terminating the
|
|
||||||
query.
|
|
||||||
|
|
||||||
2) .cancel() is called. This method tries to abort the current query and
|
|
||||||
will block until the query is aborted or fully executed. The return
|
|
||||||
value is True if the query was successfully aborted or False if it
|
|
||||||
was executed. Query result are discarded in both cases.
|
|
||||||
|
|
||||||
3) .execute() is called again on the same cursor (.execute() on a
|
|
||||||
different cursor will simply raise an exception.) This waits for the
|
|
||||||
complete execution of the current query, discard any data and execute
|
|
||||||
the new one.
|
|
||||||
|
|
||||||
Note that calling .execute() two times in a row will not abort the former
|
|
||||||
query and will temporarily go to synchronous mode until the first of the two
|
|
||||||
queries is executed.
|
|
||||||
|
|
||||||
Cursors now have some extra methods that make them usefull during
|
|
||||||
asynchronous queries:
|
|
||||||
|
|
||||||
.fileno()
|
|
||||||
Returns the file descriptor associated with the current connection and
|
|
||||||
make possible to use a cursor in a context where a file object would be
|
|
||||||
expected (like in a select() call.)
|
|
||||||
|
|
||||||
.isbusy()
|
|
||||||
Returns True if the backend is still processing the query or false if
|
|
||||||
data is ready to be fetched (by one of the .fetchXXX() methods.)
|
|
||||||
|
|
||||||
A code snippet that shows how to use the cursor object in a select() call:
|
|
||||||
|
|
||||||
import psycopg
|
|
||||||
import select
|
|
||||||
|
|
||||||
conn = psycopg.connect(database='test')
|
|
||||||
curs = conn.cursor()
|
|
||||||
curs.execute("SEECT * from test WHERE fielda > %s", (1971,), async=1)
|
|
||||||
|
|
||||||
# wait for input with a maximum timeout of 5 seconds
|
|
||||||
query_ended = False
|
|
||||||
while not query_ended:
|
|
||||||
rread, rwrite, rspec = select([cursor, another_file], [], [], 5)
|
|
||||||
if not cursor.isbusy():
|
|
||||||
query_ended = True
|
|
||||||
# manage input from other sources like other_file, etc.
|
|
||||||
print "Query Results:"
|
|
||||||
for row in cursor:
|
|
||||||
print row
|
|
210
doc/conf.py
Normal file
210
doc/conf.py
Normal file
|
@ -0,0 +1,210 @@
|
||||||
|
# -*- coding: utf-8 -*-
|
||||||
|
#
|
||||||
|
# Psycopg documentation build configuration file, created by
|
||||||
|
# sphinx-quickstart on Sun Feb 7 13:48:41 2010.
|
||||||
|
#
|
||||||
|
# This file is execfile()d with the current directory set to its containing dir.
|
||||||
|
#
|
||||||
|
# Note that not all possible configuration values are present in this
|
||||||
|
# autogenerated file.
|
||||||
|
#
|
||||||
|
# All configuration values have a default; values that are commented out
|
||||||
|
# serve to show the default.
|
||||||
|
|
||||||
|
import sys, os
|
||||||
|
|
||||||
|
# If extensions (or modules to document with autodoc) are in another directory,
|
||||||
|
# add these directories to sys.path here. If the directory is relative to the
|
||||||
|
# documentation root, use os.path.abspath to make it absolute, like shown here.
|
||||||
|
#sys.path.append(os.path.abspath('.'))
|
||||||
|
|
||||||
|
# -- General configuration -----------------------------------------------------
|
||||||
|
|
||||||
|
# Add any Sphinx extension module names here, as strings. They can be extensions
|
||||||
|
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
|
||||||
|
extensions = ['sphinx.ext.autodoc', 'sphinx.ext.todo', 'sphinx.ext.ifconfig']
|
||||||
|
|
||||||
|
# Add any paths that contain templates here, relative to this directory.
|
||||||
|
templates_path = ['_templates']
|
||||||
|
|
||||||
|
# The suffix of source filenames.
|
||||||
|
source_suffix = '.rst'
|
||||||
|
|
||||||
|
# The encoding of source files.
|
||||||
|
#source_encoding = 'utf-8'
|
||||||
|
|
||||||
|
# The master toctree document.
|
||||||
|
master_doc = 'index'
|
||||||
|
|
||||||
|
# General information about the project.
|
||||||
|
project = u'Psycopg'
|
||||||
|
copyright = u'2001-2010, Federico Di Gregorio. Documentation by Daniele Varrazzo'
|
||||||
|
|
||||||
|
# The version info for the project you're documenting, acts as replacement for
|
||||||
|
# |version| and |release|, also used in various other places throughout the
|
||||||
|
# built documents.
|
||||||
|
#
|
||||||
|
# The short X.Y version.
|
||||||
|
version = '2.0'
|
||||||
|
# The full version, including alpha/beta/rc tags.
|
||||||
|
release = '2.0.11'
|
||||||
|
|
||||||
|
# The language for content autogenerated by Sphinx. Refer to documentation
|
||||||
|
# for a list of supported languages.
|
||||||
|
#language = None
|
||||||
|
|
||||||
|
# There are two options for replacing |today|: either, you set today to some
|
||||||
|
# non-false value, then it is used:
|
||||||
|
#today = ''
|
||||||
|
# Else, today_fmt is used as the format for a strftime call.
|
||||||
|
#today_fmt = '%B %d, %Y'
|
||||||
|
|
||||||
|
# List of documents that shouldn't be included in the build.
|
||||||
|
#unused_docs = []
|
||||||
|
|
||||||
|
# List of directories, relative to source directory, that shouldn't be searched
|
||||||
|
# for source files.
|
||||||
|
exclude_trees = ['_build']
|
||||||
|
|
||||||
|
# The reST default role (used for this markup: `text`) to use for all documents.
|
||||||
|
#default_role = None
|
||||||
|
|
||||||
|
# If true, '()' will be appended to :func: etc. cross-reference text.
|
||||||
|
#add_function_parentheses = True
|
||||||
|
|
||||||
|
# If true, the current module name will be prepended to all description
|
||||||
|
# unit titles (such as .. function::).
|
||||||
|
#add_module_names = True
|
||||||
|
|
||||||
|
# If true, sectionauthor and moduleauthor directives will be shown in the
|
||||||
|
# output. They are ignored by default.
|
||||||
|
#show_authors = False
|
||||||
|
|
||||||
|
# The name of the Pygments (syntax highlighting) style to use.
|
||||||
|
pygments_style = 'sphinx'
|
||||||
|
|
||||||
|
# A list of ignored prefixes for module index sorting.
|
||||||
|
#modindex_common_prefix = []
|
||||||
|
|
||||||
|
# Include TODO items in the documentation
|
||||||
|
todo_include_todos = True
|
||||||
|
|
||||||
|
rst_epilog = """
|
||||||
|
.. |DBAPI 2.0| replace:: `DBAPI 2.0`_
|
||||||
|
|
||||||
|
.. _DBAPI 2.0: http://www.python.org/dev/peps/pep-0249/
|
||||||
|
|
||||||
|
.. _transaction isolation level:
|
||||||
|
http://www.postgresql.org/docs/8.4/static/transaction-iso.html
|
||||||
|
|
||||||
|
.. _serializable isolation level:
|
||||||
|
http://www.postgresql.org/docs/8.4/static/transaction-iso.html#XACT-SERIALIZABLE
|
||||||
|
|
||||||
|
.. _mx.DateTime: http://www.egenix.com/products/python/mxBase/mxDateTime/
|
||||||
|
"""
|
||||||
|
|
||||||
|
# -- Options for HTML output ---------------------------------------------------
|
||||||
|
|
||||||
|
# The theme to use for HTML and HTML Help pages. Major themes that come with
|
||||||
|
# Sphinx are currently 'default' and 'sphinxdoc'.
|
||||||
|
html_theme = 'default'
|
||||||
|
|
||||||
|
# Theme options are theme-specific and customize the look and feel of a theme
|
||||||
|
# further. For a list of options available for each theme, see the
|
||||||
|
# documentation.
|
||||||
|
#html_theme_options = {}
|
||||||
|
|
||||||
|
# Add any paths that contain custom themes here, relative to this directory.
|
||||||
|
#html_theme_path = []
|
||||||
|
|
||||||
|
# The name for this set of Sphinx documents. If None, it defaults to
|
||||||
|
# "<project> v<release> documentation".
|
||||||
|
#html_title = None
|
||||||
|
|
||||||
|
# A shorter title for the navigation bar. Default is the same as html_title.
|
||||||
|
#html_short_title = None
|
||||||
|
|
||||||
|
# The name of an image file (relative to this directory) to place at the top
|
||||||
|
# of the sidebar.
|
||||||
|
#html_logo = None
|
||||||
|
|
||||||
|
# The name of an image file (within the static path) to use as favicon of the
|
||||||
|
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
|
||||||
|
# pixels large.
|
||||||
|
#html_favicon = None
|
||||||
|
|
||||||
|
# Add any paths that contain custom static files (such as style sheets) here,
|
||||||
|
# relative to this directory. They are copied after the builtin static files,
|
||||||
|
# so a file named "default.css" will overwrite the builtin "default.css".
|
||||||
|
html_static_path = ['_static']
|
||||||
|
|
||||||
|
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
|
||||||
|
# using the given strftime format.
|
||||||
|
#html_last_updated_fmt = '%b %d, %Y'
|
||||||
|
|
||||||
|
# If true, SmartyPants will be used to convert quotes and dashes to
|
||||||
|
# typographically correct entities.
|
||||||
|
#html_use_smartypants = True
|
||||||
|
|
||||||
|
# Custom sidebar templates, maps document names to template names.
|
||||||
|
#html_sidebars = {}
|
||||||
|
|
||||||
|
# Additional templates that should be rendered to pages, maps page names to
|
||||||
|
# template names.
|
||||||
|
#html_additional_pages = {}
|
||||||
|
|
||||||
|
# If false, no module index is generated.
|
||||||
|
#html_use_modindex = True
|
||||||
|
|
||||||
|
# If false, no index is generated.
|
||||||
|
#html_use_index = True
|
||||||
|
|
||||||
|
# If true, the index is split into individual pages for each letter.
|
||||||
|
#html_split_index = False
|
||||||
|
|
||||||
|
# If true, links to the reST sources are added to the pages.
|
||||||
|
#html_show_sourcelink = True
|
||||||
|
|
||||||
|
# If true, an OpenSearch description file will be output, and all pages will
|
||||||
|
# contain a <link> tag referring to it. The value of this option must be the
|
||||||
|
# base URL from which the finished HTML is served.
|
||||||
|
#html_use_opensearch = ''
|
||||||
|
|
||||||
|
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
|
||||||
|
#html_file_suffix = ''
|
||||||
|
|
||||||
|
# Output file base name for HTML help builder.
|
||||||
|
htmlhelp_basename = 'psycopgdoc'
|
||||||
|
|
||||||
|
|
||||||
|
# -- Options for LaTeX output --------------------------------------------------
|
||||||
|
|
||||||
|
# The paper size ('letter' or 'a4').
|
||||||
|
#latex_paper_size = 'letter'
|
||||||
|
|
||||||
|
# The font size ('10pt', '11pt' or '12pt').
|
||||||
|
#latex_font_size = '10pt'
|
||||||
|
|
||||||
|
# Grouping the document tree into LaTeX files. List of tuples
|
||||||
|
# (source start file, target name, title, author, documentclass [howto/manual]).
|
||||||
|
latex_documents = [
|
||||||
|
('index', 'psycopg.tex', u'Psycopg Documentation',
|
||||||
|
u'Federico Di Gregorio', 'manual'),
|
||||||
|
]
|
||||||
|
|
||||||
|
# The name of an image file (relative to this directory) to place at the top of
|
||||||
|
# the title page.
|
||||||
|
#latex_logo = None
|
||||||
|
|
||||||
|
# For "manual" documents, if this is true, then toplevel headings are parts,
|
||||||
|
# not chapters.
|
||||||
|
#latex_use_parts = False
|
||||||
|
|
||||||
|
# Additional stuff for the LaTeX preamble.
|
||||||
|
#latex_preamble = ''
|
||||||
|
|
||||||
|
# Documents to append as an appendix to all manuals.
|
||||||
|
#latex_appendices = []
|
||||||
|
|
||||||
|
# If false, no module index is generated.
|
||||||
|
#latex_use_modindex = True
|
192
doc/connection.rst
Normal file
192
doc/connection.rst
Normal file
|
@ -0,0 +1,192 @@
|
||||||
|
The ``connection`` class
|
||||||
|
========================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. class:: connection
|
||||||
|
|
||||||
|
Handles the connection to a PostgreSQL database instance. It encapsulates
|
||||||
|
a database session.
|
||||||
|
|
||||||
|
Connections are created using the factory function
|
||||||
|
:func:`psycopg2.connect()`.
|
||||||
|
|
||||||
|
Connections are thread safe and can be shared among many thread. See
|
||||||
|
:ref:`thread-safety` for details.
|
||||||
|
|
||||||
|
.. method:: cursor([name] [, cursor_factory])
|
||||||
|
|
||||||
|
Return a new :class:`cursor` object using the connection.
|
||||||
|
|
||||||
|
If :obj:`name` is specified, the returned cursor will be a *server
|
||||||
|
side* (or *named*) cursor. Otherwise the cursor will be *client side*.
|
||||||
|
See :ref:`server-side-cursors` for further details.
|
||||||
|
|
||||||
|
The ``cursor_factory`` argument can be used to create non-standard
|
||||||
|
cursors. The class returned should be a subclass of
|
||||||
|
:class:`extensions.cursor`. See :ref:`subclassing-cursor` for details.
|
||||||
|
|
||||||
|
.. method:: commit()
|
||||||
|
|
||||||
|
Commit any pending transaction to the database. Psycopg can be set to
|
||||||
|
perform automatic commits at each operation, see
|
||||||
|
:meth:`connection.set_isolation_level()`.
|
||||||
|
|
||||||
|
.. method:: rollback()
|
||||||
|
|
||||||
|
Roll back to the start of any pending transaction. Closing a
|
||||||
|
connection without committing the changes first will cause an implicit
|
||||||
|
rollback to be performed.
|
||||||
|
|
||||||
|
.. method:: close()
|
||||||
|
|
||||||
|
Close the connection now (rather than whenever ``__del__`` is called).
|
||||||
|
The connection will be unusable from this point forward; a
|
||||||
|
:exc:`psycopg2.Error` (or subclass) exception will be raised if any
|
||||||
|
operation is attempted with the connection. The same applies to all
|
||||||
|
cursor objects trying to use the connection. Note that closing a
|
||||||
|
connection without committing the changes first will cause an implicit
|
||||||
|
rollback to be performed (unless a different isolation level has been
|
||||||
|
selected: see :meth:`connection.set_isolation_level()`).
|
||||||
|
|
||||||
|
The above methods are the only ones defined by the |DBAPI 2.0|_ protocol.
|
||||||
|
The Psycopg connection objects exports the following additional methods
|
||||||
|
and attributes.
|
||||||
|
|
||||||
|
.. attribute:: closed
|
||||||
|
|
||||||
|
Read-only attribute reporting whether the database connection is open
|
||||||
|
(0) or closed (1).
|
||||||
|
|
||||||
|
.. attribute:: dsn
|
||||||
|
|
||||||
|
Read-only string containing the connection string used by the
|
||||||
|
connection.
|
||||||
|
|
||||||
|
.. attribute:: isolation_level
|
||||||
|
.. method:: set_isolation_level(level)
|
||||||
|
|
||||||
|
Read or set the `transaction isolation level`_ for the current session.
|
||||||
|
The level defines the different phenomena that can happen in the
|
||||||
|
database between concurrent transactions.
|
||||||
|
|
||||||
|
The value set or read is an integer: symbolic constants are defined in
|
||||||
|
the module :mod:`psycopg2.extensions`: see
|
||||||
|
:ref:`isolation-level-constants` for the available values.
|
||||||
|
|
||||||
|
The default level is ``READ COMMITTED``: in this level a transaction
|
||||||
|
is automatically started every time a database command is executed. If
|
||||||
|
you want an *autocommit* mode, set the connection in ``AUTOCOMMIT``
|
||||||
|
mode before executing any command::
|
||||||
|
|
||||||
|
>>> conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
|
||||||
|
|
||||||
|
.. attribute:: encoding
|
||||||
|
.. method:: set_client_encoding(enc)
|
||||||
|
|
||||||
|
Read or set the client encoding for the current session. The default
|
||||||
|
is the encoding defined by the database. It should be one of the
|
||||||
|
`characters set supported by PostgreSQL`__
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/multibyte.html
|
||||||
|
|
||||||
|
|
||||||
|
.. method:: get_backend_pid()
|
||||||
|
|
||||||
|
Returns the process ID (PID) of the backend server process handling
|
||||||
|
this connection.
|
||||||
|
|
||||||
|
Note that the PID belongs to a process executing on the database
|
||||||
|
server host, not the local host!
|
||||||
|
|
||||||
|
.. seealso:: libpq docs for `PQbackendPID()`__ for details.
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-status.html#AEN33590
|
||||||
|
|
||||||
|
.. method:: get_parameter_status(parameter)
|
||||||
|
|
||||||
|
Look up a current parameter setting of the server.
|
||||||
|
|
||||||
|
Potential values for ``parameter`` are: ``server_version``,
|
||||||
|
``server_encoding``, ``client_encoding``, ``is_superuser``,
|
||||||
|
``session_authorization``, ``DateStyle``, ``TimeZone``,
|
||||||
|
``integer_datetimes``, and ``standard_conforming_strings``.
|
||||||
|
|
||||||
|
If server did not report requested parameter, return ``None``.
|
||||||
|
|
||||||
|
.. seealso:: libpq docs for `PQparameterStatus()`__ for details.
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-status.html#AEN33499
|
||||||
|
|
||||||
|
.. method:: get_transaction_status()
|
||||||
|
|
||||||
|
Return the current session transaction status as an integer. Symbolic
|
||||||
|
constants for the vaules are defined in the module
|
||||||
|
:mod:`psycopg2.extensions`: see :ref:`transaction-status-constants`
|
||||||
|
for the available values.
|
||||||
|
|
||||||
|
.. seealso:: libpq docs for `PQtransactionStatus()`__ for details.
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-status.html#AEN33480
|
||||||
|
|
||||||
|
.. attribute:: protocol_version
|
||||||
|
|
||||||
|
A read-ony integer representing frontend/backend protocol being used.
|
||||||
|
It can be 2 or 3.
|
||||||
|
|
||||||
|
.. seealso:: libpq docs for `PQprotocolVersion()`__ for details.
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-status.html#AEN33546
|
||||||
|
|
||||||
|
.. attribute:: server_version
|
||||||
|
|
||||||
|
A read-only integer representing the backend version.
|
||||||
|
|
||||||
|
The number is formed by converting the major, minor, and revision
|
||||||
|
numbers into two-decimal-digit numbers and appending them together.
|
||||||
|
For example, version 8.1.5 will be returned as 80105,
|
||||||
|
|
||||||
|
.. seealso:: libpq docs for `PQserverVersion()`__ for details.
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-status.html#AEN33556
|
||||||
|
|
||||||
|
|
||||||
|
.. attribute:: status
|
||||||
|
|
||||||
|
A read-only integer representing the status of the connection.
|
||||||
|
Symbolic constants for the values are defined in the module
|
||||||
|
:mod:`psycopg2.extensions`: see :ref:`connection-status-constants`
|
||||||
|
for the available values.
|
||||||
|
|
||||||
|
|
||||||
|
.. method:: lobject([oid [, mode [, new_oid [, new_file [, lobject_factory]]]]])
|
||||||
|
|
||||||
|
Return a new database large object.
|
||||||
|
|
||||||
|
The ``lobject_factory`` argument can be used to create non-standard
|
||||||
|
lobjects by passing a class different from the default. Note that the
|
||||||
|
new class *should* be a sub-class of
|
||||||
|
:class:`psycopg2.extensions.lobject`.
|
||||||
|
|
||||||
|
.. todo:: conn.lobject details
|
||||||
|
|
||||||
|
.. attribute:: notifies
|
||||||
|
|
||||||
|
.. todo:: describe conn.notifies
|
||||||
|
|
||||||
|
.. attribute:: notices
|
||||||
|
|
||||||
|
.. todo:: describe conn.notices
|
||||||
|
|
||||||
|
.. attribute:: binary_types
|
||||||
|
|
||||||
|
.. todo:: describe binary_types
|
||||||
|
|
||||||
|
.. attribute:: string_types
|
||||||
|
|
||||||
|
.. todo:: describe string_types
|
||||||
|
|
||||||
|
|
||||||
|
The :class:`connection` also exposes the same `Error` classes available in
|
||||||
|
the :mod:`psycopg2` module as attributes.
|
||||||
|
|
391
doc/cursor.rst
Normal file
391
doc/cursor.rst
Normal file
|
@ -0,0 +1,391 @@
|
||||||
|
The ``cursor`` class
|
||||||
|
====================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. class:: cursor
|
||||||
|
|
||||||
|
Allows Python code to execute PostgreSQL command in a database session.
|
||||||
|
Cursors are created by the :meth:`connection.cursor`: they are bound to
|
||||||
|
the connection for the entire lifetime and all the commands are executed
|
||||||
|
in the context of the database session wrapped by the connection.
|
||||||
|
|
||||||
|
Cursors created from the same connection are not isolated, i.e., any
|
||||||
|
changes done to the database by a cursor are immediately visible by the
|
||||||
|
other cursors. Cursors created from different connections can or can not
|
||||||
|
be isolated, depending on the :attr:`connection.isolation_level`. See also
|
||||||
|
:meth:`connection.rollback()` and :meth:`connection.commit()` methods.
|
||||||
|
|
||||||
|
Cursors are *not* thread safe: a multithread application can create
|
||||||
|
many cursors from the same same connection and should use each cursor from
|
||||||
|
a single thread. See :ref:`thread-safety` for details.
|
||||||
|
|
||||||
|
|
||||||
|
.. attribute:: description
|
||||||
|
|
||||||
|
This read-only attribute is a sequence of 7-item sequences.
|
||||||
|
|
||||||
|
Each of these sequences contains information describing one result
|
||||||
|
column:
|
||||||
|
|
||||||
|
- ``name``
|
||||||
|
- ``type_code``
|
||||||
|
- ``display_size``
|
||||||
|
- ``internal_size``
|
||||||
|
- ``precision``
|
||||||
|
- ``scale``
|
||||||
|
- ``null_ok``
|
||||||
|
|
||||||
|
The first two items (``name`` and ``type_code``) are mandatory, the
|
||||||
|
other five are optional and are set to ``None`` if no meaningful
|
||||||
|
values can be provided.
|
||||||
|
|
||||||
|
This attribute will be ``None`` for operations that do not return rows
|
||||||
|
or if the cursor has not had an operation invoked via the
|
||||||
|
|execute*|_ methods yet.
|
||||||
|
|
||||||
|
The type_code can be interpreted by comparing it to the Type Objects
|
||||||
|
specified in the section :ref:`type-objects-and-costructors`.
|
||||||
|
|
||||||
|
.. method:: close()
|
||||||
|
|
||||||
|
Close the cursor now (rather than whenever ``__del__`` is called).
|
||||||
|
The cursor will be unusable from this point forward; an :exc:`Error` (or
|
||||||
|
subclass) exception will be raised if any operation is attempted with
|
||||||
|
the cursor.
|
||||||
|
|
||||||
|
.. attribute:: closed
|
||||||
|
|
||||||
|
Read-only boolean attribute: specifies if the cursor is closed
|
||||||
|
(``True``) or not (``False``).
|
||||||
|
|
||||||
|
.. attribute:: connection
|
||||||
|
|
||||||
|
Read-only attribute returning a reference to the :class:`connection`
|
||||||
|
object on which the cursor was created.
|
||||||
|
|
||||||
|
.. attribute:: name
|
||||||
|
|
||||||
|
Read-only attribute containing the name of the cursor if it was
|
||||||
|
creates as named cursor by :meth:`connection.cursor`, or ``None`` if
|
||||||
|
it is a client side cursor. See :ref:`server-side-cursors`.
|
||||||
|
|
||||||
|
.. |execute*| replace:: :obj:`execute*()`
|
||||||
|
|
||||||
|
.. _execute*:
|
||||||
|
|
||||||
|
.. method:: execute(operation [, parameters] [, async])
|
||||||
|
|
||||||
|
Prepare and execute a database operation (query or command).
|
||||||
|
|
||||||
|
Parameters may be provided as sequence or mapping and will be bound to
|
||||||
|
variables in the operation. Variables are specified either with
|
||||||
|
positional (``%s``) or named (``%(name)s``) placeholders. See
|
||||||
|
:ref:`query-parameters`.
|
||||||
|
|
||||||
|
The method returns `None`. If a query was executed, the returned
|
||||||
|
values can be retrieved using |fetch*|_ methods.
|
||||||
|
|
||||||
|
A reference to the operation will be retained by the cursor. If the
|
||||||
|
same operation object is passed in again, then the cursor can optimize
|
||||||
|
its behavior. This is most effective for algorithms where the same
|
||||||
|
operation is used, but different parameters are bound to it (many
|
||||||
|
times).
|
||||||
|
|
||||||
|
.. todo:: does Psycopg2 do the above?
|
||||||
|
|
||||||
|
If :obj:`async` is ``True``, query execution will be asynchronous: the
|
||||||
|
function returns immediately while the query is executed by the
|
||||||
|
backend. Use the :attr:`isready` attribute to see if the data is
|
||||||
|
ready for return via |fetch*|_ methods. See
|
||||||
|
:ref:`asynchronous-queries`.
|
||||||
|
|
||||||
|
.. method:: mogrify(operation [, parameters)
|
||||||
|
|
||||||
|
Return a query string after arguments binding. The string returned is
|
||||||
|
exactly the one that would be sent to the database running the
|
||||||
|
:meth:`execute()` method or similar.
|
||||||
|
|
||||||
|
.. method:: executemany(operation, seq_of_parameters)
|
||||||
|
|
||||||
|
Prepare a database operation (query or command) and then execute it
|
||||||
|
against all parameter sequences or mappings found in the sequence
|
||||||
|
seq_of_parameters.
|
||||||
|
|
||||||
|
The function is mostly useful for commands that update the database:
|
||||||
|
any result set returned by the query is discarded.
|
||||||
|
|
||||||
|
Parameters are bounded to the query using the same rules described in
|
||||||
|
the :meth:`execute()` method.
|
||||||
|
|
||||||
|
.. method:: callproc(procname [, parameters] [, async])
|
||||||
|
|
||||||
|
Call a stored database procedure with the given name. The sequence of
|
||||||
|
parameters must contain one entry for each argument that the procedure
|
||||||
|
expects. The result of the call is returned as modified copy of the
|
||||||
|
input sequence. Input parameters are left untouched, output and
|
||||||
|
input/output parameters replaced with possibly new values.
|
||||||
|
|
||||||
|
The procedure may also provide a result set as output. This must then
|
||||||
|
be made available through the standard |fetch*|_ methods.
|
||||||
|
|
||||||
|
If :obj:`async` is ``True``, procedure execution will be asynchronous:
|
||||||
|
the function returns immediately while the procedure is executed by
|
||||||
|
the backend. Use the :attr:`isready` attribute to see if the data is
|
||||||
|
ready for return via |fetch*|_ methods. See
|
||||||
|
:ref:`asynchronous-queries`.
|
||||||
|
|
||||||
|
.. attribute:: query
|
||||||
|
|
||||||
|
Read-only attribute containing the body of the last query sent to the
|
||||||
|
backend (including bound arguments). ``None`` if no query has been
|
||||||
|
executed yet::
|
||||||
|
|
||||||
|
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
|
||||||
|
>>> cur.query
|
||||||
|
"INSERT INTO test (num, data) VALUES (42, E'bar')"
|
||||||
|
|
||||||
|
.. attribute:: statusmessage
|
||||||
|
|
||||||
|
Return the message returned by the last command::
|
||||||
|
|
||||||
|
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
|
||||||
|
>>> cur.statusmessage
|
||||||
|
'INSERT 0 1'
|
||||||
|
|
||||||
|
.. method:: isready()
|
||||||
|
|
||||||
|
Return ``False`` if the backend is still processing an asynchronous
|
||||||
|
query or ``True`` if data is ready to be fetched by one of the
|
||||||
|
|fetch*|_ methods. See :ref:`asynchronous-queries`.
|
||||||
|
|
||||||
|
.. method:: fileno()
|
||||||
|
|
||||||
|
Return the file descriptor associated with the current connection and
|
||||||
|
make possible to use a cursor in a context where a file object would
|
||||||
|
be expected (like in a :func:`select()` call). See
|
||||||
|
:ref:`asynchronous-queries`.
|
||||||
|
|
||||||
|
|
||||||
|
.. |fetch*| replace:: :obj:`fetch*()`
|
||||||
|
|
||||||
|
.. _fetch*:
|
||||||
|
|
||||||
|
The followig methods are used to read data from the database after an
|
||||||
|
:meth:`execute()` call.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
:class:`cursor` objects are iterable, so, instead of calling
|
||||||
|
explicitely :meth:`fetchone()` in a loop, the object itself can be
|
||||||
|
used::
|
||||||
|
|
||||||
|
>>> cur.execute("SELECT * FROM test;")
|
||||||
|
>>> for record in cur:
|
||||||
|
... print record
|
||||||
|
...
|
||||||
|
(1, 100, "abc'def")
|
||||||
|
(2, None, 'dada')
|
||||||
|
(4, 42, 'bar')
|
||||||
|
|
||||||
|
.. method:: fetchone()
|
||||||
|
|
||||||
|
Fetch the next row of a query result set, returning a single tuple,
|
||||||
|
or ``None`` when no more data is available::
|
||||||
|
|
||||||
|
>>> cur.execute("SELECT * FROM test WHERE id = %s", (4,))
|
||||||
|
>>> cur.fetchone()
|
||||||
|
(4, 42, 'bar')
|
||||||
|
|
||||||
|
An :exc:`Error` (or subclass) exception is raised if the previous call
|
||||||
|
to |execute*|_ did not produce any result set or no call was issued
|
||||||
|
yet.
|
||||||
|
|
||||||
|
.. method:: fetchmany([size=cursor.arraysize])
|
||||||
|
|
||||||
|
Fetch the next set of rows of a query result, returning a list of
|
||||||
|
tuples. An empty list is returned when no more rows are available.
|
||||||
|
|
||||||
|
The number of rows to fetch per call is specified by the parameter.
|
||||||
|
If it is not given, the cursor's :attr:`arraysize` determines the
|
||||||
|
number of rows to be fetched. The method should try to fetch as many
|
||||||
|
rows as indicated by the size parameter. If this is not possible due
|
||||||
|
to the specified number of rows not being available, fewer rows may be
|
||||||
|
returned::
|
||||||
|
|
||||||
|
>>> cur.execute("SELECT * FROM test;")
|
||||||
|
>>> cur.fetchmany(2)
|
||||||
|
[(1, 100, "abc'def"), (2, None, 'dada')]
|
||||||
|
>>> cur.fetchmany(2)
|
||||||
|
[(4, 42, 'bar')]
|
||||||
|
>>> cur.fetchmany(2)
|
||||||
|
[]
|
||||||
|
|
||||||
|
An :exc:`Error` (or subclass) exception is raised if the previous
|
||||||
|
call to |execute*|_ did not produce any result set or no call was
|
||||||
|
issued yet.
|
||||||
|
|
||||||
|
Note there are performance considerations involved with the size
|
||||||
|
parameter. For optimal performance, it is usually best to use the
|
||||||
|
:attr:`arraysize` attribute. If the size parameter is used, then it
|
||||||
|
is best for it to retain the same value from one :meth:`fetchmany()`
|
||||||
|
call to the next.
|
||||||
|
|
||||||
|
.. method:: fetchall()
|
||||||
|
|
||||||
|
Fetch all (remaining) rows of a query result, returning them as a list
|
||||||
|
of tuples. Note that the cursor's :attr:`arraysize` attribute can
|
||||||
|
affect the performance of this operation::
|
||||||
|
|
||||||
|
>>> cur.execute("SELECT * FROM test;")
|
||||||
|
>>> cur.fetchall()
|
||||||
|
[(1, 100, "abc'def"), (2, None, 'dada'), (4, 42, 'bar')]
|
||||||
|
|
||||||
|
.. todo:: does arraysize influence fetchall()?
|
||||||
|
|
||||||
|
An :exc:`Error` (or subclass) exception is raised if the previous call
|
||||||
|
to |execute*|_ did not produce any result set or no call was issued
|
||||||
|
yet.
|
||||||
|
|
||||||
|
.. method:: scroll(value[,mode='relative'])
|
||||||
|
|
||||||
|
Scroll the cursor in the result set to a new position according
|
||||||
|
to mode.
|
||||||
|
|
||||||
|
If mode is ``relative`` (default), value is taken as offset to
|
||||||
|
the current position in the result set, if set to ``absolute``,
|
||||||
|
value states an absolute target position.
|
||||||
|
|
||||||
|
If the scroll operation would leave the result set, a
|
||||||
|
:exc:`ProgrammingError` is raised and the cursor position is not
|
||||||
|
changed.
|
||||||
|
|
||||||
|
.. todo:: dbapi says should have been IndexError...
|
||||||
|
|
||||||
|
The method can be used both for client-side cursors and server-side
|
||||||
|
(named) cursors.
|
||||||
|
|
||||||
|
.. attribute:: arraysize
|
||||||
|
|
||||||
|
This read/write attribute specifies the number of rows to fetch at a
|
||||||
|
time with :meth:`fetchmany()`. It defaults to 1 meaning to fetch a
|
||||||
|
single row at a time.
|
||||||
|
|
||||||
|
Implementations must observe this value with respect to the
|
||||||
|
:meth:`fetchmany()` method, but are free to interact with the database
|
||||||
|
a single row at a time. It may also be used in the implementation of
|
||||||
|
:meth:`executemany()`.
|
||||||
|
|
||||||
|
.. todo:: copied from dbapi: better specify what psycopg does with
|
||||||
|
arraysize
|
||||||
|
|
||||||
|
.. attribute:: rowcount
|
||||||
|
|
||||||
|
This read-only attribute specifies the number of rows that the last
|
||||||
|
|execute*|_ produced (for DQL statements like ``SELECT``) or
|
||||||
|
affected (for DML statements like ``UPDATE`` or ``INSERT``).
|
||||||
|
|
||||||
|
The attribute is -1 in case no |execute*| has been performed on
|
||||||
|
the cursor or the rowcount of the last operation is cannot be
|
||||||
|
determined by the interface.
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
The |DBAPI 2.0|_ interface reserves to redefine the latter case to
|
||||||
|
have the object return ``None`` instead of -1 in future versions
|
||||||
|
of the specification.
|
||||||
|
|
||||||
|
.. attribute:: rownumber
|
||||||
|
|
||||||
|
This read-only attribute provides the current 0-based index of the
|
||||||
|
cursor in the result set or ``None`` if the index cannot be
|
||||||
|
determined.
|
||||||
|
|
||||||
|
The index can be seen as index of the cursor in a sequence (the result
|
||||||
|
set). The next fetch operation will fetch the row indexed by
|
||||||
|
:attr:`rownumber` in that sequence.
|
||||||
|
|
||||||
|
.. index:: oid
|
||||||
|
|
||||||
|
.. attribute:: lastrowid
|
||||||
|
|
||||||
|
This read-only attribute provides the *oid* of the last row inserted
|
||||||
|
by the cursor. If the table wasn't created with oid support or the
|
||||||
|
last operation is not a single record insert, the attribute is set to
|
||||||
|
``None``.
|
||||||
|
|
||||||
|
PostgreSQL currently advises to not create oid on the tables and the
|
||||||
|
default for |CREATE-TABLE|__ is to not support them. The
|
||||||
|
|INSERT-RETURNING|__ syntax available from PostgreSQL 8.3 allows more
|
||||||
|
flexibility:
|
||||||
|
|
||||||
|
.. |CREATE-TABLE| replace:: ``CREATE TABLE``
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/sql-createtable.html
|
||||||
|
|
||||||
|
.. |INSERT-RETURNING| replace:: ``INSERT ... RETURNING``
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/sql-insert.html
|
||||||
|
|
||||||
|
|
||||||
|
.. method:: nextset()
|
||||||
|
|
||||||
|
This method is not supported (PostgreSQL does not have multiple data
|
||||||
|
sets) and will raise a :exc:`NotSupportedError` exception.
|
||||||
|
|
||||||
|
.. method:: setinputsizes(sizes)
|
||||||
|
|
||||||
|
This method currently does nothing but it is safe to call it.
|
||||||
|
|
||||||
|
.. method:: setoutputsize(size [, column])
|
||||||
|
|
||||||
|
This method currently does nothing but it is safe to call it.
|
||||||
|
|
||||||
|
|
||||||
|
.. method:: copy_expert(sql, file [, size])
|
||||||
|
|
||||||
|
Submit a user-composed COPY statement.
|
||||||
|
|
||||||
|
:obj:`file` must be an open, readable file for ``COPY FROM`` or an
|
||||||
|
open, writeable file for ``COPY TO``. The optional :obj:`size`
|
||||||
|
argument, when specified for a ``COPY FROM`` statement, will be passed
|
||||||
|
to file's read method to control the read buffer size.
|
||||||
|
|
||||||
|
.. todo::
|
||||||
|
|
||||||
|
I'm sure copy_expert can be described better!
|
||||||
|
|
||||||
|
.. method:: copy_from(file, table, sep='\t', null='\N', columns=None)
|
||||||
|
|
||||||
|
Read data *from* the file-like object :obj:`file` appending them to
|
||||||
|
the table named :obj:`table`. See :ref:`copy`.
|
||||||
|
|
||||||
|
:obj:`file` must have both ``read()`` and ``readline()`` method.
|
||||||
|
|
||||||
|
The optional arguments: :obj:`sep` is the columns separator and
|
||||||
|
:obj:`null` represents ``NULL`` values in the file.
|
||||||
|
|
||||||
|
.. todo:: columns argument in copy_from
|
||||||
|
|
||||||
|
.. method:: copy_to(file, table, sep='\t', null='\N', columns=None)
|
||||||
|
|
||||||
|
Write the content of the table named :obj:`table` *to* the file-like
|
||||||
|
object :obj:`file`. See :ref:`copy`.
|
||||||
|
|
||||||
|
:obj:`file` must have a ``write()`` method.
|
||||||
|
|
||||||
|
The optional arguments: :obj:`sep` is the columns separator and
|
||||||
|
:obj:`null` represents ``NULL`` values in the file.
|
||||||
|
|
||||||
|
.. todo:: columns argument in copy_to
|
||||||
|
|
||||||
|
.. attribute:: row_factory
|
||||||
|
|
||||||
|
.. todo:: cursor.row_factory
|
||||||
|
|
||||||
|
.. attribute:: typecaster
|
||||||
|
|
||||||
|
.. todo:: cursor.typecaster
|
||||||
|
|
||||||
|
.. attribute:: tzinfo_factory
|
||||||
|
|
||||||
|
.. todo:: tzinfo_factory
|
||||||
|
|
||||||
|
|
|
@ -1,260 +1,154 @@
|
||||||
=======================================
|
:mod:`psycopg2.extensions` -- Extensions to the DBAPI
|
||||||
psycopg 2 extensions to the DBAPI 2.0
|
=====================================================
|
||||||
=======================================
|
|
||||||
|
|
||||||
This document is a short summary of the extensions built in psycopg 2.0.x over
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
the standard `Python Database API Specification 2.0`__, usually called simply
|
|
||||||
DBAPI-2.0 or even PEP-249. Before reading on this document please make sure
|
|
||||||
you already know how to program in Python using a DBAPI-2.0 compliant driver:
|
|
||||||
basic concepts like opening a connection, executing queries and commiting or
|
|
||||||
rolling back a transaction will not be explained but just used.
|
|
||||||
|
|
||||||
.. __: http://www.python.org/peps/pep-0249.html
|
.. module:: psycopg2.extensions
|
||||||
|
|
||||||
Many objects and extension functions are defined in the `psycopg2.extensions`
|
The module contains a few objects and function extending the minimum set of
|
||||||
module.
|
functionalities defined by the |DBAPI 2.0|.
|
||||||
|
|
||||||
|
|
||||||
Connection and cursor factories
|
.. class:: connection
|
||||||
===============================
|
|
||||||
|
|
||||||
psycopg 2 exposes two new-style classes that can be sub-classed and expanded to
|
Is the class usually returned by the :func:`psycopg2.connect()` function.
|
||||||
adapt them to the needs of the programmer: `cursor` and `connection`. The
|
It is exposed by the :mod:`extensions` module in order to allow
|
||||||
`connection` class is usually sub-classed only to provide an easy way to create
|
subclassing to extend its behaviour: the subclass should be passed to the
|
||||||
customized cursors but other uses are possible. `cursor` is much more
|
:func:`connect()` function using the :obj:`connection_factory` parameter.
|
||||||
interesting, because it is the class where query building, execution and result
|
See also :ref:`subclassing-connection`.
|
||||||
type-casting into Python variables happens.
|
|
||||||
|
|
||||||
An example of cursor subclass performing logging is::
|
For a complete description of the class, see :class:`connection`.
|
||||||
|
|
||||||
import psycopg2
|
.. class:: cursor
|
||||||
import psycopg2.extensions
|
|
||||||
import logging
|
|
||||||
|
|
||||||
class LoggingCursor(psycopg2.extensions.cursor):
|
It is the class usually returnded by the :meth:`connection.cursor()`
|
||||||
def execute(self, sql, args=None):
|
method. It is exposed by the :mod:`extensions` module in order to allow
|
||||||
logger = logging.getLogger('sql_debug')
|
subclassing to extend its behaviour: the subclass should be passed to the
|
||||||
logger.info(self.mogrify(sql, args))
|
``cursor()`` method using the :obj:`cursor_factory` parameter. See
|
||||||
|
also :ref:`subclassing-cursor`.
|
||||||
|
|
||||||
try:
|
For a complete description of the class, see :class:`cursor`.
|
||||||
psycopg2.extensions.cursor.execute(self, sql, args)
|
|
||||||
except Exception, exc:
|
|
||||||
logger.error("%s: %s" % (exc.__class__.__name__, exc))
|
|
||||||
raise
|
|
||||||
|
|
||||||
conn = psycopg2.connect(DSN)
|
.. todo:: row factories
|
||||||
curs = conn.cursor(cursor_factory=LoggingCursor)
|
|
||||||
curs.execute("INSERT INTO mytable VALUES (%s, %s, %s);",
|
.. class:: lobject
|
||||||
(10, 20, 30))
|
|
||||||
|
.. todo:: class lobject
|
||||||
|
|
||||||
|
|
||||||
Row factories
|
.. todo:: finish module extensions
|
||||||
-------------
|
|
||||||
|
|
||||||
tzinfo factories
|
|
||||||
----------------
|
|
||||||
|
|
||||||
|
|
||||||
Setting transaction isolation levels
|
.. _isolation-level-constants:
|
||||||
====================================
|
|
||||||
|
|
||||||
psycopg2 connection objects hold informations about the PostgreSQL `transaction
|
Isolation level constants
|
||||||
isolation level`_. The current transaction level can be read from the
|
-------------------------
|
||||||
`.isolation_level` attribute. The default isolation level is ``READ
|
|
||||||
COMMITTED``. A different isolation level con be set through the
|
|
||||||
`.set_isolation_level()` method. The level can be set to one of the following
|
|
||||||
constants, defined in `psycopg2.extensions`:
|
|
||||||
|
|
||||||
`ISOLATION_LEVEL_AUTOCOMMIT`
|
Psycopg2 connection objects hold informations about the PostgreSQL
|
||||||
No transaction is started when command are issued and no
|
`transaction isolation level`_. The current transaction level can be read
|
||||||
`.commit()`/`.rollback()` is required. Some PostgreSQL command such as
|
from the :attr:`connection.isolation_level` attribute. The default isolation
|
||||||
``CREATE DATABASE`` can't run into a transaction: to run such command use
|
level is ``READ COMMITTED``. A different isolation level con be set through
|
||||||
`.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)`.
|
the :meth:`connection.set_isolation_level()` method. The level can be set to
|
||||||
|
one of the following constants:
|
||||||
|
|
||||||
|
.. data:: ISOLATION_LEVEL_AUTOCOMMIT
|
||||||
|
|
||||||
|
No transaction is started when command are issued and no ``commit()`` or
|
||||||
|
``rollback()`` is required. Some PostgreSQL command such as ``CREATE
|
||||||
|
DATABASE`` can't run into a transaction: to run such command use::
|
||||||
|
|
||||||
|
>>> conn.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)
|
||||||
|
|
||||||
|
.. data:: ISOLATION_LEVEL_READ_UNCOMMITTED
|
||||||
|
|
||||||
|
This isolation level is defined in the SQL standard but not available in
|
||||||
|
the MVCC model of PostgreSQL: it is replaced by the stricter ``READ
|
||||||
|
COMMITTED``.
|
||||||
|
|
||||||
|
.. data:: ISOLATION_LEVEL_READ_COMMITTED
|
||||||
|
|
||||||
`ISOLATION_LEVEL_READ_COMMITTED`
|
|
||||||
This is the default value. A new transaction is started at the first
|
This is the default value. A new transaction is started at the first
|
||||||
`.execute()` command on a cursor and at each new `.execute()` after a
|
:meth:`cursor.execute()` command on a cursor and at each new ``execute()``
|
||||||
`.commit()` or a `.rollback()`. The transaction runs in the PostgreSQL
|
after a :meth:`connection.commit()` or a :meth:`connection.rollback()`.
|
||||||
``READ COMMITTED`` isolation level.
|
The transaction runs in the PostgreSQL ``READ COMMITTED`` isolation level.
|
||||||
|
|
||||||
`ISOLATION_LEVEL_SERIALIZABLE`
|
.. data:: ISOLATION_LEVEL_REPEATABLE_READ
|
||||||
Transactions are run at a ``SERIALIZABLE`` isolation level.
|
|
||||||
|
This isolation level is defined in the SQL standard but not available in
|
||||||
|
the MVCC model of PostgreSQL: it is replaced by the stricter
|
||||||
|
``SERIALIZABLE``.
|
||||||
|
|
||||||
|
.. data:: ISOLATION_LEVEL_SERIALIZABLE
|
||||||
|
|
||||||
|
Transactions are run at a ``SERIALIZABLE`` isolation level. This is the
|
||||||
|
strictest transactions isolation level, equivalent to having the
|
||||||
|
transactions executed serially rather than concurrently. However
|
||||||
|
applications using this level must be prepared to retry reansactions due
|
||||||
|
to serialization failures. See `serializable isolation level`_ in
|
||||||
|
PostgreSQL documentation.
|
||||||
|
|
||||||
|
|
||||||
.. _transaction isolation level:
|
.. _transaction-status-constants:
|
||||||
http://www.postgresql.org/docs/8.1/static/transaction-iso.html
|
|
||||||
|
Transaction status constants
|
||||||
|
----------------------------
|
||||||
|
|
||||||
|
These values represent the possible status of a transaction: the current value
|
||||||
|
can be read using the :meth:`connection.get_transaction_status()` method.
|
||||||
|
|
||||||
|
.. data:: TRANSACTION_STATUS_IDLE
|
||||||
|
|
||||||
|
The session is idle and there is no current transaction.
|
||||||
|
|
||||||
|
.. data:: TRANSACTION_STATUS_ACTIVE
|
||||||
|
|
||||||
|
A command is currently in progress.
|
||||||
|
|
||||||
|
.. data:: TRANSACTION_STATUS_INTRANS
|
||||||
|
|
||||||
|
The session is idle in a valid transaction block.
|
||||||
|
|
||||||
|
.. data:: TRANSACTION_STATUS_INERROR
|
||||||
|
|
||||||
|
The session is idle in a failed transaction block.
|
||||||
|
|
||||||
|
.. data:: TRANSACTION_STATUS_UNKNOWN
|
||||||
|
|
||||||
|
Reported if the connection with the server is bad.
|
||||||
|
|
||||||
|
|
||||||
Adaptation of Python values to SQL types
|
.. _connection-status-constants:
|
||||||
========================================
|
|
||||||
|
|
||||||
psycopg2 casts Python variables to SQL literals by type. Standard Python types
|
Connection status constants
|
||||||
are already adapted to the proper SQL literal.
|
---------------------------
|
||||||
|
|
||||||
Example: the Python function::
|
These values represent the possible status of a connection: the current value
|
||||||
|
can be read from the :data:`connection.status` attribute.
|
||||||
|
|
||||||
curs.execute("""INSERT INTO atable (anint, adate, astring)
|
.. todo:: check if these values are really useful or not.
|
||||||
VALUES (%s, %s, %s)""",
|
|
||||||
(10, datetime.date(2005, 11, 18), "O'Reilly"))
|
|
||||||
|
|
||||||
is converted into the SQL command::
|
.. data:: STATUS_SETUP
|
||||||
|
|
||||||
INSERT INTO atable (anint, adate, astring)
|
Defined but not used.
|
||||||
VALUES (10, '2005-11-18', 'O''Reilly');
|
|
||||||
|
|
||||||
Named arguments are supported too with ``%(name)s`` placeholders. Notice that:
|
.. data:: STATUS_READY
|
||||||
|
|
||||||
- The Python string operator ``%`` is not used: the `.execute()` function
|
Connection established.
|
||||||
accepts the values tuple or dictionary as second parameter.
|
|
||||||
|
|
||||||
- The variables placeholder must always be a ``%s``, even if a different
|
.. data:: STATUS_BEGIN
|
||||||
placeholder (such as a ``%d`` for an integer) may look more appropriate.
|
|
||||||
|
|
||||||
- For positional variables binding, the second argument must always be a
|
Connection established. A transaction is in progress.
|
||||||
tuple, even if it contains a single variable.
|
|
||||||
|
|
||||||
- Only variable values should be bound via this method: it shouldn't be used
|
.. data:: STATUS_IN_TRANSACTION
|
||||||
to set table or field names. For these elements, ordinary string formatting
|
|
||||||
should be used before running `.execute()`.
|
An alias for :data:`STATUS_BEGIN`
|
||||||
|
|
||||||
|
.. data:: STATUS_SYNC
|
||||||
|
|
||||||
|
Defined but not used.
|
||||||
|
|
||||||
|
.. data:: STATUS_ASYNC
|
||||||
|
|
||||||
|
Defined but not used.
|
||||||
|
|
||||||
|
|
||||||
Adapting new types
|
|
||||||
------------------
|
|
||||||
|
|
||||||
Any Python class or type can be adapted to an SQL string. Adaptation mechanism
|
|
||||||
is similar to the Object Adaptation proposed in the `PEP-246`_ and is exposed
|
|
||||||
by the `adapt()` function.
|
|
||||||
|
|
||||||
psycopg2 `.execute()` method adapts its ``vars`` arguments to the `ISQLQuote`
|
|
||||||
protocol. Objects that conform to this protocol expose a ``getquoted()`` method
|
|
||||||
returning the SQL representation of the object as a string.
|
|
||||||
|
|
||||||
The easiest way to adapt an object to an SQL string is to register an adapter
|
|
||||||
function via the `register_adapter()` function. The adapter function must take
|
|
||||||
the value to be adapted as argument and return a conform object. A convenient
|
|
||||||
object is the `AsIs` wrapper, whose ``getquoted()`` result is simply the
|
|
||||||
``str()``\ ingification of the wrapped object.
|
|
||||||
|
|
||||||
Example: mapping of a ``Point`` class into the ``point`` PostgreSQL geometric
|
|
||||||
type::
|
|
||||||
|
|
||||||
from psycopg2.extensions import adapt, register_adapter, AsIs
|
|
||||||
|
|
||||||
class Point(object):
|
|
||||||
def __init__(self, x=0.0, y=0.0):
|
|
||||||
self.x = x
|
|
||||||
self.y = y
|
|
||||||
|
|
||||||
def adapt_point(point):
|
|
||||||
return AsIs("'(%s,%s)'" % (adapt(point.x), adapt(point.y)))
|
|
||||||
|
|
||||||
register_adapter(Point, adapt_point)
|
|
||||||
|
|
||||||
curs.execute("INSERT INTO atable (apoint) VALUES (%s)",
|
|
||||||
(Point(1.23, 4.56),))
|
|
||||||
|
|
||||||
The above function call results in the SQL command::
|
|
||||||
|
|
||||||
INSERT INTO atable (apoint) VALUES ((1.23, 4.56));
|
|
||||||
|
|
||||||
|
|
||||||
.. _PEP-246: http://www.python.org/peps/pep-0246.html
|
|
||||||
|
|
||||||
|
|
||||||
Type casting of SQL types into Python values
|
|
||||||
============================================
|
|
||||||
|
|
||||||
PostgreSQL objects read from the database can be adapted to Python objects
|
|
||||||
through an user-defined adapting function. An adapter function takes two
|
|
||||||
argments: the object string representation as returned by PostgreSQL and the
|
|
||||||
cursor currently being read, and should return a new Python object. For
|
|
||||||
example, the following function parses a PostgreSQL ``point`` into the
|
|
||||||
previously defined ``Point`` class::
|
|
||||||
|
|
||||||
def cast_point(value, curs):
|
|
||||||
if value is not None:
|
|
||||||
# Convert from (f1, f2) syntax using a regular expression.
|
|
||||||
m = re.match("\((.*),(.*)\)", value)
|
|
||||||
if m:
|
|
||||||
return Point(float(m.group(1)), float(m.group(2)))
|
|
||||||
|
|
||||||
To create a mapping from the PostgreSQL type (either standard or user-defined),
|
|
||||||
its ``oid`` must be known. It can be retrieved either by the second column of
|
|
||||||
the cursor description::
|
|
||||||
|
|
||||||
curs.execute("SELECT NULL::point")
|
|
||||||
point_oid = curs.description[0][1] # usually returns 600
|
|
||||||
|
|
||||||
or by querying the system catalogs for the type name and namespace (the
|
|
||||||
namespace for system objects is ``pg_catalog``)::
|
|
||||||
|
|
||||||
curs.execute("""
|
|
||||||
SELECT pg_type.oid
|
|
||||||
FROM pg_type JOIN pg_namespace
|
|
||||||
ON typnamespace = pg_namespace.oid
|
|
||||||
WHERE typname = %(typename)s
|
|
||||||
AND nspname = %(namespace)s""",
|
|
||||||
{'typename': 'point', 'namespace': 'pg_catalog'})
|
|
||||||
|
|
||||||
point_oid = curs.fetchone()[0]
|
|
||||||
|
|
||||||
After you know the object ``oid``, you must can and register the new type::
|
|
||||||
|
|
||||||
POINT = psycopg2.extensions.new_type((point_oid,), "POINT", cast_point)
|
|
||||||
psycopg2.extensions.register_type(POINT)
|
|
||||||
|
|
||||||
The `new_type()` function binds the object oids (more than one can be
|
|
||||||
specified) to the adapter function. `register_type()` completes the spell.
|
|
||||||
Conversion is automatically performed when a column whose type is a registered
|
|
||||||
``oid`` is read::
|
|
||||||
|
|
||||||
curs.execute("SELECT '(10.2,20.3)'::point")
|
|
||||||
point = curs.fetchone()[0]
|
|
||||||
print type(point), point.x, point.y
|
|
||||||
# Prints: "<class '__main__.Point'> 10.2 20.3"
|
|
||||||
|
|
||||||
|
|
||||||
Working with times and dates
|
|
||||||
============================
|
|
||||||
|
|
||||||
|
|
||||||
Receiving NOTIFYs
|
|
||||||
=================
|
|
||||||
|
|
||||||
|
|
||||||
Using COPY TO and COPY FROM
|
|
||||||
===========================
|
|
||||||
|
|
||||||
psycopg2 `cursor` object provides an interface to the efficient `PostgreSQL
|
|
||||||
COPY command`__ to move data from files to tables and back.
|
|
||||||
|
|
||||||
The `.copy_to(file, table)` method writes the content of the table
|
|
||||||
named ``table`` *to* the file-like object ``file``. ``file`` must have a
|
|
||||||
``write()`` method.
|
|
||||||
|
|
||||||
The `.copy_from(file, table)` reads data *from* the file-like object
|
|
||||||
``file`` appending them to the table named ``table``. ``file`` must have both
|
|
||||||
``read()`` and ``readline()`` method.
|
|
||||||
|
|
||||||
Both methods accept two optional arguments: ``sep`` (defaulting to a tab) is
|
|
||||||
the columns separator and ``null`` (defaulting to ``\N``) represents ``NULL``
|
|
||||||
values in the file.
|
|
||||||
|
|
||||||
.. __: http://www.postgresql.org/docs/8.1/static/sql-copy.html
|
|
||||||
|
|
||||||
|
|
||||||
PostgreSQL status message and executed query
|
|
||||||
============================================
|
|
||||||
|
|
||||||
`cursor` objects have two special fields related to the last executed query:
|
|
||||||
|
|
||||||
- `.query` is the textual representation (str or unicode, depending on what
|
|
||||||
was passed to `.execute()` as first argument) of the query *after* argument
|
|
||||||
binding and mogrification has been applied. To put it another way, `.query`
|
|
||||||
is the *exact* query that was sent to the PostgreSQL backend.
|
|
||||||
|
|
||||||
- `.statusmessage` is the status message that the backend sent upon query
|
|
||||||
execution. It usually contains the basic type of the query (SELECT,
|
|
||||||
INSERT, UPDATE, ...) and some additional information like the number of
|
|
||||||
rows updated and so on. Refer to the PostgreSQL manual for more
|
|
||||||
information.
|
|
||||||
|
|
11
doc/extras.rst
Normal file
11
doc/extras.rst
Normal file
|
@ -0,0 +1,11 @@
|
||||||
|
:mod:`psycopg2.extras` -- Miscellaneous goodies for Psycopg 2
|
||||||
|
=============================================================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. module:: psycopg2.extras
|
||||||
|
|
||||||
|
This module is a generic place used to hold little helper functions and
|
||||||
|
classes until a better place in the distribution is found.
|
||||||
|
|
||||||
|
.. todo:: psycopg2.extras
|
57
doc/index.rst
Normal file
57
doc/index.rst
Normal file
|
@ -0,0 +1,57 @@
|
||||||
|
=================================================
|
||||||
|
Psycopg -- PostgreSQL database adapter for Python
|
||||||
|
=================================================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
Psycopg is a PostgreSQL_ database adapter for the Python_ programming
|
||||||
|
language. Its main advantages are that it supports the full Python |DBAPI 2.0|
|
||||||
|
and it is thread safe (threads can share the connections). It was designed for
|
||||||
|
heavily multi-threaded applications that create and destroy lots of cursors and
|
||||||
|
make a conspicuous number of concurrent INSERTs or UPDATEs. The psycopg
|
||||||
|
distribution includes ZPsycopgDA, a Zope_ Database Adapter.
|
||||||
|
|
||||||
|
Psycopg 2 is an almost complete rewrite of the psycopg 1.1.x branch. Psycopg 2
|
||||||
|
features complete libpq_ v3 protocol, `COPY TO/COPY FROM`__ and full object
|
||||||
|
adaptation for all basic Python 2.3 types: strings (including unicode), ints,
|
||||||
|
longs, floats, buffers (binary objects), booleans, `mx.DateTime`_ and builtin
|
||||||
|
datetime types. It also supports unicode queries and Python lists mapped to
|
||||||
|
PostgreSQL arrays.
|
||||||
|
|
||||||
|
.. _PostgreSQL: http://www.postgresql.org/
|
||||||
|
.. _Python: http://www.python.org/
|
||||||
|
.. _Zope: http://www.zope.org/
|
||||||
|
.. _libpq: http://www.postgresql.org/docs/8.4/static/libpq.html
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/sql-copy.html
|
||||||
|
|
||||||
|
Contents:
|
||||||
|
|
||||||
|
.. toctree::
|
||||||
|
:maxdepth: 2
|
||||||
|
|
||||||
|
usage
|
||||||
|
module
|
||||||
|
connection
|
||||||
|
cursor
|
||||||
|
advanced
|
||||||
|
extensions
|
||||||
|
tz
|
||||||
|
extras
|
||||||
|
|
||||||
|
.. ifconfig:: todo_include_todos
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
**To Do items in the documentation**
|
||||||
|
|
||||||
|
.. todolist::
|
||||||
|
|
||||||
|
|
||||||
|
Indices and tables
|
||||||
|
==================
|
||||||
|
|
||||||
|
* :ref:`genindex`
|
||||||
|
* :ref:`modindex`
|
||||||
|
* :ref:`search`
|
||||||
|
|
||||||
|
|
258
doc/module.rst
Normal file
258
doc/module.rst
Normal file
|
@ -0,0 +1,258 @@
|
||||||
|
The :mod:`psycopg2` module content
|
||||||
|
==================================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. module:: psycopg2
|
||||||
|
|
||||||
|
The module interface respects the standard defined in the |DBAPI 2.0|.
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Connection string
|
||||||
|
double: Connection; Parameters
|
||||||
|
single: Username; Connection
|
||||||
|
single: Password; Connection
|
||||||
|
single: Host; Connection
|
||||||
|
single: Port; Connection
|
||||||
|
single: DSN (Database Source Name)
|
||||||
|
|
||||||
|
.. function:: connect(dsn or params[, connection_factory])
|
||||||
|
|
||||||
|
Create a new database session and return a new :class:`connection` object.
|
||||||
|
|
||||||
|
You can specify the connection parameters either as a string::
|
||||||
|
|
||||||
|
conn = psycopg2.connect("dbname=test user=postgres password=secret")
|
||||||
|
|
||||||
|
or using a set of keyword arguments::
|
||||||
|
|
||||||
|
conn = psycopg2.connect(database="test", user="postgres", password="secret")
|
||||||
|
|
||||||
|
The full list of available parameters is:
|
||||||
|
|
||||||
|
- ``dbname`` the database name (only in dsn string)
|
||||||
|
- ``database`` the database name (only as keyword argument)
|
||||||
|
- ``user`` user name used to authenticate
|
||||||
|
- ``password`` password used to authenticate
|
||||||
|
- ``host`` database host address (defaults to UNIX socket if not provided)
|
||||||
|
- ``port`` connection port number (defaults to 5432 if not provided)
|
||||||
|
- ``sslmode`` `SSL TCP/IP negotiation`__ mode
|
||||||
|
|
||||||
|
.. __: http://www.postgresql.org/docs/8.4/static/libpq-ssl.html#LIBPQ-SSL-SSLMODE-STATEMENTS
|
||||||
|
|
||||||
|
Using the :obj:`connection_factory` parameter a different class or
|
||||||
|
connections factory can be specified. It should be a callable object
|
||||||
|
taking a :obj:`dsn` argument. See :ref:`subclassing-connection` for
|
||||||
|
details.
|
||||||
|
|
||||||
|
.. data:: apilevel
|
||||||
|
|
||||||
|
String constant stating the supported DB API level. For :mod:`psycopg2` is
|
||||||
|
``2.0``.
|
||||||
|
|
||||||
|
.. data:: threadsafety
|
||||||
|
|
||||||
|
Integer constant stating the level of thread safety the interface
|
||||||
|
supports. For :mod:`psycopg2` is ``2``, i.e. threads can share the module
|
||||||
|
and the connection. See :ref:`thread-safety` for details.
|
||||||
|
|
||||||
|
.. data:: paramstyle
|
||||||
|
|
||||||
|
String constant stating the type of parameter marker formatting expected
|
||||||
|
by the interface. For :mod:`psycopg2` is ``pyformat``. See also
|
||||||
|
:ref:`query-parameters`.
|
||||||
|
|
||||||
|
|
||||||
|
.. index:: Exceptions
|
||||||
|
|
||||||
|
Exceptions
|
||||||
|
----------
|
||||||
|
|
||||||
|
In compliance with the |DBAPI 2.0|, the module makes informations about errors
|
||||||
|
available through the following exceptions:
|
||||||
|
|
||||||
|
.. todo::
|
||||||
|
There are actually a couple of extra extensions defined in _psycopg and
|
||||||
|
imported in the connection, but not in this module: shouldn't be there
|
||||||
|
them too?
|
||||||
|
|
||||||
|
|
||||||
|
.. exception:: Warning
|
||||||
|
|
||||||
|
Exception raised for important warnings like data truncations while
|
||||||
|
inserting, etc. It is a subclass of the Python |StandardError|_ (defined in
|
||||||
|
the module exceptions).
|
||||||
|
|
||||||
|
.. exception:: Error
|
||||||
|
|
||||||
|
Exception that is the base class of all other error exceptions. You can
|
||||||
|
use this to catch all errors with one single ``except`` statement. Warnings
|
||||||
|
are not considered errors and thus should not use this class as base. It
|
||||||
|
is a subclass of the Python |StandardError|_ (defined in the module
|
||||||
|
exceptions).
|
||||||
|
|
||||||
|
.. exception:: InterfaceError
|
||||||
|
|
||||||
|
Exception raised for errors that are related to the database interface
|
||||||
|
rather than the database itself. It is a subclass of :exc:`Error`.
|
||||||
|
|
||||||
|
.. exception:: DatabaseError
|
||||||
|
|
||||||
|
Exception raised for errors that are related to the database. It is a
|
||||||
|
subclass of :exc:`Error`.
|
||||||
|
|
||||||
|
.. exception:: DataError
|
||||||
|
|
||||||
|
Exception raised for errors that are due to problems with the processed
|
||||||
|
data like division by zero, numeric value out of range, etc. It is a
|
||||||
|
subclass of :exc:`DatabaseError`.
|
||||||
|
|
||||||
|
.. exception:: OperationalError
|
||||||
|
|
||||||
|
Exception raised for errors that are related to the database's operation
|
||||||
|
and not necessarily under the control of the programmer, e.g. an
|
||||||
|
unexpected disconnect occurs, the data source name is not found, a
|
||||||
|
transaction could not be processed, a memory allocation error occurred
|
||||||
|
during processing, etc. It is a subclass of :exc:`DatabaseError`.
|
||||||
|
|
||||||
|
.. exception:: IntegrityError
|
||||||
|
|
||||||
|
Exception raised when the relational integrity of the database is
|
||||||
|
affected, e.g. a foreign key check fails. It is a subclass of
|
||||||
|
:exc:`DatabaseError`.
|
||||||
|
|
||||||
|
.. exception:: InternalError
|
||||||
|
|
||||||
|
Exception raised when the database encounters an internal error, e.g. the
|
||||||
|
cursor is not valid anymore, the transaction is out of sync, etc. It is a
|
||||||
|
subclass of :exc:`DatabaseError`.
|
||||||
|
|
||||||
|
.. exception:: ProgrammingError
|
||||||
|
|
||||||
|
Exception raised for programming errors, e.g. table not found or already
|
||||||
|
exists, syntax error in the SQL statement, wrong number of parameters
|
||||||
|
specified, etc. It is a subclass of :exc:`DatabaseError`.
|
||||||
|
|
||||||
|
.. exception:: NotSupportedError
|
||||||
|
|
||||||
|
Exception raised in case a method or database API was used which is not
|
||||||
|
supported by the database, e.g. requesting a .rollback() on a connection
|
||||||
|
that does not support transaction or has transactions turned off. It is a
|
||||||
|
subclass of :exc:`DatabaseError`.
|
||||||
|
|
||||||
|
|
||||||
|
This is the exception inheritance layout:
|
||||||
|
|
||||||
|
- |StandardError|_
|
||||||
|
|
||||||
|
- :exc:`Warning`
|
||||||
|
- :exc:`Error`
|
||||||
|
|
||||||
|
- :exc:`InterfaceError`
|
||||||
|
- :exc:`DatabaseError`
|
||||||
|
|
||||||
|
- :exc:`DataError`
|
||||||
|
- :exc:`OperationalError`
|
||||||
|
- :exc:`IntegrityError`
|
||||||
|
- :exc:`InternalError`
|
||||||
|
- :exc:`ProgrammingError`
|
||||||
|
- :exc:`NotSupportedError`
|
||||||
|
|
||||||
|
|
||||||
|
.. |StandardError| replace:: ``StandardError``
|
||||||
|
.. _StandardError: http://docs.python.org/library/exceptions.html#exceptions.StandardError
|
||||||
|
|
||||||
|
|
||||||
|
.. _type-objects-and-costructors:
|
||||||
|
|
||||||
|
Type Objects and Constructors
|
||||||
|
-----------------------------
|
||||||
|
|
||||||
|
.. note:: This section is mostly copied verbatim from the |DBAPI 2.0|_
|
||||||
|
specification. While these objects are exposed in compliance to the
|
||||||
|
DBAPI, Psycopg offers very accurate tools to convert data between Python
|
||||||
|
and PostgreSQL formats. See :ref:`adapting-new-types` and
|
||||||
|
:ref:`type-casting-from-sql-to-python`
|
||||||
|
|
||||||
|
Many databases need to have the input in a particular format for
|
||||||
|
binding to an operation's input parameters. For example, if an
|
||||||
|
input is destined for a DATE column, then it must be bound to the
|
||||||
|
database in a particular string format. Similar problems exist
|
||||||
|
for "Row ID" columns or large binary items (e.g. blobs or RAW
|
||||||
|
columns). This presents problems for Python since the parameters
|
||||||
|
to the .execute*() method are untyped. When the database module
|
||||||
|
sees a Python string object, it doesn't know if it should be bound
|
||||||
|
as a simple CHAR column, as a raw BINARY item, or as a DATE.
|
||||||
|
|
||||||
|
To overcome this problem, a module must provide the constructors
|
||||||
|
defined below to create objects that can hold special values.
|
||||||
|
When passed to the cursor methods, the module can then detect the
|
||||||
|
proper type of the input parameter and bind it accordingly.
|
||||||
|
|
||||||
|
A Cursor Object's description attribute returns information about
|
||||||
|
each of the result columns of a query. The type_code must compare
|
||||||
|
equal to one of Type Objects defined below. Type Objects may be
|
||||||
|
equal to more than one type code (e.g. DATETIME could be equal to
|
||||||
|
the type codes for date, time and timestamp columns; see the
|
||||||
|
Implementation Hints below for details).
|
||||||
|
|
||||||
|
The module exports the following constructors and singletons:
|
||||||
|
|
||||||
|
.. function:: Date(year,month,day)
|
||||||
|
|
||||||
|
This function constructs an object holding a date value.
|
||||||
|
|
||||||
|
.. function:: Time(hour,minute,second)
|
||||||
|
|
||||||
|
This function constructs an object holding a time value.
|
||||||
|
|
||||||
|
.. function:: Timestamp(year,month,day,hour,minute,second)
|
||||||
|
|
||||||
|
This function constructs an object holding a time stamp value.
|
||||||
|
|
||||||
|
.. function:: DateFromTicks(ticks)
|
||||||
|
|
||||||
|
This function constructs an object holding a date value from the given
|
||||||
|
ticks value (number of seconds since the epoch; see the documentation of
|
||||||
|
the standard Python time module for details).
|
||||||
|
|
||||||
|
.. function:: TimeFromTicks(ticks)
|
||||||
|
|
||||||
|
This function constructs an object holding a time value from the given
|
||||||
|
ticks value (number of seconds since the epoch; see the documentation of
|
||||||
|
the standard Python time module for details).
|
||||||
|
|
||||||
|
.. function:: TimestampFromTicks(ticks)
|
||||||
|
|
||||||
|
This function constructs an object holding a time stamp value from the
|
||||||
|
given ticks value (number of seconds since the epoch; see the
|
||||||
|
documentation of the standard Python time module for details).
|
||||||
|
|
||||||
|
.. function:: Binary(string)
|
||||||
|
|
||||||
|
This function constructs an object capable of holding a binary (long)
|
||||||
|
string value.
|
||||||
|
|
||||||
|
|
||||||
|
.. data:: STRING
|
||||||
|
|
||||||
|
This type object is used to describe columns in a database that are
|
||||||
|
string-based (e.g. CHAR).
|
||||||
|
|
||||||
|
.. data:: BINARY
|
||||||
|
|
||||||
|
This type object is used to describe (long) binary columns in a database
|
||||||
|
(e.g. LONG, RAW, BLOBs).
|
||||||
|
|
||||||
|
.. data:: NUMBER
|
||||||
|
|
||||||
|
This type object is used to describe numeric columns in a database.
|
||||||
|
|
||||||
|
.. data:: DATETIME
|
||||||
|
|
||||||
|
This type object is used to describe date/time columns in a database.
|
||||||
|
|
||||||
|
.. data:: ROWID
|
||||||
|
|
||||||
|
This type object is used to describe the "Row ID" column in a database.
|
||||||
|
|
18
doc/tz.rst
Normal file
18
doc/tz.rst
Normal file
|
@ -0,0 +1,18 @@
|
||||||
|
:mod:`psycopg2.tz` -- ``tzinfo`` implementations for Psycopg 2
|
||||||
|
===============================================================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. module:: psycopg2.tz
|
||||||
|
|
||||||
|
This module holds two different tzinfo implementations that can be used as the
|
||||||
|
:obj:`tzinfo` argument to datetime constructors, directly passed to psycopg
|
||||||
|
functions or used to set the :attr:`cursor.tzinfo_factory` attribute in
|
||||||
|
cursors.
|
||||||
|
|
||||||
|
.. todo:: tz module
|
||||||
|
|
||||||
|
.. autoclass:: psycopg2.tz.FixedOffsetTimezone
|
||||||
|
|
||||||
|
.. autoclass:: psycopg2.tz.LocalTimezone
|
||||||
|
|
357
doc/usage.rst
Normal file
357
doc/usage.rst
Normal file
|
@ -0,0 +1,357 @@
|
||||||
|
Basic module usage
|
||||||
|
==================
|
||||||
|
|
||||||
|
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
pair: Example; Usage
|
||||||
|
|
||||||
|
The basic psycopg usage is common to all the database adapters implementing
|
||||||
|
the |DBAPI 2.0| protocol. Here is an interactive session showing some of the
|
||||||
|
basic commands::
|
||||||
|
|
||||||
|
>>> import psycopg2
|
||||||
|
|
||||||
|
# Connect to an existing database
|
||||||
|
>>> conn = psycopg2.connect("dbname=test user=postgres")
|
||||||
|
|
||||||
|
# Open a curstor to perform database operations
|
||||||
|
>>> cur = conn.cursor()
|
||||||
|
|
||||||
|
# Execute a command: this creates a new table
|
||||||
|
>>> cur.execute("CREATE TABLE test (id serial PRIMARY KEY, num integer, data varchar);")
|
||||||
|
|
||||||
|
# Pass data to fill a query placeholders and let psycopg perform
|
||||||
|
# the correct conversion (no more SQL injections!)
|
||||||
|
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)",
|
||||||
|
... (100, "abc'def"))
|
||||||
|
|
||||||
|
# Query the database and obtain data as Python objects
|
||||||
|
>>> cur.execute("SELECT * FROM test;")
|
||||||
|
>>> cur.fetchone()
|
||||||
|
(1, 100, "abc'def")
|
||||||
|
|
||||||
|
# Make the changes to the database persistent
|
||||||
|
>>> conn.commit()
|
||||||
|
|
||||||
|
# Close communication with the database
|
||||||
|
>>> cur.close()
|
||||||
|
>>> conn.close()
|
||||||
|
|
||||||
|
|
||||||
|
The main entry point of Psycopg are:
|
||||||
|
|
||||||
|
- The function :func:`psycopg2.connect()` creates a new database session and
|
||||||
|
returns a new :class:`connection` instance.
|
||||||
|
|
||||||
|
- The class :class:`connection` encapsulates a database session. It allows to:
|
||||||
|
|
||||||
|
- terminate the session using the methods :meth:`connection.commit()` and
|
||||||
|
:meth:`connection.rollback()`,
|
||||||
|
|
||||||
|
- create new :class:`cursor`\ s to execute database commands and queries
|
||||||
|
using the method :meth:`connection.cursor()`.
|
||||||
|
|
||||||
|
- The class :class:`cursor` allows interaction with the database:
|
||||||
|
|
||||||
|
- send command using the methods :meth:`cursor.execute()` and
|
||||||
|
:meth:`cursor.executemany()`,
|
||||||
|
|
||||||
|
- retrieve data using the methods :meth:`cursor.fetchone()`,
|
||||||
|
:meth:`cursor.fetchmany()`, :meth:`cursor.fetchall()`.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index:: Security, SQL injection
|
||||||
|
|
||||||
|
.. _sql-injection:
|
||||||
|
|
||||||
|
The problem with the query parameters
|
||||||
|
-------------------------------------
|
||||||
|
|
||||||
|
The SQL representation for many data types is often not the same of the Python
|
||||||
|
string representation. The classic example is with single quotes in the
|
||||||
|
strings: SQL uses them as string constants bounds and requires them to be
|
||||||
|
escaped, whereas in Python single quotes can be left unescaped in strings
|
||||||
|
bounded by double quotes. For this reason a naïve approach to the composition
|
||||||
|
of query strings, e.g. using string concatenation, is a recipe for terrible
|
||||||
|
problems::
|
||||||
|
|
||||||
|
>>> SQL = "INSERT INTO authors (name) VALUES ('%s');" # NEVER DO THIS
|
||||||
|
>>> data = ("O'Reilly", )
|
||||||
|
>>> cur.execute(SQL % data) # THIS WILL FAIL MISERABLY
|
||||||
|
ProgrammingError: syntax error at or near "Really"
|
||||||
|
LINE 1: INSERT INTO authors (name) VALUES ('O'Really')
|
||||||
|
^
|
||||||
|
|
||||||
|
If the variable containing the data to be sent to the database comes from an
|
||||||
|
untrusted source (e.g. a form published on a web site) an attacker could
|
||||||
|
easily craft a malformed string either gaining access to unauthorized data or
|
||||||
|
performing destructive operations on the database. This form of attack is
|
||||||
|
called `SQL injection`_ and is known to be one of the most widespread forms of
|
||||||
|
attack to servers. Before continuing, please print `this page`__ as a memo and
|
||||||
|
hang it onto your desktop.
|
||||||
|
|
||||||
|
.. _SQL injection: http://en.wikipedia.org/wiki/SQL_injection
|
||||||
|
.. __: http://xkcd.com/327/
|
||||||
|
|
||||||
|
Psycopg can `convert automatically Python objects into and from SQL
|
||||||
|
literals`__: using this feature your code will result more robust and
|
||||||
|
reliable. It is really the case to stress this point:
|
||||||
|
|
||||||
|
.. __: python-types-adaptation_
|
||||||
|
|
||||||
|
.. warning::
|
||||||
|
|
||||||
|
Never, **never**, **NEVER** use Python string concatenation (``+``) or
|
||||||
|
string parameters interpolation (``%``) to pass variables to a SQL query
|
||||||
|
string. Not even at gunpoint.
|
||||||
|
|
||||||
|
The correct way to pass variables in a SQL command is using the second
|
||||||
|
argument of the :meth:`cursor.execute()` method::
|
||||||
|
|
||||||
|
>>> SQL = "INSERT INTO authors (name) VALUES (%s);" # Notice: no quotes
|
||||||
|
>>> data = ("O'Reilly", )
|
||||||
|
>>> cur.execute(SQL, data) # Notice: no % operator
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
pair: Query; Parameters
|
||||||
|
|
||||||
|
.. _query-parameters:
|
||||||
|
|
||||||
|
Passing parameters to SQL queries
|
||||||
|
---------------------------------
|
||||||
|
|
||||||
|
Psycopg casts Python variables to SQL literals by type. `Standard Python types
|
||||||
|
are already adapted to the proper SQL literal`__.
|
||||||
|
|
||||||
|
.. __: python-types-adaptation_
|
||||||
|
|
||||||
|
Example: the Python function call::
|
||||||
|
|
||||||
|
>>> cur.execute(
|
||||||
|
... """INSERT INTO some_table (an_int, a_date, a_string)
|
||||||
|
... VALUES (%s, %s, %s);""",
|
||||||
|
... (10, datetime.date(2005, 11, 18), "O'Reilly"))
|
||||||
|
|
||||||
|
is converted into the SQL command::
|
||||||
|
|
||||||
|
INSERT INTO some_table (an_int, a_date, a_string)
|
||||||
|
VALUES (10, '2005-11-18', 'O''Reilly');
|
||||||
|
|
||||||
|
Named arguments are supported too using ``%(name)s`` placeholders. Using named
|
||||||
|
arguments the values can be passed to the query in any order and many
|
||||||
|
placeholder can use the the same values::
|
||||||
|
|
||||||
|
>>> cur.execute(
|
||||||
|
... """INSERT INTO some_table (an_int, a_date, another_date, a_string)
|
||||||
|
... VALUES (%(int)s, %(date)s, %(date)s, %(str)s);""",
|
||||||
|
... {'int': 10, 'str': "O'Reilly", 'date': datetime.date(2005, 11, 18)})
|
||||||
|
|
||||||
|
Notice that:
|
||||||
|
|
||||||
|
- The Python string operator ``%`` is not used: the :meth:`cursor.execute()`
|
||||||
|
method accepts a tuple or dictionary of values as second parameter.
|
||||||
|
|sql-warn|__.
|
||||||
|
|
||||||
|
.. |sql-warn| replace:: **Never** use ``%`` or ``+`` to merge values
|
||||||
|
into queries
|
||||||
|
|
||||||
|
.. __: sql-injection_
|
||||||
|
|
||||||
|
- The variables placeholder must always be a ``%s``, even if a different
|
||||||
|
placeholder (such as a ``%d`` for an integer) may look more appropriate::
|
||||||
|
|
||||||
|
>>> cur.execute("INSERT INTO numbers VALUES (%d)", (42,)) # WRONG
|
||||||
|
>>> cur.execute("INSERT INTO numbers VALUES (%s)", (42,)) # correct
|
||||||
|
|
||||||
|
- For positional variables binding, the second argument must always be a
|
||||||
|
tuple, even if it contains a single variable::
|
||||||
|
|
||||||
|
>>> cur.execute("INSERT INTO foo VALUES (%s)", "bar") # WRONG
|
||||||
|
>>> cur.execute("INSERT INTO foo VALUES (%s)", ("bar",)) # correct
|
||||||
|
|
||||||
|
- Only variable values should be bound via this method: it shouldn't be used
|
||||||
|
to set table or field names. For these elements, ordinary string formatting
|
||||||
|
should be used before running :meth:`cursor.execute()`.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
pair: Objects; Adaptation
|
||||||
|
single: Data types; Adaptation
|
||||||
|
|
||||||
|
.. _python-types-adaptation:
|
||||||
|
|
||||||
|
Adaptation of Python values to SQL types
|
||||||
|
----------------------------------------
|
||||||
|
|
||||||
|
Many standards Python types are adapted into SQL and returned as Python
|
||||||
|
objects when a query is executed.
|
||||||
|
|
||||||
|
If you need to convert other Python types to and from PostgreSQL data types,
|
||||||
|
see :ref:`adapting-new-types` and :ref:`type-casting-from-sql-to-python`.
|
||||||
|
|
||||||
|
In the following examples the method :meth:`cursor.mogrify()` is used to show
|
||||||
|
the SQL string that would be sent to the database.
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: None; Adaptation
|
||||||
|
single: NULL; Adaptation
|
||||||
|
single: Boolean; Adaptation
|
||||||
|
|
||||||
|
- Python ``None`` and boolean values are converted into the proper SQL
|
||||||
|
literals::
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s, %s, %s;", (None, True, False))
|
||||||
|
>>> 'SELECT NULL, true, false;'
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Integer; Adaptation
|
||||||
|
single: Float; Adaptation
|
||||||
|
single: Decimal; Adaptation
|
||||||
|
|
||||||
|
- Numeric objects: ``int``, ``long``, ``float``, ``Decimal`` are converted in
|
||||||
|
the PostgreSQL numerical representation::
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s, %s, %s, %s;", (10, 10L, 10.0, Decimal("10.00")))
|
||||||
|
>>> 'SELECT 10, 10, 10.0, 10.00;'
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Strings; Adaptation
|
||||||
|
single: Unicode; Adaptation
|
||||||
|
single: Buffer; Adaptation
|
||||||
|
single: bytea; Adaptation
|
||||||
|
single: Binary string
|
||||||
|
|
||||||
|
- String types: ``str``, ``unicode`` are converted in SQL string syntax.
|
||||||
|
``buffer`` is converted in PostgreSQL binary string syntax, suitable for
|
||||||
|
``bytea`` fields.
|
||||||
|
|
||||||
|
.. todo:: unicode not working?
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Date objects; Adaptation
|
||||||
|
single: Time objects; Adaptation
|
||||||
|
single: Interval objects; Adaptation
|
||||||
|
single: mx.DateTime; Adaptation
|
||||||
|
|
||||||
|
- Date and time objects: ``datetime.datetime``, ``datetime.date``,
|
||||||
|
``datetime.time``. ``datetime.timedelta`` are converted into PostgreSQL's
|
||||||
|
``timestamp``, ``date``, ``time``, ``interval`` data types. Time zones are
|
||||||
|
supported too. The Egenix `mx.DateTime`_ objects are adapted the same way::
|
||||||
|
|
||||||
|
>>> dt = datetime.datetime.now()
|
||||||
|
>>> dt
|
||||||
|
datetime.datetime(2010, 2, 8, 1, 40, 27, 425337)
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s, %s, %s;", (dt, dt.date(), dt.time()))
|
||||||
|
"SELECT '2010-02-08T01:40:27.425337', '2010-02-08', '01:40:27.425337';"
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s;", (dt - datetime.datetime(2010,1,1),))
|
||||||
|
"SELECT '38 days 6027.425337 seconds';"
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Array; Adaptation
|
||||||
|
single: Lists; Adaptation
|
||||||
|
|
||||||
|
- Python lists are converted into PostgreSQL arrays::
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s;", ([10, 20, 30], ))
|
||||||
|
'SELECT ARRAY[10, 20, 30];'
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
single: Tuple; Adaptation
|
||||||
|
single: IN operator
|
||||||
|
|
||||||
|
- Python tuples are converted in a syntax suitable for the SQL ``IN``
|
||||||
|
operator::
|
||||||
|
|
||||||
|
>>> cur.mogrify("SELECT %s IN %s;", (10, (10, 20, 30)))
|
||||||
|
'SELECT 10 IN (10, 20, 30);'
|
||||||
|
|
||||||
|
.. note::
|
||||||
|
|
||||||
|
SQL doesn't allow an empty list in the IN operator, so your code should
|
||||||
|
guard against empty tuples.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
pair: Server side; Cursor
|
||||||
|
pair: Named; Cursor
|
||||||
|
pair: DECLARE; SQL command
|
||||||
|
pair: FETCH; SQL command
|
||||||
|
pair: MOVE; SQL command
|
||||||
|
|
||||||
|
.. _server-side-cursors:
|
||||||
|
|
||||||
|
Server side cursors
|
||||||
|
-------------------
|
||||||
|
|
||||||
|
When a database query is executed, the Psycopg :class:`cursor` usually fetches
|
||||||
|
all the returned records, transferring them to the client process. If the
|
||||||
|
query returned an huge amount of data, a proportionally large amount of memory
|
||||||
|
will be allocated by the client.
|
||||||
|
|
||||||
|
If the dataset is too large to be pratically handled on the client side, it is
|
||||||
|
possible to create a *server side* cursor. Using this kind of cursor it is
|
||||||
|
possible to transfer to the client only a controlled amount of data, so that a
|
||||||
|
large dataset can be examined without keeping it entirely in memory.
|
||||||
|
|
||||||
|
Server side cursor are created in PostgreSQL using the |DECLARE|_ command and
|
||||||
|
subsequently handled using ``MOVE``, ``FETCH`` and ``CLOSE`` commands.
|
||||||
|
|
||||||
|
Psycopg wraps the database server side cursor in *named cursors*. A name
|
||||||
|
cursor is created using the :meth:`connection.cursor` method specifying the
|
||||||
|
:obj:`name` parameter. Such cursor will behave mostly like a regular cursor,
|
||||||
|
allowing the user to move in the dataset using the :meth:`cursor.scroll`
|
||||||
|
methog and to read the data using :meth:`cursor.fetchone` and
|
||||||
|
:meth:`cursor.fetchmany` methods.
|
||||||
|
|
||||||
|
.. |DECLARE| replace:: ``DECLARE``
|
||||||
|
.. _DECLARE: http://www.postgresql.org/docs/8.4/static/sql-declare.html
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index:: Thread safety, Multithread
|
||||||
|
|
||||||
|
.. _thread-safety:
|
||||||
|
|
||||||
|
Thread safety
|
||||||
|
-------------
|
||||||
|
|
||||||
|
The Psycopg module is *thread-safe*: threads can access the same database
|
||||||
|
using separate session (by creating a :class:`connection` per thread) or using
|
||||||
|
the same session (accessing to the same connection and creating separate
|
||||||
|
:class:`cursor`\ s). In |DBAPI 2.0|_ parlance, Psycopg is *level 2 thread safe*.
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
.. index::
|
||||||
|
pair: COPY; SQL command
|
||||||
|
|
||||||
|
.. _copy:
|
||||||
|
|
||||||
|
Using COPY TO and COPY FROM
|
||||||
|
---------------------------
|
||||||
|
|
||||||
|
Psycopg :class:`cursor` objects provide an interface to the efficient
|
||||||
|
PostgreSQL `COPY command`_ to move data from files to tables and back.
|
||||||
|
|
||||||
|
The :meth:`cursor.copy_to()` method writes the content of a table *to* a
|
||||||
|
file-like object. The target file must have a ``write()`` method.
|
||||||
|
|
||||||
|
The :meth:`cursor.copy_from()` reads data *from* a file-like object appending
|
||||||
|
them to a database table. The source file must have both ``read()`` and
|
||||||
|
``readline()`` method.
|
||||||
|
|
||||||
|
Both methods accept two optional arguments: ``sep`` (defaulting to a tab) is
|
||||||
|
the columns separator and ``null`` (defaulting to ``\N``) represents ``NULL``
|
||||||
|
values in the file.
|
||||||
|
|
||||||
|
.. _COPY command: http://www.postgresql.org/docs/8.4/static/sql-copy.html
|
||||||
|
|
||||||
|
|
Loading…
Reference in New Issue
Block a user