Compare commits

..

No commits in common. "master" and "2_0_5_1" have entirely different histories.

267 changed files with 14062 additions and 44695 deletions

4
.github/FUNDING.yml vendored
View File

@ -1,4 +0,0 @@
github:
- dvarrazzo
custom:
- "https://www.paypal.me/dvarrazzo"

View File

@ -1,23 +0,0 @@
---
name: Problem installing psycopg2
about: Report a case in which psycopg2 failed to install on your platform
title: ''
labels: ''
assignees: ''
---
**This is a bug tracker**
If you have a question, such has "how do you do X with Python/PostgreSQL/psycopg2" please [write to the mailing list](https://lists.postgresql.org/manage/) or [open a question](https://github.com/psycopg/psycopg2/discussions) instead.
**Before opening this ticket, please confirm that:**
- [ ] I am running the latest version of pip, i.e. typing ``pip --version`` you get [this version](https://pypi.org/project/pip/).
- [ ] I have read the [installation documentation](https://www.psycopg.org/docs/install.html) and the [frequently asked questions](https://www.psycopg.org/docs/faq.html)
- [ ] If install failed, I typed `pg_config` on the command line and I obtained an output instead of an error.
**Please complete the following information:**
- OS:
- Psycopg version:
- Python version:
- PostgreSQL version:
- pip version

View File

@ -1,27 +0,0 @@
---
name: Problem using psycopg2
about: Report a case in which psycopg2 is not working as expected
title: ''
labels: ''
assignees: ''
---
**This is a bug tracker**
If you have a question, such has "how do you do X with Python/PostgreSQL/psycopg2" please [write to the mailing list](https://lists.postgresql.org/manage/) or [open a question](https://github.com/psycopg/psycopg2/discussions) instead.
**Please complete the following information:**
- OS:
- Psycopg version:
- Python version:
- PostgreSQL version:
- pip version
**Describe the bug**
Please let us know:
1: what you did
2: what you expected to happen
3: what happened instead
If possible, provide a script reproducing the issue.

View File

@ -1,6 +0,0 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "monthly"

View File

@ -1,18 +0,0 @@
name: Build documentation
on:
push:
branches:
# This should match the DOC_BRANCH value in the psycopg-website Makefile
- master
jobs:
docs:
runs-on: ubuntu-latest
steps:
- name: Trigger docs build
uses: peter-evans/repository-dispatch@v3
with:
repository: psycopg/psycopg-website
event-type: psycopg2-commit
token: ${{ secrets.ACCESS_TOKEN }}

View File

@ -1,266 +0,0 @@
---
name: Build packages
on:
- workflow_dispatch
env:
PIP_BREAK_SYSTEM_PACKAGES: "1"
LIBPQ_VERSION: "16.0"
OPENSSL_VERSION: "1.1.1w"
jobs:
sdist: # {{{
if: true
strategy:
fail-fast: false
matrix:
include:
- package_name: psycopg2
- package_name: psycopg2-binary
runs-on: ubuntu-latest
steps:
- name: Checkout repos
uses: actions/checkout@v4
- name: Build sdist
run: ./scripts/build/build_sdist.sh
env:
PACKAGE_NAME: ${{ matrix.package_name }}
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: sdist-${{ matrix.package_name }}
path: |
dist/*.tar.gz
env:
PSYCOPG2_TESTDB: postgres
PSYCOPG2_TESTDB_HOST: 172.17.0.1
PSYCOPG2_TESTDB_USER: postgres
PSYCOPG2_TESTDB_PASSWORD: password
PSYCOPG2_TEST_FAST: 1
services:
postgresql:
image: postgres:16
env:
POSTGRES_PASSWORD: password
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
# }}}
linux: # {{{
if: true
strategy:
fail-fast: false
matrix:
platform: [manylinux, musllinux]
arch: [x86_64, i686, aarch64, ppc64le]
pyver: [cp38, cp39, cp310, cp311, cp312, cp313]
runs-on: ubuntu-latest
steps:
- name: Checkout repos
uses: actions/checkout@v4
- name: Set up QEMU for multi-arch build
uses: docker/setup-qemu-action@v3
- name: Cache libpq build
uses: actions/cache@v4
with:
path: /tmp/libpq.build
key: libpq-${{ env.LIBPQ_VERSION }}-${{ matrix.platform }}-${{ matrix.arch }}
- name: Build wheels
uses: pypa/cibuildwheel@v2.23.2
env:
CIBW_MANYLINUX_X86_64_IMAGE: manylinux2014
CIBW_MANYLINUX_I686_IMAGE: manylinux2014
CIBW_MANYLINUX_AARCH64_IMAGE: manylinux2014
CIBW_MANYLINUX_PPC64LE_IMAGE: manylinux2014
CIBW_BUILD: ${{matrix.pyver}}-${{matrix.platform}}_${{matrix.arch}}
CIBW_ARCHS_LINUX: auto aarch64 ppc64le
CIBW_BEFORE_ALL_LINUX: ./scripts/build/wheel_linux_before_all.sh
CIBW_REPAIR_WHEEL_COMMAND: >-
./scripts/build/strip_wheel.sh {wheel}
&& auditwheel repair -w {dest_dir} {wheel}
CIBW_TEST_COMMAND: >-
export PYTHONPATH={project} &&
python -c "import tests; tests.unittest.main(defaultTest='tests.test_suite')"
CIBW_ENVIRONMENT_PASS_LINUX: LIBPQ_VERSION OPENSSL_VERSION
CIBW_ENVIRONMENT: >-
PACKAGE_NAME=psycopg2-binary
LIBPQ_BUILD_PREFIX=/host/tmp/libpq.build
PATH="$LIBPQ_BUILD_PREFIX/bin:$PATH"
LD_LIBRARY_PATH="$LIBPQ_BUILD_PREFIX/lib:$LIBPQ_BUILD_PREFIX/lib64"
PSYCOPG2_TESTDB=postgres
PSYCOPG2_TESTDB_HOST=172.17.0.1
PSYCOPG2_TESTDB_USER=postgres
PSYCOPG2_TESTDB_PASSWORD=password
PSYCOPG2_TEST_FAST=1
- uses: actions/upload-artifact@v4
with:
name: linux-${{matrix.pyver}}-${{matrix.platform}}_${{matrix.arch}}
path: ./wheelhouse/*.whl
services:
postgresql:
image: postgres:16
env:
POSTGRES_PASSWORD: password
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
# }}}
macos: # {{{
runs-on: macos-latest
if: true
strategy:
fail-fast: false
matrix:
# These archs require an Apple M1 runner: [arm64, universal2]
arch: [x86_64, arm64]
pyver: [cp39, cp310, cp311, cp312, cp313]
steps:
- name: Checkout repos
uses: actions/checkout@v4
- name: Cache libpq build
uses: actions/cache@v4
with:
path: /tmp/libpq.build
key: libpq-${{ env.LIBPQ_VERSION }}-macos-${{ matrix.arch }}
- name: Build wheels
uses: pypa/cibuildwheel@v2.23.2
env:
CIBW_BUILD: ${{matrix.pyver}}-macosx_${{matrix.arch}}
CIBW_ARCHS_MACOS: ${{matrix.arch}}
MACOSX_ARCHITECTURE: ${{matrix.arch}}
CIBW_BEFORE_ALL_MACOS: ./scripts/build/wheel_macos_before_all.sh
CIBW_TEST_COMMAND: >-
export PYTHONPATH={project} &&
python -c "import tests; tests.unittest.main(defaultTest='tests.test_suite')"
CIBW_ENVIRONMENT: >-
PG_VERSION=16
PACKAGE_NAME=psycopg2-binary
PSYCOPG2_TESTDB=postgres
PATH="/tmp/libpq.build/bin:$PATH"
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: macos-${{matrix.pyver}}-macos-${{matrix.arch}}
path: ./wheelhouse/*.whl
# }}}
windows: # {{{
runs-on: windows-latest
if: true
strategy:
fail-fast: false
matrix:
arch: [win_amd64]
pyver: [cp38, cp39, cp310, cp311, cp312, cp313]
package_name: [psycopg2, psycopg2-binary]
defaults:
run:
shell: bash
steps:
# there are some other libpq in PATH
- name: Drop spurious libpq in the path
run: rm -rf c:/tools/php C:/Strawberry/c/bin
- name: Checkout repo
uses: actions/checkout@v4
- name: Start PostgreSQL service for test
run: |
$PgSvc = Get-Service "postgresql*"
Set-Service $PgSvc.Name -StartupType manual
$PgSvc.Start()
shell: powershell
- name: Export GitHub Actions cache environment variables
uses: actions/github-script@v7
with:
script: |
const path = require('path')
core.exportVariable('ACTIONS_CACHE_URL', process.env.ACTIONS_CACHE_URL || '');
core.exportVariable('ACTIONS_RUNTIME_TOKEN', process.env.ACTIONS_RUNTIME_TOKEN || '');
core.addPath(path.join(process.env.VCPKG_INSTALLATION_ROOT, 'installed/x64-windows-release/lib'));
core.addPath(path.join(process.env.VCPKG_INSTALLATION_ROOT, 'installed/x64-windows-release/bin'));
- name: Create the binary package source tree
run: >-
sed -i 's/^setup(name="psycopg2"/setup(name="${{matrix.package_name}}"/'
setup.py
if: ${{ matrix.package_name != 'psycopg2' }}
- name: Build wheels
uses: pypa/cibuildwheel@v2.23.2
env:
VCPKG_BINARY_SOURCES: "clear;x-gha,readwrite" # cache vcpkg
CIBW_BUILD: ${{matrix.pyver}}-${{matrix.arch}}
CIBW_ARCHS_WINDOWS: AMD64 x86
CIBW_BEFORE_BUILD_WINDOWS: '.\scripts\build\wheel_win32_before_build.bat'
CIBW_REPAIR_WHEEL_COMMAND_WINDOWS: >-
delvewheel repair -w {dest_dir}
--no-mangle "libiconv-2.dll;libwinpthread-1.dll" {wheel}
CIBW_TEST_COMMAND: >-
set PYTHONPATH={project} &&
python -c "import tests; tests.unittest.main(defaultTest='tests.test_suite')"
# Note: no fast test because we don't run Windows tests
CIBW_ENVIRONMENT_WINDOWS: >-
PSYCOPG2_TESTDB=postgres
PSYCOPG2_TESTDB_USER=postgres
PSYCOPG2_TESTDB_HOST=localhost
- name: Upload artifacts
uses: actions/upload-artifact@v4
with:
name: windows-${{ matrix.package_name }}-${{matrix.pyver}}-${{matrix.arch}}
path: ./wheelhouse/*.whl
# }}}
merge: # {{{
runs-on: ubuntu-latest
needs:
- sdist
- linux
- macos
- windows
steps:
- name: Merge Artifacts
uses: actions/upload-artifact/merge@v4
with:
name: psycopg2-artifacts
delete-merged: true
# }}}

View File

@ -1,79 +0,0 @@
name: Tests
env:
PIP_BREAK_SYSTEM_PACKAGES: "1"
on:
push:
pull_request:
jobs:
linux:
runs-on: ubuntu-latest
if: true
strategy:
fail-fast: false
matrix:
include:
- {python: "3.8", postgres: "12"}
- {python: "3.9", postgres: "13"}
- {python: "3.10", postgres: "14"}
- {python: "3.11", postgres: "15"}
- {python: "3.12", postgres: "16"}
- {python: "3.13", postgres: "17"}
# Opposite extremes of the supported Py/PG range, other architecture
- {python: "3.8", postgres: "17", architecture: "x86"}
- {python: "3.9", postgres: "16", architecture: "x86"}
- {python: "3.10", postgres: "15", architecture: "x86"}
- {python: "3.11", postgres: "14", architecture: "x86"}
- {python: "3.12", postgres: "13", architecture: "x86"}
- {python: "3.13", postgres: "12", architecture: "x86"}
env:
PSYCOPG2_TESTDB: postgres
PSYCOPG2_TESTDB_HOST: 127.0.0.1
PSYCOPG2_TESTDB_USER: postgres
PSYCOPG2_TESTDB_PASSWORD: password
services:
postgresql:
image: postgres:${{ matrix.postgres }}
env:
POSTGRES_PASSWORD: password
ports:
- 5432:5432
# Set health checks to wait until postgres has started
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
steps:
- uses: actions/checkout@v4
# Can enable to test an unreleased libpq version.
- name: install libpq 16
if: false
run: |
set -x
rel=$(lsb_release -c -s)
echo "deb http://apt.postgresql.org/pub/repos/apt ${rel}-pgdg main 16" \
| sudo tee -a /etc/apt/sources.list.d/pgdg.list
sudo apt-get -qq update
pqver=$(apt-cache show libpq5 | grep ^Version: | head -1 \
| awk '{print $2}')
sudo apt-get -qq -y install "libpq-dev=${pqver}" "libpq5=${pqver}"
- name: Install tox
run: pip install "tox < 4"
- uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python }}
- name: Run tests
env:
MATRIX_PYTHON: ${{ matrix.python }}
run: tox -e ${MATRIX_PYTHON%-dev}
timeout-minutes: 5

18
.gitignore vendored
View File

@ -1,18 +0,0 @@
MANIFEST
*~
*.userprefs
*.pidb
*.pyc
*.sw[po]
*.egg-info/
dist/*
/build
env
env?
.idea
.tox
.vscode/
/rel
/wheels
/packages
/wheelhouse

11
AUTHORS
View File

@ -1,15 +1,8 @@
Main authors:
Federico Di Gregorio <fog@debian.org>
Daniele Varrazzo <daniele.varrazzo@gmail.com>
For the win32 port:
Jason Erickson <jerickso@indian.com>
Jason Erickson <jerickso@indian.com> (most of his changes are still in 2.0)
Additional Help:
Peter Fein contributed a logging connection/cursor class that even if it
was not used directly heavily influenced the implementation currently in
psycopg2.extras.
Jan Urbański (re)started the work on asynchronous queries and contributed
both on that and on other parts of psycopg2.

1219
ChangeLog Normal file

File diff suppressed because it is too large Load Diff

50
INSTALL
View File

@ -1,4 +1,48 @@
Installation instructions are included in the docs.
Compiling and installing psycopg
********************************
Please check the 'doc/src/install.rst' file or online at
<https://www.psycopg.org/docs/install.html>.
** Important note: if you plan to use psyopg2 in a multithreaed application
make sure that your libpq has been compiled with the --with-thread-safety
option. psycopg2 will work correctly even with a non-thread-safe libpq but
libpq will leak memory.
While psycopg 1.x used autoconf for its build process psycopg 2 switched to
the more pythoning setup.py. Currently both psycopg's author and distutils
have some limitations so the file setup.cfg is almost unused and most build
options are hidden in setup.py. Before building psycopg look at setup.cfg file
and change any settings to follow your system (or taste); then:
python setup.py build
to build in the local directory; and:
python setup.py install
to install system-wide.
Using setuptools and EasyInstall
================================
If setuptools are installed on your system you can easily create an egg for
psycopg and install it. Download the source distribution (if you're reading
this file you probably already have) and then edit setup.cfg to your taste
and build from the source distribution top-level directory using:
easy_install .
Compiling under Windows with mingw32
====================================
You can compile psycopg under Windows platform with mingw32
(http://www.mingw.org/) compiler. MinGW is also shipped with IDEs such as
Dev-C++ (http://www.bloodshed.net/devcpp.html) and Code::Blocks
(http://www.codeblocks.org). gcc binaries should be in your PATH.
You need a PostgreSQL with include and libary files installed. At least v8.0 is required.
First you need to create a libpython2X.a as described in
http://starship.python.net/crew/kernr/mingw32/Notes.html. Then run:
python setup.py build_ext --compiler=mingw32 install

69
LICENSE
View File

@ -1,38 +1,35 @@
psycopg2 and the LGPL
---------------------
psycopg and the GPL
===================
psycopg2 is free software: you can redistribute it and/or modify it
under the terms of the GNU Lesser General Public License as published
by the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
psycopg is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version. See file COPYING for details.
psycopg2 is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public
License for more details.
As a special exception, specific permission is granted for the GPLed
code in this distribition to be linked to OpenSSL and PostgreSQL libpq
without invoking GPL clause 2(b).
In addition, as a special exception, the copyright holders give
permission to link this program with the OpenSSL library (or with
modified versions of OpenSSL that use the same license as OpenSSL),
and distribute linked combinations including the two.
Note that the GPL was chosen to avoid proprietary adapters based on
psycopg code. Using psycopg in a proprietary product (even bundling
psycopg with the proprietary product) is fine as long as:
You must obey the GNU Lesser General Public License in all respects for
all of the code used other than OpenSSL. If you modify file(s) with this
exception, you may extend this exception to your version of the file(s),
but you are not obligated to do so. If you do not wish to do so, delete
this exception statement from your version. If you delete this exception
statement from all source files in the program, then also delete it here.
You should have received a copy of the GNU Lesser General Public License
along with psycopg2 (see the doc/ directory.)
If not, see <https://www.gnu.org/licenses/>.
1. psycopg is called from Python only using only the provided API
(i.e., no linking with C code and no C modules based on it); and
2. all the other points of the GPL are respected (you offer a copy
of psycopg's source code, and so on.)
Alternative licenses
--------------------
====================
The following BSD-like license applies (at your option) to the files following
the pattern ``psycopg/adapter*.{h,c}`` and ``psycopg/microprotocol*.{h,c}``:
If you prefer you can use the Zope Database Adapter ZPsycopgDA (i.e.,
every file inside the ZPsycopgDA directory) user the ZPL license as
published on the Zope web site, http://www.zope.org/Resources/ZPL.
Also, the following BSD-like license applies (at your option) to the
files following the pattern psycopg/adapter*.{h,c} and
psycopg/microprotocol*.{h,c}:
Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
@ -42,8 +39,22 @@ the pattern ``psycopg/adapter*.{h,c}`` and ``psycopg/microprotocol*.{h,c}``:
claim that you wrote the original software. If you use this
software in a product, an acknowledgment in the product documentation
would be appreciated but is not required.
2. Altered source versions must be plainly marked as such, and must not
be misrepresented as being the original software.
3. This notice may not be removed or altered from any source distribution.
psycopg is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
Proprietary licenses
====================
A non-exclusive license is available for companies that want to include
psycopg in their proprietary products without respecting the spirit of the
GPL. The price of the license is one day of development done by the author,
at the consulting fee he applies to his usual customers at the day of the
request.

View File

@ -1,9 +1,12 @@
recursive-include psycopg *.c *.h *.manifest
recursive-include psycopg *.c *.h
recursive-include lib *.py
recursive-include tests *.py
include doc/README.rst doc/SUCCESS doc/COPYING.LESSER doc/pep-0249.txt
include doc/Makefile doc/requirements.txt
recursive-include doc/src *.rst *.py *.css Makefile
recursive-include ZPsycopgDA *.py *.gif *.dtml
recursive-include examples *.py somehackers.jpg whereareyou.jpg
recursive-include debian *
recursive-include doc TODO HACKING SUCCESS ChangeLog-1.x async.txt
recursive-include scripts *.py *.sh
include AUTHORS README.rst INSTALL LICENSE NEWS
include MANIFEST.in setup.py setup.cfg Makefile
include scripts/maketypes.sh scripts/buildtypes.py
include AUTHORS README INSTALL LICENSE ChangeLog
include PKG-INFO MANIFEST.in MANIFEST setup.py setup.cfg
recursive-include doc *.rst *.css *.html

104
Makefile
View File

@ -1,104 +0,0 @@
# Makefile for psycopg2. Do you want to...
#
# Build the library::
#
# make
#
# Build the documentation::
#
# make env (once)
# make docs
#
# Create a source package::
#
# make sdist
#
# Run the test::
#
# make check # this requires setting up a test database with the correct user
PYTHON := python$(PYTHON_VERSION)
PYTHON_VERSION ?= $(shell $(PYTHON) -c 'import sys; print ("%d.%d" % sys.version_info[:2])')
BUILD_DIR = $(shell pwd)/build/lib.$(PYTHON_VERSION)
SOURCE_C := $(wildcard psycopg/*.c psycopg/*.h)
SOURCE_PY := $(wildcard lib/*.py)
SOURCE_TESTS := $(wildcard tests/*.py)
SOURCE_DOC := $(wildcard doc/src/*.rst)
SOURCE := $(SOURCE_C) $(SOURCE_PY) $(SOURCE_TESTS) $(SOURCE_DOC)
PACKAGE := $(BUILD_DIR)/psycopg2
PLATLIB := $(PACKAGE)/_psycopg.so
PURELIB := $(patsubst lib/%,$(PACKAGE)/%,$(SOURCE_PY))
BUILD_OPT := --build-lib=$(BUILD_DIR)
BUILD_EXT_OPT := --build-lib=$(BUILD_DIR)
SDIST_OPT := --formats=gztar
ifdef PG_CONFIG
BUILD_EXT_OPT += --pg-config=$(PG_CONFIG)
endif
VERSION := $(shell grep PSYCOPG_VERSION setup.py | head -1 | sed -e "s/.*'\(.*\)'/\1/")
SDIST := dist/psycopg2-$(VERSION).tar.gz
.PHONY: check clean
default: package
all: package sdist
package: $(PLATLIB) $(PURELIB)
docs: docs-html
docs-html: doc/html/genindex.html
# for PyPI documentation
docs-zip: doc/docs.zip
sdist: $(SDIST)
env:
$(MAKE) -C doc $@
check:
PYTHONPATH=$(BUILD_DIR) $(PYTHON) -c "import tests; tests.unittest.main(defaultTest='tests.test_suite')" --verbose
testdb:
@echo "* Creating $(TESTDB)"
@if psql -l | grep -q " $(TESTDB) "; then \
dropdb $(TESTDB) >/dev/null; \
fi
createdb $(TESTDB)
# Note to packagers: this requires the postgres user running the test
# to be a superuser. You may change this line to use the superuser only
# to install the contrib. Feel free to suggest a better way to set up the
# testing environment (as the current is enough for development).
psql -f `pg_config --sharedir`/contrib/hstore.sql $(TESTDB)
$(PLATLIB): $(SOURCE_C)
$(PYTHON) setup.py build_ext $(BUILD_EXT_OPT)
$(PACKAGE)/%.py: lib/%.py
$(PYTHON) setup.py build_py $(BUILD_OPT)
touch $@
$(PACKAGE)/tests/%.py: tests/%.py
$(PYTHON) setup.py build_py $(BUILD_OPT)
touch $@
$(SDIST): $(SOURCE)
$(PYTHON) setup.py sdist $(SDIST_OPT)
# docs depend on the build as it partly use introspection.
doc/html/genindex.html: $(PLATLIB) $(PURELIB) $(SOURCE_DOC)
$(MAKE) -C doc html
doc/docs.zip: doc/html/genindex.html
(cd doc/html && zip -r ../docs.zip *)
clean:
rm -rf build
$(MAKE) -C doc clean

1283
NEWS

File diff suppressed because it is too large Load Diff

39
README Normal file
View File

@ -0,0 +1,39 @@
psycopg - Python-PostgreSQL Database Adapter
********************************************
psycopg is a PostgreSQL database adapter for the Python programming
language. This is version 2, a complete rewrite of the original code to
provide new-style classes for connection and cursor objects and other
sweet candies. Like the original, psycopg 2 was written with the aim of
being very small and fast, and stable as a rock.
psycopg is different from the other database adapter because it was
designed for heavily multi-threaded applications that create and destroy
lots of cursors and make a conspicuous number of concurrent INSERTs or
UPDATEs. psycopg 2 also provide full asycronous operations for the really
brave programmer.
There are confirmed reports of psycopg 1.x compiling and running on Linux
and FreeBSD on i386, Solaris, MacOS X and win32 architectures. psycopg 2
does not introduce build-wise incompatible changes so it should be able to
compile on all architectures just as its predecessor did.
Now go read the INSTALL file. More information about psycopg extensions to
the DBAPI-2.0 is available in the files located in the doc/ direcory.
psycopg is free software ("free as in freedom" but I like beer too.)
Licensing information is available in the LICENSE file.
Contributors
------------
A short list of contributors to psycopg2 follows (if you feel you belong
to this list and you can't find yourself here just drop me a mail):
* Kudos to piro for all the documentation work.
* Peter Fein contributed a logging connection/cursor class that even if it
was not used directly heavily influenced the implementation currently in
psycopg2.extras.

View File

@ -1,80 +0,0 @@
psycopg2 - Python-PostgreSQL Database Adapter
=============================================
Psycopg is the most popular PostgreSQL database adapter for the Python
programming language. Its main features are the complete implementation of
the Python DB API 2.0 specification and the thread safety (several threads can
share the same connection). It was designed for heavily multi-threaded
applications that create and destroy lots of cursors and make a large number
of concurrent "INSERT"s or "UPDATE"s.
Psycopg 2 is mostly implemented in C as a libpq wrapper, resulting in being
both efficient and secure. It features client-side and server-side cursors,
asynchronous communication and notifications, "COPY TO/COPY FROM" support.
Many Python types are supported out-of-the-box and adapted to matching
PostgreSQL data types; adaptation can be extended and customized thanks to a
flexible objects adaptation system.
Psycopg 2 is both Unicode and Python 3 friendly.
.. Note::
The psycopg2 package is still widely used and actively maintained, but it
is not expected to receive new features.
`Psycopg 3`__ is the evolution of psycopg2 and is where `new features are
being developed`__: if you are starting a new project you should probably
start from 3!
.. __: https://pypi.org/project/psycopg/
.. __: https://www.psycopg.org/psycopg3/docs/index.html
Documentation
-------------
Documentation is included in the ``doc`` directory and is `available online`__.
.. __: https://www.psycopg.org/docs/
For any other resource (source code repository, bug tracker, mailing list)
please check the `project homepage`__.
.. __: https://psycopg.org/
Installation
------------
Building Psycopg requires a few prerequisites (a C compiler, some development
packages): please check the install_ and the faq_ documents in the ``doc`` dir
or online for the details.
If prerequisites are met, you can install psycopg like any other Python
package, using ``pip`` to download it from PyPI_::
$ pip install psycopg2
or using ``setup.py`` if you have downloaded the source package locally::
$ python setup.py build
$ sudo python setup.py install
You can also obtain a stand-alone package, not requiring a compiler or
external libraries, by installing the `psycopg2-binary`_ package from PyPI::
$ pip install psycopg2-binary
The binary package is a practical choice for development and testing but in
production it is advised to use the package built from sources.
.. _PyPI: https://pypi.org/project/psycopg2/
.. _psycopg2-binary: https://pypi.org/project/psycopg2-binary/
.. _install: https://www.psycopg.org/docs/install.html#install-from-source
.. _faq: https://www.psycopg.org/docs/faq.html#faq-compile
:Build status: |gh-actions|
.. |gh-actions| image:: https://github.com/psycopg/psycopg2/actions/workflows/tests.yml/badge.svg
:target: https://github.com/psycopg/psycopg2/actions/workflows/tests.yml
:alt: Build status

375
ZPsycopgDA/DA.py Normal file
View File

@ -0,0 +1,375 @@
# ZPsycopgDA/DA.py - ZPsycopgDA Zope product: Database Connection
#
# Copyright (C) 2004 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# Or, at your option this program (ZPsycopgDA) can be distributed under the
# Zope Public License (ZPL) Version 1.0, as published on the Zope web site,
# http://www.zope.org/Resources/ZPL.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the LICENSE file for details.
ALLOWED_PSYCOPG_VERSIONS = ('2.0.1', '2.0.2', '2.0.3', '2.0.4', '2.0.5')
import sys
import time
import db
import re
import Acquisition
import Shared.DC.ZRDB.Connection
from db import DB
from Globals import HTMLFile
from ExtensionClass import Base
from App.Dialogs import MessageDialog
from DateTime import DateTime
# Build Zope version in a float for later cheks
import App
zope_version = App.version_txt.getZopeVersion()
zope_version = float("%s.%s" %(zope_version[:2]))
# ImageFile is deprecated in Zope >= 2.9
if zope_version < 2.9:
from ImageFile import ImageFile
else:
from App.ImageFile import ImageFile
# import psycopg and functions/singletons needed for date/time conversions
import psycopg2
from psycopg2 import NUMBER, STRING, ROWID, DATETIME
from psycopg2.extensions import INTEGER, LONGINTEGER, FLOAT, BOOLEAN, DATE
from psycopg2.extensions import TIME, INTERVAL
from psycopg2.extensions import new_type, register_type
# add a new connection to a folder
manage_addZPsycopgConnectionForm = HTMLFile('dtml/add',globals())
def manage_addZPsycopgConnection(self, id, title, connection_string,
zdatetime=None, tilevel=2,
check=None, REQUEST=None):
"""Add a DB connection to a folder."""
self._setObject(id, Connection(id, title, connection_string,
zdatetime, check, tilevel))
if REQUEST is not None: return self.manage_main(self, REQUEST)
# the connection object
class Connection(Shared.DC.ZRDB.Connection.Connection):
"""ZPsycopg Connection."""
_isAnSQLConnection = 1
id = 'Psycopg2_database_connection'
database_type = 'Psycopg2'
meta_type = title = 'Z Psycopg 2 Database Connection'
icon = 'misc_/conn'
def __init__(self, id, title, connection_string,
zdatetime, check=None, tilevel=2, encoding=''):
self.zdatetime = zdatetime
self.id = str(id)
self.edit(title, connection_string, zdatetime,
check=check, tilevel=tilevel, encoding=encoding)
def factory(self):
return DB
## connection parameters editing ##
def edit(self, title, connection_string,
zdatetime, check=None, tilevel=2, encoding=''):
self.title = title
self.connection_string = connection_string
self.zdatetime = zdatetime
self.tilevel = tilevel
self.encoding = encoding
self.set_type_casts()
if check: self.connect(self.connection_string)
manage_properties = HTMLFile('dtml/edit', globals())
def manage_edit(self, title, connection_string,
zdatetime=None, check=None, tilevel=2, encoding='UTF-8',
REQUEST=None):
"""Edit the DB connection."""
self.edit(title, connection_string, zdatetime,
check=check, tilevel=tilevel, encoding=encoding)
if REQUEST is not None:
msg = "Connection edited."
return self.manage_main(self,REQUEST,manage_tabs_message=msg)
def connect(self, s):
try:
self._v_database_connection.close()
except:
pass
# check psycopg version and raise exception if does not match
if psycopg2.__version__[:5] not in ALLOWED_PSYCOPG_VERSIONS:
raise ImportError("psycopg version mismatch (imported %s)" %
psycopg2.__version__)
self.set_type_casts()
self._v_connected = ''
dbf = self.factory()
# TODO: let the psycopg exception propagate, or not?
self._v_database_connection = dbf(
self.connection_string, self.tilevel, self.encoding)
self._v_database_connection.open()
self._v_connected = DateTime()
return self
def set_type_casts(self):
# note that in both cases order *is* important
if self.zdatetime:
# use zope internal datetime routines
register_type(ZDATETIME)
register_type(ZDATE)
register_type(ZTIME)
else:
# use the standard
register_type(DATETIME)
register_type(DATE)
register_type(TIME)
## browsing and table/column management ##
manage_options = Shared.DC.ZRDB.Connection.Connection.manage_options
# + (
# {'label': 'Browse', 'action':'manage_browse'},)
#manage_tables = HTMLFile('dtml/tables', globals())
#manage_browse = HTMLFile('dtml/browse', globals())
info = None
def table_info(self):
return self._v_database_connection.table_info()
def __getitem__(self, name):
if name == 'tableNamed':
if not hasattr(self, '_v_tables'): self.tpValues()
return self._v_tables.__of__(self)
raise KeyError, name
def tpValues(self):
res = []
conn = self._v_database_connection
for d in conn.tables(rdb=0):
try:
name = d['TABLE_NAME']
b = TableBrowser()
b.__name__ = name
b._d = d
b._c = c
try:
b.icon = table_icons[d['TABLE_TYPE']]
except:
pass
r.append(b)
except:
pass
return res
## database connection registration data ##
classes = (Connection,)
meta_types = ({'name':'Z Psycopg 2 Database Connection',
'action':'manage_addZPsycopgConnectionForm'},)
folder_methods = {
'manage_addZPsycopgConnection': manage_addZPsycopgConnection,
'manage_addZPsycopgConnectionForm': manage_addZPsycopgConnectionForm}
__ac_permissions__ = (
('Add Z Psycopg Database Connections',
('manage_addZPsycopgConnectionForm', 'manage_addZPsycopgConnection')),)
# add icons
misc_={'conn': ImageFile('Shared/DC/ZRDB/www/DBAdapterFolder_icon.gif')}
for icon in ('table', 'view', 'stable', 'what', 'field', 'text', 'bin',
'int', 'float', 'date', 'time', 'datetime'):
misc_[icon] = ImageFile('icons/%s.gif' % icon, globals())
## zope-specific psycopg typecasters ##
# convert an ISO timestamp string from postgres to a Zope DateTime object
def _cast_DateTime(iso, curs):
if iso:
return DateTime(re.split("GMT\+?|GMT-?", iso)[0])
# this will split us into [date, time, GMT/AM/PM(if there)]
# dt = str.split(' ')
# if len(dt) > 1:
# # we now should split out any timezone info
# dt[1] = dt[1].split('-')[0]
# dt[1] = dt[1].split('+')[0]
# return DateTime(' '.join(dt[:2]))
# else:
# return DateTime(dt[0])
# convert an ISO date string from postgres to a Zope DateTime object
def _cast_Date(iso, curs):
if iso:
return DateTime(iso)
# Convert a time string from postgres to a Zope DateTime object.
# NOTE: we set the day as today before feeding to DateTime so
# that it has the same DST settings.
def _cast_Time(iso, curs):
if iso:
return DateTime(time.strftime('%Y-%m-%d %H:%M:%S',
time.localtime(time.time())[:3]+
time.strptime(iso[:8], "%H:%M:%S")[3:]))
# NOTE: we don't cast intervals anymore because they are passed
# untouched to Zope.
def _cast_Interval(iso, curs):
return iso
ZDATETIME = new_type((1184, 1114), "ZDATETIME", _cast_DateTime)
ZINTERVAL = new_type((1186,), "ZINTERVAL", _cast_Interval)
ZDATE = new_type((1082,), "ZDATE", _cast_Date)
ZTIME = new_type((1083,), "ZTIME", _cast_Time)
## table browsing helpers ##
class TableBrowserCollection(Acquisition.Implicit):
pass
class Browser(Base):
def __getattr__(self, name):
try:
return self._d[name]
except KeyError:
raise AttributeError, name
class values:
def len(self):
return 1
def __getitem__(self, i):
try:
return self._d[i]
except AttributeError:
pass
self._d = self._f()
return self._d[i]
class TableBrowser(Browser, Acquisition.Implicit):
icon = 'what'
Description = check = ''
info = HTMLFile('table_info', globals())
menu = HTMLFile('table_menu', globals())
def tpValues(self):
v = values()
v._f = self.tpValues_
return v
def tpValues_(self):
r=[]
tname=self.__name__
for d in self._c.columns(tname):
b=ColumnBrowser()
b._d=d
try: b.icon=field_icons[d['Type']]
except: pass
b.TABLE_NAME=tname
r.append(b)
return r
def tpId(self): return self._d['TABLE_NAME']
def tpURL(self): return "Table/%s" % self._d['TABLE_NAME']
def Name(self): return self._d['TABLE_NAME']
def Type(self): return self._d['TABLE_TYPE']
manage_designInput=HTMLFile('designInput',globals())
def manage_buildInput(self, id, source, default, REQUEST=None):
"Create a database method for an input form"
args=[]
values=[]
names=[]
columns=self._columns
for i in range(len(source)):
s=source[i]
if s=='Null': continue
c=columns[i]
d=default[i]
t=c['Type']
n=c['Name']
names.append(n)
if s=='Argument':
values.append("<dtml-sqlvar %s type=%s>'" %
(n, vartype(t)))
a='%s%s' % (n, boboType(t))
if d: a="%s=%s" % (a,d)
args.append(a)
elif s=='Property':
values.append("<dtml-sqlvar %s type=%s>'" %
(n, vartype(t)))
else:
if isStringType(t):
if find(d,"\'") >= 0: d=join(split(d,"\'"),"''")
values.append("'%s'" % d)
elif d:
values.append(str(d))
else:
raise ValueError, (
'no default was given for <em>%s</em>' % n)
class ColumnBrowser(Browser):
icon='field'
def check(self):
return ('\t<input type=checkbox name="%s.%s">' %
(self.TABLE_NAME, self._d['Name']))
def tpId(self): return self._d['Name']
def tpURL(self): return "Column/%s" % self._d['Name']
def Description(self):
d=self._d
if d['Scale']:
return " %(Type)s(%(Precision)s,%(Scale)s) %(Nullable)s" % d
else:
return " %(Type)s(%(Precision)s) %(Nullable)s" % d
table_icons={
'TABLE': 'table',
'VIEW':'view',
'SYSTEM_TABLE': 'stable',
}
field_icons={
NUMBER.name: 'i',
STRING.name: 'text',
DATETIME.name: 'date',
INTEGER.name: 'int',
FLOAT.name: 'float',
BOOLEAN.name: 'bin',
ROWID.name: 'int'
}

31
ZPsycopgDA/__init__.py Normal file
View File

@ -0,0 +1,31 @@
# ZPsycopgDA/__init__.py - ZPsycopgDA Zope product
#
# Copyright (C) 2004 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# Or, at your option this program (ZPsycopgDA) can be distributed under the
# Zope Public License (ZPL) Version 1.0, as published on the Zope web site,
# http://www.zope.org/Resources/ZPL.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the LICENSE file for details.
__doc__ = "ZPsycopg Database Adapter Registration."
__version__ = '2.0'
import DA
def initialize(context):
context.registerClass(
DA.Connection,
permission = 'Add Z Psycopg 2 Database Connections',
constructors = (DA.manage_addZPsycopgConnectionForm,
DA.manage_addZPsycopgConnection),
icon = SOFTWARE_HOME + '/Shared/DC/ZRDB/www/DBAdapterFolder_icon.gif')

206
ZPsycopgDA/db.py Normal file
View File

@ -0,0 +1,206 @@
# ZPsycopgDA/db.py - query execution
#
# Copyright (C) 2004 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# Or, at your option this program (ZPsycopgDA) can be distributed under the
# Zope Public License (ZPL) Version 1.0, as published on the Zope web site,
# http://www.zope.org/Resources/ZPL.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the LICENSE file for details.
from Shared.DC.ZRDB.TM import TM
from Shared.DC.ZRDB import dbi_db
from ZODB.POSException import ConflictError
import site
import pool
import psycopg2
from psycopg2.extensions import INTEGER, LONGINTEGER, FLOAT, BOOLEAN, DATE, TIME
from psycopg2 import NUMBER, STRING, ROWID, DATETIME
# the DB object, managing all the real query work
class DB(TM, dbi_db.DB):
_p_oid = _p_changed = _registered = None
def __init__(self, dsn, tilevel, enc='utf-8'):
self.dsn = dsn
self.tilevel = tilevel
self.encoding = enc
self.failures = 0
self.calls = 0
self.make_mappings()
def getconn(self, create=True):
conn = pool.getconn(self.dsn)
conn.set_isolation_level(int(self.tilevel))
return conn
def putconn(self, close=False):
try:
conn = pool.getconn(self.dsn, False)
except AttributeError:
pass
pool.putconn(self.dsn, conn, close)
def getcursor(self):
conn = self.getconn()
return conn.cursor()
def _finish(self, *ignored):
try:
conn = self.getconn(False)
conn.commit()
self.putconn()
except AttributeError:
pass
def _abort(self, *ignored):
try:
conn = self.getconn(False)
conn.rollback()
self.putconn()
except AttributeError:
pass
def open(self):
# this will create a new pool for our DSN if not already existing,
# then get and immediately release a connection
self.getconn()
self.putconn()
def close(self):
# FIXME: if this connection is closed we flush all the pool associated
# with the current DSN; does this makes sense?
pool.flushpool(self.dsn)
def sortKey(self):
return 1
def make_mappings(self):
"""Generate the mappings used later by self.convert_description()."""
self.type_mappings = {}
for t, s in [(INTEGER,'i'), (LONGINTEGER, 'i'), (NUMBER, 'n'),
(BOOLEAN,'n'), (ROWID, 'i'),
(DATETIME, 'd'), (DATE, 'd'), (TIME, 'd')]:
for v in t.values:
self.type_mappings[v] = (t, s)
def convert_description(self, desc, use_psycopg_types=False):
"""Convert DBAPI-2.0 description field to Zope format."""
items = []
for name, typ, width, ds, p, scale, null_ok in desc:
m = self.type_mappings.get(typ, (STRING, 's'))
items.append({
'name': name,
'type': use_psycopg_types and m[0] or m[1],
'width': width,
'precision': p,
'scale': scale,
'null': null_ok,
})
return items
## tables and rows ##
def tables(self, rdb=0, _care=('TABLE', 'VIEW')):
self._register()
c = self.getcursor()
c.execute(
"SELECT t.tablename AS NAME, 'TABLE' AS TYPE "
" FROM pg_tables t WHERE tableowner <> 'postgres' "
"UNION SELECT v.viewname AS NAME, 'VIEW' AS TYPE "
" FROM pg_views v WHERE viewowner <> 'postgres' "
"UNION SELECT t.tablename AS NAME, 'SYSTEM_TABLE\' AS TYPE "
" FROM pg_tables t WHERE tableowner = 'postgres' "
"UNION SELECT v.viewname AS NAME, 'SYSTEM_TABLE' AS TYPE "
"FROM pg_views v WHERE viewowner = 'postgres'")
res = []
for name, typ in c.fetchall():
if typ in _care:
res.append({'TABLE_NAME': name, 'TABLE_TYPE': typ})
self.putconn()
return res
def columns(self, table_name):
self._register()
c = self.getcursor()
try:
r = c.execute('SELECT * FROM "%s" WHERE 1=0' % table_name)
except:
return ()
self.putconn()
return self.convert_description(c.description, True)
## query execution ##
def query(self, query_string, max_rows=None, query_data=None):
self._register()
self.calls = self.calls+1
desc = ()
res = []
nselects = 0
c = self.getcursor()
try:
for qs in [x for x in query_string.split('\0') if x]:
if type(qs) == unicode:
if self.encoding:
qs = qs.encode(self.encoding)
try:
if query_data:
c.execute(qs, query_data)
else:
c.execute(qs)
except psycopg2.OperationalError, e:
try:
self.close()
except:
pass
self.open()
try:
if query_data:
c.execute(qs, query_data)
else:
c.execute(qs)
except (psycopg2.ProgrammingError,
psycopg2.IntegrityError), e:
if e.args[0].find("concurrent update") > -1:
raise ConflictError
raise e
except (psycopg2.ProgrammingError, psycopg2.IntegrityError), e:
if e.args[0].find("concurrent update") > -1:
raise ConflictError
raise e
if c.description is not None:
nselects += 1
if c.description != desc and nselects > 1:
raise psycopg2.ProgrammingError(
'multiple selects in single query not allowed')
if max_rows:
res = c.fetchmany(max_rows)
else:
res = c.fetchall()
desc = c.description
self.failures = 0
except StandardError, err:
self._abort()
raise err
return self.convert_description(desc), res

96
ZPsycopgDA/dtml/add.dtml Normal file
View File

@ -0,0 +1,96 @@
<dtml-var manage_page_header>
<dtml-var "manage_form_title(this(), _,
form_title='Add Z Psycopg 2 Database Connection',
help_product='ZPsycopgDA',
help_topic='ZPsycopgDA-Method-Add.stx'
)">
<p class="form-help">
A Zope Psycopg 2 Database Connection is used to connect and execute
queries on a PostgreSQL database.
</p>
<p class="form-help">
In the form below <em>Connection String</em> (also called the Data Source Name
or DSN for short) is a string... (TODO: finish docs)
</p>
<form action="manage_addZPsycopgConnection" method="POST">
<table cellspacing="0" cellpadding="2" border="0">
<tr>
<td align="left" valign="top">
<div class="form-label">
Id
</div>
</td>
<td align="left" valign="top">
<input type="text" name="id" size="40"
value="Psycopg2_database_connection" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-optional">
Title
</div>
</td>
<td align="left" valign="top">
<input type="text" name="title" size="40"
value="Z Psycopg 2 Database Connection"/>
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Connection string
</div>
</td>
<td align="left" valign="top">
<input type="text" name="connection_string" size="40" value="" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Connect immediately
</div>
</td>
<td align="left" valign="top">
<input type="checkbox" name="check" value="YES" checked="YES" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Use Zope's internal DateTime
</div>
</td>
<td align="left" valign="top">
<input type="checkbox" name="zdatetime" value="YES" checked="YES" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Transaction isolation level
</div>
</td>
<td align="left" valign="top">
<select name="tilevel:int">
<option value="1">Read committed</option>
<option value="2" selected="YES">Serializable</option>
</select>
</td>
</tr>
<tr>
<td align="left" valign="top" colspan="2">
<div class="form-element">
<input class="form-element" type="submit" name="submit" value=" Add " />
</div>
</td>
</tr>
</table>
</form>
<dtml-var manage_page_footer>

View File

@ -0,0 +1,11 @@
<html>
<head><title><dtml-var title_or_id >tables</title></head>
<body bgcolor="#FFFFFF" link="#000099" vlink="#555555" alink="#77003B">
<dtml-var manage_tabs>
<dtml-tree header="info">
<IMG SRC="<dtml-var SCRIPT_NAME >/misc_/ZPsycopgDA/<dtml-var icon>"
ALT="<dtml-var Type>" BORDER="0">
<dtml-var Name><dtml-var Description>
</dtml-tree>
</body>
</html>

67
ZPsycopgDA/dtml/edit.dtml Normal file
View File

@ -0,0 +1,67 @@
<dtml-var manage_page_header>
<dtml-var manage_tabs>
<form action="manage_edit" method="POST">
<table cellspacing="0" cellpadding="2" border="0">
<tr>
<td align="left" valign="top">
<div class="form-optional">
Title
</div>
</td>
<td align="left" valign="top">
<input type="text" name="title" size="40"
value="&dtml-title;"/>
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Connection string
</div>
</td>
<td align="left" valign="top">
<input type="text" name="connection_string" size="40"
value="&dtml-connection_string;" />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Use Zope's internal DateTime
</div>
</td>
<td align="left" valign="top">
<input type="checkbox" name="zdatetime" value="YES"
<dtml-if expr="zdatetime">checked="YES"</dtml-if> />
</td>
</tr>
<tr>
<td align="left" valign="top">
<div class="form-label">
Transaction isolation level
</div>
</td>
<td align="left" valign="top">
<select name="tilevel:int">
<option value="1"
<dtml-if expr="tilevel==1">selected="YES"</dtml-if>>
Read committed</option>
<option value="2"
<dtml-if expr="tilevel==2">selected="YES"</dtml-if>>
Serializable</option>
</select>
</td>
</tr>
<tr>
<td align="left" valign="top" colspan="2">
<div class="form-element">
<input class="form-element" type="submit" name="submit"
value=" Save Changes " />
</div>
</td>
</tr>
</table>
</form>
<dtml-var manage_page_footer>

View File

@ -0,0 +1,7 @@
<dtml-var standard_html_header>
<dtml-var TABLE_TYPE><dtml-if TABLE_OWNER>
owned by <dtml-var TABLE_OWNER></dtml-if>
<dtml-if REMARKS><br><dtml-var REMARKS></dtml-if>
<dtml-var standard_html_footer>

BIN
ZPsycopgDA/icons/bin.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 924 B

BIN
ZPsycopgDA/icons/date.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 930 B

Binary file not shown.

After

Width:  |  Height:  |  Size: 925 B

BIN
ZPsycopgDA/icons/field.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 915 B

BIN
ZPsycopgDA/icons/float.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 929 B

BIN
ZPsycopgDA/icons/int.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 918 B

BIN
ZPsycopgDA/icons/stable.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 884 B

BIN
ZPsycopgDA/icons/table.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 878 B

BIN
ZPsycopgDA/icons/text.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 918 B

BIN
ZPsycopgDA/icons/time.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 926 B

BIN
ZPsycopgDA/icons/view.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 893 B

BIN
ZPsycopgDA/icons/what.gif Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 894 B

51
ZPsycopgDA/pool.py Normal file
View File

@ -0,0 +1,51 @@
# ZPsycopgDA/pool.py - ZPsycopgDA Zope product: connection pooling
#
# Copyright (C) 2004 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# Or, at your option this program (ZPsycopgDA) can be distributed under the
# Zope Public License (ZPL) Version 1.0, as published on the Zope web site,
# http://www.zope.org/Resources/ZPL.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE.
#
# See the LICENSE file for details.
# all the connections are held in a pool of pools, directly accessible by the
# ZPsycopgDA code in db.py
import threading
import psycopg2.pool
_connections_pool = {}
_connections_lock = threading.Lock()
def getpool(dsn, create=True):
_connections_lock.acquire()
try:
if not _connections_pool.has_key(dsn) and create:
_connections_pool[dsn] = \
psycopg2.pool.PersistentConnectionPool(4, 200, dsn)
finally:
_connections_lock.release()
return _connections_pool[dsn]
def flushpool(dsn):
_connections_lock.acquire()
try:
_connections_pool[dsn].closeall()
del _connections_pool[dsn]
finally:
_connections_lock.release()
def getconn(dsn, create=True):
return getpool(dsn, create=create).getconn()
def putconn(dsn, conn, close=False):
getpool(dsn).putconn(conn, close=close)

12
debian/changelog vendored Normal file
View File

@ -0,0 +1,12 @@
psycopg2 (1.99.12.1-1) experimental; urgency=low
* Adapted from patches sent by W. Borgert.
* Renamed source package to psycopg2.
-- Federico Di Gregorio <fog@debian.org> Fri, 4 Mar 2005 13:11:43 +0100
psycopg2 (1.99.11-0.1) unstable; urgency=low
* Experimental package.
-- W. Borgert <debacle@debian.org> Sun, 09 Jan 2005 10:14:09 +0000

51
debian/control vendored Normal file
View File

@ -0,0 +1,51 @@
Source: psycopg2
Section: python
Priority: optional
Build-depends: postgresql-dev, debhelper (>> 3), python2.3-dev, cdbs
Maintainer: Federico Di Gregorio <fog@debian.org>
Standards-Version: 3.6.1.1
Package: python-psycopg2
Architecture: any
Section: python
Depends: python (>= 2.3), python (<< 2.4), python2.3-psycopg2
Description: Python module for PostgreSQL [dummy package]
psycopg is a PostgreSQL database adapter for the Python programming
language. It was written from scratch with the aim of being very small
and fast, and stable as a rock. The main advantages of psycopg are that
it supports the full Python DBAPI-2.0 and being thread safe at level 2.
.
psycopg 2 is the next generation psycopg, implementing a much better
type system and even more DBAPI extensions:
.
* support for Python datetime and Decimal types;
* complete implementation of adapt() from PEP 246 to convert Python
types to PostgreSQL ones;
* COPY FROM/COPY TO support;
* inehritable connection and cursor objects and support for connection
and cursor factories;
* automatic encoding conversion and support for unicode queries.
.
This dummy package just depends on the right, default version of Python
and psycopg 2.
Package: python2.3-psycopg2
Architecture: any
Section: python
Depends: ${shlibs:Depends}, python2.3
Description: Python 2.3 module for PostgreSQL
psycopg is a PostgreSQL database adapter for the Python programming
language. It was written from scratch with the aim of being very small
and fast, and stable as a rock. The main advantages of psycopg are that
it supports the full Python DBAPI-2.0 and being thread safe at level 2.
.
psycopg 2 is the next generation psycopg, implementing a much better
type system and even more DBAPI extensions:
.
* support for Python datetime and Decimal types;
* complete implementation of adapt() from PEP 246 to convert Python
types to PostgreSQL ones;
* COPY FROM/COPY TO support;
* inehritable connection and cursor objects and support for connection
and cursor factories;
* automatic encoding conversion and support for unicode queries.

10
debian/copyright vendored Normal file
View File

@ -0,0 +1,10 @@
psycopg 2 can be downloaded from:
http://initd.org/pub/software/psycopg/ALPHA/
Copyright (c) 2001-2005 Federico Di Gregorio <fog@debian.org>
This program is distributed under the GNU GPL.
On Debian GNU/Linux systems, the complete text of the GNU General
Public License can be found in '/usr/share/common-licenses/GPL'.

4
debian/rules vendored Executable file
View File

@ -0,0 +1,4 @@
#!/usr/bin/make -f
include /usr/share/cdbs/1/rules/debhelper.mk
include /usr/share/cdbs/1/class/python-distutils.mk

8
doc/.gitignore vendored
View File

@ -1,8 +0,0 @@
env
src/_build/*
html/*
psycopg2.txt
src/sqlstate_errors.rst
# Added by psycopg-website to customize published docs
src/_templates/layout.html

View File

@ -1,165 +0,0 @@
GNU LESSER GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <https://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
This version of the GNU Lesser General Public License incorporates
the terms and conditions of version 3 of the GNU General Public
License, supplemented by the additional permissions listed below.
0. Additional Definitions.
As used herein, "this License" refers to version 3 of the GNU Lesser
General Public License, and the "GNU GPL" refers to version 3 of the GNU
General Public License.
"The Library" refers to a covered work governed by this License,
other than an Application or a Combined Work as defined below.
An "Application" is any work that makes use of an interface provided
by the Library, but which is not otherwise based on the Library.
Defining a subclass of a class defined by the Library is deemed a mode
of using an interface provided by the Library.
A "Combined Work" is a work produced by combining or linking an
Application with the Library. The particular version of the Library
with which the Combined Work was made is also called the "Linked
Version".
The "Minimal Corresponding Source" for a Combined Work means the
Corresponding Source for the Combined Work, excluding any source code
for portions of the Combined Work that, considered in isolation, are
based on the Application, and not on the Linked Version.
The "Corresponding Application Code" for a Combined Work means the
object code and/or source code for the Application, including any data
and utility programs needed for reproducing the Combined Work from the
Application, but excluding the System Libraries of the Combined Work.
1. Exception to Section 3 of the GNU GPL.
You may convey a covered work under sections 3 and 4 of this License
without being bound by section 3 of the GNU GPL.
2. Conveying Modified Versions.
If you modify a copy of the Library, and, in your modifications, a
facility refers to a function or data to be supplied by an Application
that uses the facility (other than as an argument passed when the
facility is invoked), then you may convey a copy of the modified
version:
a) under this License, provided that you make a good faith effort to
ensure that, in the event an Application does not supply the
function or data, the facility still operates, and performs
whatever part of its purpose remains meaningful, or
b) under the GNU GPL, with none of the additional permissions of
this License applicable to that copy.
3. Object Code Incorporating Material from Library Header Files.
The object code form of an Application may incorporate material from
a header file that is part of the Library. You may convey such object
code under terms of your choice, provided that, if the incorporated
material is not limited to numerical parameters, data structure
layouts and accessors, or small macros, inline functions and templates
(ten or fewer lines in length), you do both of the following:
a) Give prominent notice with each copy of the object code that the
Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the object code with a copy of the GNU GPL and this license
document.
4. Combined Works.
You may convey a Combined Work under terms of your choice that,
taken together, effectively do not restrict modification of the
portions of the Library contained in the Combined Work and reverse
engineering for debugging such modifications, if you also do each of
the following:
a) Give prominent notice with each copy of the Combined Work that
the Library is used in it and that the Library and its use are
covered by this License.
b) Accompany the Combined Work with a copy of the GNU GPL and this license
document.
c) For a Combined Work that displays copyright notices during
execution, include the copyright notice for the Library among
these notices, as well as a reference directing the user to the
copies of the GNU GPL and this license document.
d) Do one of the following:
0) Convey the Minimal Corresponding Source under the terms of this
License, and the Corresponding Application Code in a form
suitable for, and under terms that permit, the user to
recombine or relink the Application with a modified version of
the Linked Version to produce a modified Combined Work, in the
manner specified by section 6 of the GNU GPL for conveying
Corresponding Source.
1) Use a suitable shared library mechanism for linking with the
Library. A suitable mechanism is one that (a) uses at run time
a copy of the Library already present on the user's computer
system, and (b) will operate properly with a modified version
of the Library that is interface-compatible with the Linked
Version.
e) Provide Installation Information, but only if you would otherwise
be required to provide such information under section 6 of the
GNU GPL, and only to the extent that such information is
necessary to install and execute a modified version of the
Combined Work produced by recombining or relinking the
Application with a modified version of the Linked Version. (If
you use option 4d0, the Installation Information must accompany
the Minimal Corresponding Source and Corresponding Application
Code. If you use option 4d1, you must provide the Installation
Information in the manner specified by section 6 of the GNU GPL
for conveying Corresponding Source.)
5. Combined Libraries.
You may place library facilities that are a work based on the
Library side by side in a single library together with other library
facilities that are not Applications and are not covered by this
License, and convey such a combined library under terms of your
choice, if you do both of the following:
a) Accompany the combined library with a copy of the same work based
on the Library, uncombined with any other library facilities,
conveyed under the terms of this License.
b) Give prominent notice with the combined library that part of it
is a work based on the Library, and explaining where to find the
accompanying uncombined form of the same work.
6. Revised Versions of the GNU Lesser General Public License.
The Free Software Foundation may publish revised and/or new versions
of the GNU Lesser General Public License from time to time. Such new
versions will be similar in spirit to the present version, but may
differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the
Library as you received it specifies that a certain numbered version
of the GNU Lesser General Public License "or any later version"
applies to it, you have the option of following the terms and
conditions either of that published version or of any later version
published by the Free Software Foundation. If the Library as you
received it does not specify a version number of the GNU Lesser
General Public License, you may choose any version of the GNU Lesser
General Public License ever published by the Free Software Foundation.
If the Library as you received it specifies that a proxy can decide
whether future versions of the GNU Lesser General Public License shall
apply, that proxy's public statement of acceptance of any version is
permanent authorization for you to choose that version for the
Library.

1744
doc/ChangeLog-1.x Normal file

File diff suppressed because it is too large Load Diff

43
doc/HACKING Normal file
View File

@ -0,0 +1,43 @@
General information
*******************
Some help to people wanting to hack on psycopg. First of all, note that
*every* function in the psycopg module source code is prefixed by one of the
following words:
psyco is used for function directly callable from python (i.e., functions
in the psycopg module itself.) the only notable exception is the
source code for the module itself, that uses "psyco" even for C-only
functions.
conn is used for functions related to connection objects.
curs is used for functions related to cursor objects.
typecast is used for typecasters and utility function related to
typecaster creation and registration.
Pythonic definition of types and functions available from python are defined
in *_type.c files. Internal functions, callable only from C are located in
*_int.c files and extensions to the DBAPI can be found in the *_ext.c files.
Patches
*******
If you submit a patch, please send a diff generated with the "-u" switch.
Also note that I don't like that much cosmetic changes (like renaming
already existing variables) and I will rewrap the patch to 78 columns
anyway, so it is much better if you do that beforehand.
The type system
***************
Simple types, like integers and strings, are converted to python base types
(the conversion functions are in typecast_base.c). Complex types are
converted to ad-hoc types, defined in the typeobj_*.{c,h} files. The
conversion function are in the other typecast_*.c files. typecast.c defines
the basic utility functions (available through the psycopg module) used when
defining new typecasters from C and python.

View File

@ -1,39 +0,0 @@
.PHONY: env help clean html package doctest
docs: html
check: doctest
# The environment is currently required to build the documentation.
# It is not clean by 'make clean'
PYTHON := python$(PYTHON_VERSION)
PYTHON_VERSION ?= $(shell $(PYTHON) -c 'import sys; print("%d.%d" % sys.version_info[:2])')
BUILD_DIR = $(shell pwd)/../build/lib.$(PYTHON_VERSION)
SPHINXBUILD ?= $$(pwd)/env/bin/sphinx-build
SPHOPTS = SPHINXBUILD=$(SPHINXBUILD)
html: package src/sqlstate_errors.rst
$(MAKE) $(SPHOPTS) -C src $@
cp -r src/_build/html .
src/sqlstate_errors.rst: ../psycopg/sqlstate_errors.h $(BUILD_DIR)
./env/bin/python src/tools/make_sqlstate_docs.py $< > $@
$(BUILD_DIR):
$(MAKE) PYTHON=$(PYTHON) -C .. package
doctest:
$(MAKE) PYTHON=$(PYTHON) -C .. package
$(MAKE) $(SPHOPTS) -C src $@
clean:
$(MAKE) $(SPHOPTS) -C src $@
rm -rf html src/sqlstate_errors.rst
env: requirements.txt
$(PYTHON) -m venv env
./env/bin/pip install -r requirements.txt
echo "$$(pwd)/../build/lib.$(PYTHON_VERSION)" \
> env/lib/python$(PYTHON_VERSION)/site-packages/psycopg.pth

View File

@ -1,20 +0,0 @@
How to build psycopg documentation
----------------------------------
Building the documentation usually requires building the library too for
introspection, so you will need the same prerequisites_. The only extra
prerequisite is virtualenv_: the packages needed to build the docs will be
installed when building the env.
.. _prerequisites: https://www.psycopg.org/docs/install.html#install-from-source
.. _virtualenv: https://virtualenv.pypa.io/en/latest/
Build the env once with::
make env
Then you can build the documentation with::
make
You should find the rendered documentation in the ``html`` directory.

View File

@ -1,10 +1,10 @@
From: Jack Moffitt <jack@xiph.org>
To: Psycopg Mailing List <psycopg@lists.initd.org>
Subject: Re: [Psycopg] preparing for 1.0
Date: 22 Oct 2001 11:16:21 -0600
Date: 22 Oct 2001 11:16:21 -0600
www.vorbis.com is serving from 5-10k pages per day with psycopg serving
data for most of that.
www.vorbis.com is serving from 5-10k pages per day with psycopg serving
data for most of that.
I plan to use it for several of our other sites, so that number will
increase.
@ -19,11 +19,11 @@ jack.
From: Yury Don <gercon@vpcit.ru>
To: Psycopg Mailing List <psycopg@lists.initd.org>
Subject: Re: [Psycopg] preparing for 1.0
Date: 23 Oct 2001 09:53:11 +0600
Date: 23 Oct 2001 09:53:11 +0600
We use psycopg and psycopg zope adapter since fisrt public
release (it seems version 0.4). Now it works on 3 our sites and in intranet
applications. We had few problems, but all problems were quickly
applications. We had few problems, but all problems were quckly
solved. The strong side of psycopg is that it's code is well organized
and easy to understand. When I found a problem with non-ISO datestyle in first
version of psycopg, it took for me 15 or 20 minutes to learn code and
@ -32,7 +32,7 @@ to solve the problem, even thouth my knowledge of c were poor.
BTW, segfault with dictfetchall on particular data set (see [Psycopg]
dictfetchXXX() problems) disappeared in 0.99.8pre2.
--
--
Best regards,
Yury Don
@ -42,7 +42,7 @@ To: Federico Di Gregorio <fog@debian.org>
Cc: Psycopg Mailing List <psycopg@lists.initd.org>
Subject: Re: [Psycopg] preparing for 1.0
Date: 23 Oct 2001 08:25:52 -0400
The US Govt Department of Labor's Office of Disability Employment
Policy's DisabilityDirect website is run on zope and zpsycopg.
@ -50,7 +50,7 @@ Policy's DisabilityDirect website is run on zope and zpsycopg.
From: Scott Leerssen <sleerssen@racemi.com>
To: Federico Di Gregorio <fog@debian.org>
Subject: Re: [Psycopg] preparing for 1.0
Date: 23 Oct 2001 09:56:10 -0400
Date: 23 Oct 2001 09:56:10 -0400
Racemi's load management software infrastructure uses psycopg to handle
complex server allocation decisions, plus storage and access of
@ -66,10 +66,10 @@ From: Andre Schubert <andre.schubert@geyer.kabeljournal.de>
To: Federico Di Gregorio <fog@debian.org>
Cc: Psycopg Mailing List <psycopg@lists.initd.org>
Subject: Re: [Psycopg] preparing for 1.0
Date: 23 Oct 2001 11:46:07 +0200
Date: 23 Oct 2001 11:46:07 +0200
i have changed the psycopg version to 0.99.8pre2 on all devel-machines
and all segfaults are gone. after my holiday i wil change to 0.99.8pre2
and all segfaults are gone. after my holiday i wil change to 0.99.8pre2
or 1.0 on our production-server.
this server contains several web-sites which are all connected to
postgres over ZPsycopgDA.
@ -81,7 +81,7 @@ From: Fred Wilson Horch <fhorch@ecoaccess.org>
To: <psycopg@lists.initd.org>
Subject: [Psycopg] Success story for psycopg
Date: 23 Oct 2001 10:59:17 -0400
Due to various quirks of PyGreSQL and PoPy, EcoAccess has been looking for
a reliable, fast and relatively bug-free Python-PostgreSQL interface for
our project.
@ -98,7 +98,7 @@ reports and feature requests, and we're looking forward to using psycopg
as the Python interface for additional database-backed web applications.
Keep up the good work!
--
--
Fred Wilson Horch mailto:fhorch@ecoaccess.org
Executive Director, EcoAccess http://ecoaccess.org/

33
doc/TODO Normal file
View File

@ -0,0 +1,33 @@
TODO list for psycopg 2 or later
********************************
Move items to the DONE section only after writing a test for the
implementation. Also add a note on how the item was resolved.
(Obviously I was joking about the test..)
* Find a better way to compile the type-casting code instead of including it
in typecast.c directy. (Including is not that bad, but the need to touch
psycopg/typecast.c every time is bad bad bad.)
* executemany() should _not_ take the async flag, remove it and force multiple
queries to be synchronous.
* Fix all the docstrings.
* Support the protocols API fully.
* Unify the common code in typecast_datetime.c and typecast_mxdatetime.c.
* Port typecasters to new-style classes.
* Write a complete postgresql<->python encodings table.
* Implement binary typecasters (should be easy, but it will take time.)
DONE
====
* Convert type-casters to new-style types in Python 2.2+.
* callproc() never worked, fix it or remove it and raise right exception.
[Removed callproc code, now an exception is raised.]

138
doc/api-screen.css Normal file
View File

@ -0,0 +1,138 @@
/* Based on the Epydoc "default.css"
** with some missing reST-related classes
** and Python syntax support (from SilverCity)
*/
/* Body color */
body { background: #ffffff; color: #000000; }
/* Tables */
table.summary, table.details, table.index
{ background: #e8f0f8; color: #000000; }
tr.summary, tr.details, tr.index
{ background: #70b0f0; color: #000000;
text-align: left; font-size: 120%; }
tr.group { background: #c0e0f8; color: #000000;
text-align: left; font-size: 120%;
font-style: italic; }
/* Documentation page titles */
h2.module { margin-top: 0.2em; }
h2.class { margin-top: 0.2em; }
/* Headings */
h1.heading { font-size: +140%; font-style: italic;
font-weight: bold; }
h2.heading { font-size: +125%; font-style: italic;
font-weight: bold; }
h3.heading { font-size: +110%; font-style: italic;
font-weight: normal; }
/* Base tree */
pre.base-tree { font-size: 80%; margin: 0; }
/* TOC */
p.toc { margin: 0; }
/* Details Sections */
table.func-details { background: #e8f0f8; color: #000000;
border: 2px groove #c0d0d0;
padding: 0 1em 0 1em; margin: 0.4em 0 0 0; }
h3.func-detail { background: transparent; color: #000000;
margin: 0 0 1em 0; }
table.var-details { background: #e8f0f8; color: #000000;
border: 2px groove #c0d0d0;
padding: 0 1em 0 1em; margin: 0.4em 0 0 0; }
h3.var-details { background: transparent; color: #000000;
margin: 0 0 1em 0; }
/* Function signatures */
.sig { background: transparent; color: #000000;
font-weight: bold; }
.sig-name { background: transparent; color: #006080; }
.sig-arg, .sig-kwarg, .sig-vararg
{ background: transparent; color: #008060; }
.sig-default { background: transparent; color: #602000; }
.summary-sig { background: transparent; color: #000000; }
.summary-sig-name { background: transparent; color: #204080; }
.summary-sig-arg, .summary-sig-kwarg, .summary-sig-vararg
{ background: transparent; color: #008060; }
/* Doctest blocks */
.py-src { background: transparent; color: #000000; }
.py-prompt { background: transparent; color: #005050;
font-weight: bold;}
.py-string { background: transparent; color: #006030; }
.py-comment { background: transparent; color: #003060; }
.py-keyword { background: transparent; color: #600000; }
.py-output { background: transparent; color: #404040; }
div.code-block,
pre.literal-block,
pre.doctestblock { background: #f4faff; color: #000000;
padding: .5em; margin: 1em;
border: 1px solid #708890; }
table pre.doctestblock
{ background: #dce4ec; color: #000000;
padding: .5em; margin: 1em;
border: 1px solid #708890; }
div.code-block { font-family: monospace; }
/* Variable values */
pre.variable { background: #dce4ec; color: #000000;
padding: .5em; margin: 0;
border: 1px solid #708890; }
.variable-linewrap { background: transparent; color: #604000; }
.variable-ellipsis { background: transparent; color: #604000; }
.variable-quote { background: transparent; color: #604000; }
.re { background: transparent; color: #000000; }
.re-char { background: transparent; color: #006030; }
.re-op { background: transparent; color: #600000; }
.re-group { background: transparent; color: #003060; }
.re-ref { background: transparent; color: #404040; }
/* Navigation bar */
table.navbar { background: #a0c0ff; color: #0000ff;
border: 2px groove #c0d0d0; }
th.navbar { background: #a0c0ff; color: #0000ff; }
th.navselect { background: #70b0ff; color: #000000; }
.nomargin { margin: 0; }
/* Links */
a:link { background: transparent; color: #0000ff; }
a:visited { background: transparent; color: #204080; }
a.navbar:link { background: transparent; color: #0000ff;
text-decoration: none; }
a.navbar:visited { background: transparent; color: #204080;
text-decoration: none; }
/* Admonitions */
div.warning,
div.note { background-color: #c0e0f8;
border: thin solid black;
padding: 1em;
margin-left: 1em;
margin-right: 1em; }
div.warning .first,
div.note .first { font-family: sans-serif;
font-size: 110%;
margin-right: 0.5em; }
/* Lists */
ul { margin-top: 0; }
/* Python syntax */
.p_character { color: olive; }
.p_classname { color: blue; font-weight: bold; }
.p_commentblock {color: gray; font-style: italic; }
.p_commentline { color: green; font-style: italic; }
.p_default {}
.p_defname { color: #009999; font-weight: bold; }
.p_identifier { color: black; }
.p_number { color: #009999; }
.p_operator { color: black; }
.p_string { color: #7F007F; }
.p_stringeol { color: #7F007F; }
.p_triple { color: #7F0000; }
.p_tripledouble { color: #7F0000; }
.p_word { color: navy; font-weight: bold; }

67
doc/async.txt Normal file
View File

@ -0,0 +1,67 @@
psycopg asynchronous API
************************
** Important: async quaeries are not enabled for 2.0 **
Program code can initiate an asynchronous query by passing an 'async=1' flag
to the .execute() method. A very simple example, from the connection to the
query:
conn = psycopg.connect(database='test')
curs = conn.cursor()
curs.execute("SEECT * from test WHERE fielda > %s", (1971,), async=1)
From then on any query on other cursors derived from the same connection is
doomed to fail (and raise an exception) until the original cursor (the one
executing the query) complete the asynchronous operation. This can happen in
a number of different ways:
1) one of the .fetchXXX() methods is called, effectively blocking untill
data has been sent from the backend to the client, terminating the
query.
2) .cancel() is called. This method tries to abort the current query and
will block until the query is aborted or fully executed. The return
value is True if the query was successfully aborted or False if it
was executed. Query result are discarded in both cases.
3) .execute() is called again on the same cursor (.execute() on a
different cursor will simply raise an exception.) This waits for the
complete execution of the current query, discard any data and execute
the new one.
Note that calling .execute() two times in a row will not abort the former
query and will temporarily go to synchronous mode until the first of the two
queries is executed.
Cursors now have some extra methods that make them usefull during
asynchronous queries:
.fileno()
Returns the file descriptor associated with the current connection and
make possible to use a cursor in a context where a file object would be
expected (like in a select() call.)
.isbusy()
Returns True if the backend is still processing the query or false if
data is ready to be fetched (by one of the .fetchXXX() methods.)
A code snippet that shows how to use the cursor object in a select() call:
import psycopg
import select
conn = psycopg.connect(database='test')
curs = conn.cursor()
curs.execute("SEECT * from test WHERE fielda > %s", (1971,), async=1)
# wait for input with a maximum timeout of 5 seconds
query_ended = False
while not query_ended:
rread, rwrite, rspec = select([cursor, another_file], [], [], 5)
if not cursor.isbusy():
query_ended = True
# manage input from other sources like other_file, etc.
print "Query Results:"
for row in cursor:
print row

260
doc/extensions.rst Normal file
View File

@ -0,0 +1,260 @@
=======================================
psycopg 2 extensions to the DBAPI 2.0
=======================================
This document is a short summary of the extensions built in psycopg 2.0.x over
the standard `Python Database API Specification 2.0`__, usually called simply
DBAPI-2.0 or even PEP-249. Before reading on this document please make sure
you already know how to program in Python using a DBAPI-2.0 compliant driver:
basic concepts like opening a connection, executing queries and commiting or
rolling back a transaction will not be explained but just used.
.. __: http://www.python.org/peps/pep-0249.html
Many objects and extension functions are defined in the `psycopg2.extensions`
module.
Connection and cursor factories
===============================
psycopg 2 exposes two new-style classes that can be sub-classed and expanded to
adapt them to the needs of the programmer: `cursor` and `connection`. The
`connection` class is usually sub-classed only to provide an easy way to create
customized cursors but other uses are possible. `cursor` is much more
interesting, because it is the class where query building, execution and result
type-casting into Python variables happens.
An example of cursor subclass performing logging is::
import psycopg2
import psycopg2.extensions
import logging
class LoggingCursor(psycopg2.extensions.cursor):
def execute(self, sql, args=None):
logger = logging.getLogger('sql_debug')
logger.info(self.mogrify(sql, args))
try:
psycopg2.extensions.cursor.execute(self, sql, args)
except Exception, exc:
logger.error("%s: %s" % (exc.__class__.__name__, exc))
raise
conn = psycopg2.connect(DSN)
curs = conn.cursor(cursor_factory=LoggingCursor)
curs.execute("INSERT INTO mytable VALUES (%s, %s, %s);",
(10, 20, 30))
Row factories
-------------
tzinfo factories
----------------
Setting transaction isolation levels
====================================
psycopg2 connection objects hold informations about the PostgreSQL `transaction
isolation level`_. The current transaction level can be read from the
`.isolation_level` attribute. The default isolation level is ``READ
COMMITTED``. A different isolation level con be set through the
`.set_isolation_level()` method. The level can be set to one of the following
constants, defined in `psycopg2.extensions`:
`ISOLATION_LEVEL_AUTOCOMMIT`
No transaction is started when command are issued and no
`.commit()`/`.rollback()` is required. Some PostgreSQL command such as
``CREATE DATABASE`` can't run into a transaction: to run such command use
`.set_isolation_level(ISOLATION_LEVEL_AUTOCOMMIT)`.
`ISOLATION_LEVEL_READ_COMMITTED`
This is the default value. A new transaction is started at the first
`.execute()` command on a cursor and at each new `.execute()` after a
`.commit()` or a `.rollback()`. The transaction runs in the PostgreSQL
``READ COMMITTED`` isolation level.
`ISOLATION_LEVEL_SERIALIZABLE`
Transactions are run at a ``SERIALIZABLE`` isolation level.
.. _transaction isolation level:
http://www.postgresql.org/docs/8.1/static/transaction-iso.html
Adaptation of Python values to SQL types
========================================
psycopg2 casts Python variables to SQL literals by type. Standard Python types
are already adapted to the proper SQL literal.
Example: the Python function::
curs.execute("""INSERT INTO atable (anint, adate, astring)
VALUES (%s, %s, %s)""",
(10, datetime.date(2005, 11, 18), "O'Reilly"))
is converted into the SQL command::
INSERT INTO atable (anint, adate, astring)
VALUES (10, '2005-11-18', 'O''Reilly');
Named arguments are supported too with ``%(name)s`` placeholders. Notice that:
- The Python string operator ``%`` is not used: the `.execute()` function
accepts the values tuple or dictionary as second parameter.
- The variables placeholder must always be a ``%s``, even if a different
placeholder (such as a ``%d`` for an integer) may look more appropriate.
- For positional variables binding, the second argument must always be a
tuple, even if it contains a single variable.
- Only variable values should be bound via this method: it shouldn't be used
to set table or field names. For these elements, ordinary string formatting
should be used before running `.execute()`.
Adapting new types
------------------
Any Python class or type can be adapted to an SQL string. Adaptation mechanism
is similar to the Object Adaptation proposed in the `PEP-246`_ and is exposed
by the `adapt()` function.
psycopg2 `.execute()` method adapts its ``vars`` arguments to the `ISQLQuote`
protocol. Objects that conform to this protocol expose a ``getquoted()`` method
returning the SQL representation of the object as a string.
The easiest way to adapt an object to an SQL string is to register an adapter
function via the `register_adapter()` function. The adapter function must take
the value to be adapted as argument and return a conform object. A convenient
object is the `AsIs` wrapper, whose ``getquoted()`` result is simply the
``str()``\ ingification of the wrapped object.
Example: mapping of a ``Point`` class into the ``point`` PostgreSQL geometric
type::
from psycopg2.extensions import adapt, register_adapter, AsIs
class Point(object):
def __init__(self, x=0.0, y=0.0):
self.x = x
self.y = y
def adapt_point(point):
return AsIs("'(%s,%s)'" % (adapt(point.x), adapt(point.y)))
register_adapter(Point, adapt_point)
curs.execute("INSERT INTO atable (apoint) VALUES (%s)",
(Point(1.23, 4.56),))
The above function call results in the SQL command::
INSERT INTO atable (apoint) VALUES ((1.23, 4.56));
.. _PEP-246: http://www.python.org/peps/pep-0246.html
Type casting of SQL types into Python values
============================================
PostgreSQL objects read from the database can be adapted to Python objects
through an user-defined adapting function. An adapter function takes two
argments: the object string representation as returned by PostgreSQL and the
cursor currently being read, and should return a new Python object. For
example, the following function parses a PostgreSQL ``point`` into the
previously defined ``Point`` class::
def cast_point(value, curs):
if value is not None:
# Convert from (f1, f2) syntax using a regular expression.
m = re.match("\((.*),(.*)\)", value)
if m:
return Point(float(m.group(1)), float(m.group(2)))
To create a mapping from the PostgreSQL type (either standard or user-defined),
its ``oid`` must be known. It can be retrieved either by the second column of
the cursor description::
curs.execute("SELECT NULL::point")
point_oid = curs.description[0][1] # usually returns 600
or by querying the system catalogs for the type name and namespace (the
namespace for system objects is ``pg_catalog``)::
curs.execute("""
SELECT pg_type.oid
FROM pg_type JOIN pg_namespace
ON typnamespace = pg_namespace.oid
WHERE typname = %(typename)s
AND nspname = %(namespace)s""",
{'typename': 'point', 'namespace': 'pg_catalog'})
point_oid = curs.fetchone()[0]
After you know the object ``oid``, you must can and register the new type::
POINT = psycopg2.extensions.new_type((point_oid,), "POINT", cast_point)
psycopg2.extensions.register_type(POINT)
The `new_type()` function binds the object oids (more than one can be
specified) to the adapter function. `register_type()` completes the spell.
Conversion is automatically performed when a column whose type is a registered
``oid`` is read::
curs.execute("SELECT '(10.2,20.3)'::point")
point = curs.fetchone()[0]
print type(point), point.x, point.y
# Prints: "<class '__main__.Point'> 10.2 20.3"
Working with times and dates
============================
Receiving NOTIFYs
=================
Using COPY TO and COPY FROM
===========================
psycopg2 `cursor` object provides an interface to the efficient `PostgreSQL
COPY command`__ to move data from files to tables and back.
The `.copy_to(file, table)` method writes the content of the table
named ``table`` *to* the file-like object ``file``. ``file`` must have a
``write()`` method.
The `.copy_from(file, table)` reads data *from* the file-like object
``file`` appending them to the table named ``table``. ``file`` must have both
``read()`` and ``readline()`` method.
Both methods accept two optional arguments: ``sep`` (defaulting to a tab) is
the columns separator and ``null`` (defaulting to ``\N``) represents ``NULL``
values in the file.
.. __: http://www.postgresql.org/docs/8.1/static/sql-copy.html
PostgreSQL status message and executed query
============================================
`cursor` objects have two special fields related to the last executed query:
- `.query` is the textual representation (str or unicode, depending on what
was passed to `.execute()` as first argument) of the query *after* argument
binding and mogrification has been applied. To put it another way, `.query`
is the *exact* query that was sent to the PostgreSQL backend.
- `.statusmessage` is the status message that the backend sent upon query
execution. It usually contains the basic type of the query (SELECT,
INSERT, UPDATE, ...) and some additional information like the number of
rows updated and so on. Refer to the PostgreSQL manual for more
information.

View File

@ -9,15 +9,15 @@ Replaces: 248
Release-Date: 07 Apr 1999
Introduction
This API has been defined to encourage similarity between the
Python modules that are used to access databases. By doing this,
we hope to achieve a consistency leading to more easily understood
modules, code that is generally more portable across databases,
and a broader reach of database connectivity from Python.
The interface specification consists of several sections:
* Module Interface
* Connection Objects
* Cursor Objects
@ -25,7 +25,7 @@ Introduction
* Type Objects and Constructors
* Implementation Hints
* Major Changes from 1.0 to 2.0
Comments and questions about this specification may be directed
to the SIG for Database Interfacing with Python
(db-sig@python.org).
@ -41,7 +41,7 @@ Introduction
basis for new interfaces.
Module Interface
Access to the database is made available through connection
objects. The module must provide the following constructor for
these:
@ -51,17 +51,17 @@ Module Interface
Constructor for creating a connection to the database.
Returns a Connection Object. It takes a number of
parameters which are database dependent. [1]
These module globals must be defined:
apilevel
String constant stating the supported DB API level.
Currently only the strings '1.0' and '2.0' are allowed.
If not given, a DB-API 1.0 level interface should be
assumed.
threadsafety
Integer constant stating the level of thread safety the
@ -81,33 +81,33 @@ Module Interface
or other external sources that are beyond your control.
paramstyle
String constant stating the type of parameter marker
formatting expected by the interface. Possible values are
[2]:
'qmark' Question mark style,
'qmark' Question mark style,
e.g. '...WHERE name=?'
'numeric' Numeric, positional style,
'numeric' Numeric, positional style,
e.g. '...WHERE name=:1'
'named' Named style,
'named' Named style,
e.g. '...WHERE name=:name'
'format' ANSI C printf format codes,
'format' ANSI C printf format codes,
e.g. '...WHERE name=%s'
'pyformat' Python extended format codes,
'pyformat' Python extended format codes,
e.g. '...WHERE name=%(name)s'
The module should make all error information available through
these exceptions or subclasses thereof:
Warning
Warning
Exception raised for important warnings like data
truncations while inserting, etc. It must be a subclass of
the Python StandardError (defined in the module
exceptions).
Error
Error
Exception that is the base class of all other error
exceptions. You can use this to catch all errors with one
@ -115,7 +115,7 @@ Module Interface
errors and thus should not use this class as base. It must
be a subclass of the Python StandardError (defined in the
module exceptions).
InterfaceError
Exception raised for errors that are related to the
@ -126,50 +126,50 @@ Module Interface
Exception raised for errors that are related to the
database. It must be a subclass of Error.
DataError
Exception raised for errors that are due to problems with
the processed data like division by zero, numeric value
out of range, etc. It must be a subclass of DatabaseError.
OperationalError
Exception raised for errors that are related to the
database's operation and not necessarily under the control
of the programmer, e.g. an unexpected disconnect occurs,
the data source name is not found, a transaction could not
be processed, a memory allocation error occurred during
processing, etc. It must be a subclass of DatabaseError.
IntegrityError
IntegrityError
Exception raised when the relational integrity of the
database is affected, e.g. a foreign key check fails. It
must be a subclass of DatabaseError.
InternalError
InternalError
Exception raised when the database encounters an internal
error, e.g. the cursor is not valid anymore, the
transaction is out of sync, etc. It must be a subclass of
DatabaseError.
ProgrammingError
Exception raised for programming errors, e.g. table not
found or already exists, syntax error in the SQL
statement, wrong number of parameters specified, etc. It
must be a subclass of DatabaseError.
NotSupportedError
Exception raised in case a method or database API was used
which is not supported by the database, e.g. requesting a
.rollback() on a connection that does not support
transaction or has transactions turned off. It must be a
subclass of DatabaseError.
This is the exception inheritance layout:
StandardError
@ -183,17 +183,17 @@ Module Interface
|__InternalError
|__ProgrammingError
|__NotSupportedError
Note: The values of these exceptions are not defined. They should
give the user a fairly good idea of what went wrong, though.
Connection Objects
Connection Objects should respond to the following methods:
.close()
.close()
Close the connection now (rather than whenever __del__ is
called). The connection will be unusable from this point
forward; an Error (or subclass) exception will be raised
@ -203,74 +203,74 @@ Connection Objects
committing the changes first will cause an implicit
rollback to be performed.
.commit()
Commit any pending transaction to the database. Note that
if the database supports an auto-commit feature, this must
be initially off. An interface method may be provided to
turn it back on.
Database modules that do not support transactions should
implement this method with void functionality.
.rollback()
.rollback()
This method is optional since not all databases provide
transaction support. [3]
In case a database does provide transactions this method
causes the the database to roll back to the start of any
pending transaction. Closing a connection without
committing the changes first will cause an implicit
rollback to be performed.
.cursor()
Return a new Cursor Object using the connection. If the
database does not provide a direct cursor concept, the
module will have to emulate cursors using other means to
the extent needed by this specification. [4]
Cursor Objects
These objects represent a database cursor, which is used to
manage the context of a fetch operation. Cursors created from
manage the context of a fetch operation. Cursors created from
the same connection are not isolated, i.e., any changes
done to the database by a cursor are immediately visible by the
other cursors. Cursors created from different connections can
or can not be isolated, depending on how the transaction support
is implemented (see also the connection's rollback() and commit()
is implemented (see also the connection's rollback() and commit()
methods.)
Cursor Objects should respond to the following methods and
attributes:
.description
.description
This read-only attribute is a sequence of 7-item
sequences. Each of these sequences contains information
describing one result column: (name, type_code,
display_size, internal_size, precision, scale,
null_ok). The first two items (name and type_code) are
mandatory, the other five are optional and must be set to
None if meaningful values are not provided.
None if meaningfull values are not provided.
This attribute will be None for operations that
do not return rows or if the cursor has not had an
operation invoked via the executeXXX() method yet.
The type_code can be interpreted by comparing it to the
Type Objects specified in the section below.
.rowcount
.rowcount
This read-only attribute specifies the number of rows that
the last executeXXX() produced (for DQL statements like
'select') or affected (for DML statements like 'update' or
'insert').
The attribute is -1 in case no executeXXX() has been
performed on the cursor or the rowcount of the last
operation is not determinable by the interface. [7]
@ -278,96 +278,96 @@ Cursor Objects
Note: Future versions of the DB API specification could
redefine the latter case to have the object return None
instead of -1.
.callproc(procname[,parameters])
(This method is optional since not all databases provide
stored procedures. [3])
Call a stored database procedure with the given name. The
sequence of parameters must contain one entry for each
argument that the procedure expects. The result of the
call is returned as modified copy of the input
sequence. Input parameters are left untouched, output and
input/output parameters replaced with possibly new values.
The procedure may also provide a result set as
output. This must then be made available through the
standard fetchXXX() methods.
.close()
Close the cursor now (rather than whenever __del__ is
called). The cursor will be unusable from this point
forward; an Error (or subclass) exception will be raised
if any operation is attempted with the cursor.
.execute(operation[,parameters])
.execute(operation[,parameters])
Prepare and execute a database operation (query or
command). Parameters may be provided as sequence or
mapping and will be bound to variables in the operation.
Variables are specified in a database-specific notation
(see the module's paramstyle attribute for details). [5]
A reference to the operation will be retained by the
cursor. If the same operation object is passed in again,
then the cursor can optimize its behavior. This is most
effective for algorithms where the same operation is used,
but different parameters are bound to it (many times).
For maximum efficiency when reusing an operation, it is
best to use the setinputsizes() method to specify the
parameter types and sizes ahead of time. It is legal for
a parameter to not match the predefined information; the
implementation should compensate, possibly with a loss of
efficiency.
The parameters may also be specified as list of tuples to
e.g. insert multiple rows in a single operation, but this
kind of usage is depreciated: executemany() should be used
instead.
Return values are not defined.
.executemany(operation,seq_of_parameters)
.executemany(operation,seq_of_parameters)
Prepare a database operation (query or command) and then
execute it against all parameter sequences or mappings
found in the sequence seq_of_parameters.
Modules are free to implement this method using multiple
calls to the execute() method or by using array operations
to have the database process the sequence as a whole in
one call.
Use of this method for an operation which produces one or
more result sets constitutes undefined behavior, and the
implementation is permitted (but not required) to raise
implementation is permitted (but not required) to raise
an exception when it detects that a result set has been
created by an invocation of the operation.
The same comments as for execute() also apply accordingly
to this method.
Return values are not defined.
.fetchone()
.fetchone()
Fetch the next row of a query result set, returning a
single sequence, or None when no more data is
available. [6]
An Error (or subclass) exception is raised if the previous
call to executeXXX() did not produce any result set or no
call was issued yet.
fetchmany([size=cursor.arraysize])
Fetch the next set of rows of a query result, returning a
sequence of sequences (e.g. a list of tuples). An empty
sequence is returned when no more rows are available.
The number of rows to fetch per call is specified by the
parameter. If it is not given, the cursor's arraysize
determines the number of rows to be fetched. The method
@ -375,62 +375,62 @@ Cursor Objects
parameter. If this is not possible due to the specified
number of rows not being available, fewer rows may be
returned.
An Error (or subclass) exception is raised if the previous
call to executeXXX() did not produce any result set or no
call was issued yet.
Note there are performance considerations involved with
the size parameter. For optimal performance, it is
usually best to use the arraysize attribute. If the size
parameter is used, then it is best for it to retain the
same value from one fetchmany() call to the next.
.fetchall()
.fetchall()
Fetch all (remaining) rows of a query result, returning
them as a sequence of sequences (e.g. a list of tuples).
Note that the cursor's arraysize attribute can affect the
performance of this operation.
An Error (or subclass) exception is raised if the previous
call to executeXXX() did not produce any result set or no
call was issued yet.
.nextset()
.nextset()
(This method is optional since not all databases support
multiple result sets. [3])
This method will make the cursor skip to the next
available set, discarding any remaining rows from the
current set.
If there are no more sets, the method returns
None. Otherwise, it returns a true value and subsequent
calls to the fetch methods will return rows from the next
result set.
An Error (or subclass) exception is raised if the previous
call to executeXXX() did not produce any result set or no
call was issued yet.
.arraysize
This read/write attribute specifies the number of rows to
fetch at a time with fetchmany(). It defaults to 1 meaning
to fetch a single row at a time.
Implementations must observe this value with respect to
the fetchmany() method, but are free to interact with the
database a single row at a time. It may also be used in
the implementation of executemany().
.setinputsizes(sizes)
This can be used before a call to executeXXX() to
predefine memory areas for the operation's parameters.
sizes is specified as a sequence -- one item for each
input parameter. The item should be a Type Object that
corresponds to the input that will be used, or it should
@ -438,27 +438,27 @@ Cursor Objects
parameter. If the item is None, then no predefined memory
area will be reserved for that column (this is useful to
avoid predefined areas for large inputs).
This method would be used before the executeXXX() method
is invoked.
Implementations are free to have this method do nothing
and users are free to not use it.
.setoutputsize(size[,column])
Set a column buffer size for fetches of large columns
(e.g. LONGs, BLOBs, etc.). The column is specified as an
index into the result sequence. Not specifying the column
will set the default size for all large columns in the
cursor.
This method would be used before the executeXXX() method
is invoked.
Implementations are free to have this method do nothing
and users are free to not use it.
Type Objects and Constructors
@ -485,15 +485,15 @@ Type Objects and Constructors
Implementation Hints below for details).
The module exports the following constructors and singletons:
Date(year,month,day)
This function constructs an object holding a date value.
Time(hour,minute,second)
This function constructs an object holding a time value.
Timestamp(year,month,day,hour,minute,second)
This function constructs an object holding a time stamp
@ -507,12 +507,12 @@ Type Objects and Constructors
module for details).
TimeFromTicks(ticks)
This function constructs an object holding a time value
from the given ticks value (number of seconds since the
epoch; see the documentation of the standard Python time
module for details).
TimestampFromTicks(ticks)
This function constructs an object holding a time stamp
@ -521,10 +521,10 @@ Type Objects and Constructors
time module for details).
Binary(string)
This function constructs an object capable of holding a
binary (long) string value.
STRING
@ -535,22 +535,22 @@ Type Objects and Constructors
This type object is used to describe (long) binary columns
in a database (e.g. LONG, RAW, BLOBs).
NUMBER
This type object is used to describe numeric columns in a
database.
DATETIME
This type object is used to describe date/time columns in
a database.
ROWID
This type object is used to describe the "Row ID" column
in a database.
SQL NULL values are represented by the Python None singleton on
input and output.
@ -563,7 +563,7 @@ Implementation Hints for Module Authors
* The preferred object types for the date/time objects are those
defined in the mxDateTime package. It provides all necessary
constructors and methods both at Python and C level.
* The preferred object type for Binary objects are the
buffer types available in standard Python starting with
version 1.5.2. Please see the Python documentation for
@ -577,7 +577,7 @@ Implementation Hints for Module Authors
processing. However, it should be noted that this does not
expose a C API like mxDateTime does which means that integration
with C based database modules is more difficult.
* Here is a sample implementation of the Unix ticks based
constructors for date/time delegating work to the generic
constructors:
@ -645,7 +645,7 @@ Implementation Hints for Module Authors
class NotSupportedError(DatabaseError):
pass
In C you can use the PyErr_NewException(fullname,
base, NULL) API to create the exception objects.
@ -760,7 +760,7 @@ Optional DB API Extensions
Warning Message: "DB-API extension connection.messages used"
Cursor Method .next()
Return the next row from the currently executing SQL statement
using the same semantics as .fetchone(). A StopIteration
exception is raised when the result set is exhausted for Python
@ -790,13 +790,13 @@ Optional DB API Extensions
Warning Message: "DB-API extension cursor.lastrowid used"
Optional Error Handling Extension
The core DB API specification only introduces a set of exceptions
which can be raised to report errors to the user. In some cases,
exceptions may be too disruptive for the flow of a program or even
render execution impossible.
render execution impossible.
For these cases and in order to simplify error handling when
dealing with databases, database module authors may choose to
@ -806,7 +806,7 @@ Optional Error Handling Extension
Cursor/Connection Attribute .errorhandler
Read/write attribute which references an error handler to call
in case an error condition is met.
in case an error condition is met.
The handler must be a Python callable taking the following
arguments: errorhandler(connection, cursor, errorclass,
@ -836,7 +836,7 @@ Frequently Asked Questions
specification. This section covers some of the issues people
sometimes have with the specification.
Question:
Question:
How can I construct a dictionary out of the tuples returned by
.fetchxxx():
@ -855,7 +855,7 @@ Frequently Asked Questions
* Some databases don't support case-sensitive column names or
auto-convert them to all lowercase or all uppercase
characters.
* Columns in the result set which are generated by the query
(e.g. using SQL functions) don't map to table column names
and databases usually generate names for these columns in a
@ -872,9 +872,9 @@ Major Changes from Version 1.0 to Version 2.0
compared to the 1.0 version. Because some of these changes will
cause existing DB API 1.0 based scripts to break, the major
version number was adjusted to reflect this change.
These are the most important changes from 1.0 to 2.0:
* The need for a separate dbi module was dropped and the
functionality merged into the module interface itself.
@ -886,10 +886,10 @@ Major Changes from Version 1.0 to Version 2.0
* New constants (apilevel, threadlevel, paramstyle) and
methods (executemany, nextset) were added to provide better
database bindings.
* The semantics of .callproc() needed to call stored
procedures are now clearly defined.
* The definition of the .execute() return value changed.
Previously, the return value was based on the SQL statement
type (which was hard to implement right) -- it is undefined
@ -898,7 +898,7 @@ Major Changes from Version 1.0 to Version 2.0
values, but these are no longer mandated by the
specification and should be considered database interface
dependent.
* Class based exceptions were incorporated into the
specification. Module implementors are free to extend the
exception layout defined in this specification by
@ -916,10 +916,10 @@ Open Issues
questions that were left open in the 1.0 version, there are still
some remaining issues which should be addressed in future
versions:
* Define a useful return value for .nextset() for the case where
a new result set is available.
* Create a fixed point numeric type for use as loss-less
monetary and decimal interchange format.
@ -929,17 +929,17 @@ Footnotes
[1] As a guideline the connection constructor parameters should be
implemented as keyword parameters for more intuitive use and
follow this order of parameters:
dsn Data source name as string
user User name as string (optional)
password Password as string (optional)
host Hostname (optional)
database Database name (optional)
E.g. a connect could look like this:
connect(dsn='myhost:MYDB',user='guido',password='234$')
[2] Module implementors should prefer 'numeric', 'named' or
'pyformat' over the other formats because these offer more
clarity and flexibility.
@ -947,41 +947,41 @@ Footnotes
[3] If the database does not support the functionality required
by the method, the interface should throw an exception in
case the method is used.
The preferred approach is to not implement the method and
thus have Python generate an AttributeError in
case the method is requested. This allows the programmer to
check for database capabilities using the standard
hasattr() function.
For some dynamically configured interfaces it may not be
appropriate to require dynamically making the method
available. These interfaces should then raise a
NotSupportedError to indicate the non-ability
to perform the roll back when the method is invoked.
[4] a database interface may choose to support named cursors by
allowing a string argument to the method. This feature is
not part of the specification, since it complicates
semantics of the .fetchXXX() methods.
[5] The module will use the __getitem__ method of the parameters
object to map either positions (integers) or names (strings)
to parameter values. This allows for both sequences and
mappings to be used as input.
The term "bound" refers to the process of binding an input
value to a database execution buffer. In practical terms,
this means that the input value is directly used as a value
in the operation. The client should not be required to
"escape" the value so that it can be used -- the value
should be equal to the actual database value.
[6] Note that the interface may implement row fetching using
arrays and other optimizations. It is not
guaranteed that a call to this method will only move the
associated cursor forward by one row.
[7] The rowcount attribute may be coded in a way that updates
its value dynamically. This can be useful for databases that
return usable rowcount values only after the first call to

View File

@ -1,68 +0,0 @@
How to make a psycopg2 release
==============================
- Edit ``setup.py`` and set a stable version release. Use PEP 440 to choose
version numbers, e.g.
- ``2.7``: a new major release, new features
- ``2.7.1``: a bugfix release
- ``2.7.1.1``: a release to fix packaging problems
- ``2.7.2.dev0``: version held during development, non-public test packages...
- ``2.8b1``: a beta for public tests
In the rest of this document we assume you have exported the version number
into an environment variable, e.g.::
$ export VERSION=2.8.4
- Push psycopg2 to master or to the maint branch. Make sure tests on `GitHub
Actions`__.
.. __: https://github.com/psycopg/psycopg2/actions/workflows/tests.yml
- Create a signed tag with the content of the relevant NEWS bit and push it.
E.g.::
# Tag name will be 2_8_4
$ git tag -a -s ${VERSION//\./_}
Psycopg 2.8.4 released
What's new in psycopg 2.8.4
---------------------------
New features:
- Fixed bug blah (:ticket:`#42`).
...
- Create the packages:
- On GitHub Actions run manually a `package build workflow`__.
.. __: https://github.com/psycopg/psycopg2/actions/workflows/packages.yml
- When the workflows have finished download the packages from the job
artifacts.
- Only for stable packages: upload the signed packages on PyPI::
$ twine upload -s wheelhouse/psycopg2-${VERSION}/*
- Create a release and release notes in the psycopg website, announce to
psycopg and pgsql-announce mailing lists.
- Edit ``setup.py`` changing the version again (e.g. go to ``2.8.5.dev0``).
Releasing test packages
-----------------------
Test packages may be uploaded on the `PyPI testing site`__ using::
$ twine upload -s -r testpypi wheelhouse/psycopg2-${VERSION}/*
assuming `proper configuration`__ of ``~/.pypirc``.
.. __: https://test.pypi.org/project/psycopg2/
.. __: https://wiki.python.org/moin/TestPyPI

View File

@ -1,2 +0,0 @@
Sphinx
sphinx-better-theme

View File

@ -1,50 +0,0 @@
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
# pip-compile requirements.in
#
alabaster==0.7.13
# via sphinx
babel==2.12.1
# via sphinx
certifi>=2023.7.22
# via requests
charset-normalizer==3.1.0
# via requests
docutils==0.19
# via sphinx
idna==3.4
# via requests
imagesize==1.4.1
# via sphinx
jinja2==3.1.2
# via sphinx
markupsafe==2.1.2
# via jinja2
packaging==23.1
# via sphinx
pygments==2.15.0
# via sphinx
requests==2.31.0
# via sphinx
snowballstemmer==2.2.0
# via sphinx
sphinx==6.1.3
# via -r requirements.in
sphinx-better-theme==0.1.5
# via -r requirements.in
sphinxcontrib-applehelp==1.0.4
# via sphinx
sphinxcontrib-devhelp==1.0.2
# via sphinx
sphinxcontrib-htmlhelp==2.0.1
# via sphinx
sphinxcontrib-jsmath==1.0.1
# via sphinx
sphinxcontrib-qthelp==1.0.3
# via sphinx
sphinxcontrib-serializinghtml==1.1.5
# via sphinx
urllib3==1.26.17
# via requests

View File

@ -1,99 +0,0 @@
# Makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
PAPER =
BUILDDIR = _build
# DSN for the doctest database
PSYCOPG2_DSN="user=postgres dbname=test"
# Internal variables.
PAPEROPT_a4 = -D latex_paper_size=a4
PAPEROPT_letter = -D latex_paper_size=letter
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) .
.PHONY: help clean html dirhtml pickle json htmlhelp qthelp latex changes linkcheck doctest
help:
@echo "Please use \`make <target>' where <target> is one of"
@echo " html to make standalone HTML files"
@echo " dirhtml to make HTML files named index.html in directories"
@echo " pickle to make pickle files"
@echo " json to make JSON files"
@echo " htmlhelp to make HTML files and a HTML help project"
@echo " qthelp to make HTML files and a qthelp project"
@echo " latex to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
@echo " changes to make an overview of all changed/added/deprecated items"
@echo " linkcheck to check all external links for integrity"
@echo " doctest to run all doctests embedded in the documentation (if enabled)"
clean:
-rm -rf $(BUILDDIR)/*
-rm -rf ./html/*
html:
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
dirhtml:
$(SPHINXBUILD) -b dirhtml $(ALLSPHINXOPTS) $(BUILDDIR)/dirhtml
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/dirhtml."
text:
$(SPHINXBUILD) -b text $(ALLSPHINXOPTS) $(BUILDDIR)/text
@echo
@echo "Build finished. The text pages are in $(BUILDDIR)/text."
pickle:
$(SPHINXBUILD) -b pickle $(ALLSPHINXOPTS) $(BUILDDIR)/pickle
@echo
@echo "Build finished; now you can process the pickle files."
json:
$(SPHINXBUILD) -b json $(ALLSPHINXOPTS) $(BUILDDIR)/json
@echo
@echo "Build finished; now you can process the JSON files."
htmlhelp:
$(SPHINXBUILD) -b htmlhelp $(ALLSPHINXOPTS) $(BUILDDIR)/htmlhelp
@echo
@echo "Build finished; now you can run HTML Help Workshop with the" \
".hhp project file in $(BUILDDIR)/htmlhelp."
qthelp:
$(SPHINXBUILD) -b qthelp $(ALLSPHINXOPTS) $(BUILDDIR)/qthelp
@echo
@echo "Build finished; now you can run "qcollectiongenerator" with the" \
".qhcp project file in $(BUILDDIR)/qthelp, like this:"
@echo "# qcollectiongenerator $(BUILDDIR)/qthelp/psycopg.qhcp"
@echo "To view the help file:"
@echo "# assistant -collectionFile $(BUILDDIR)/qthelp/psycopg.qhc"
latex:
$(SPHINXBUILD) -b latex $(ALLSPHINXOPTS) $(BUILDDIR)/latex
@echo
@echo "Build finished; the LaTeX files are in $(BUILDDIR)/latex."
@echo "Run \`make all-pdf' or \`make all-ps' in that directory to" \
"run these through (pdf)latex."
changes:
$(SPHINXBUILD) -b changes $(ALLSPHINXOPTS) $(BUILDDIR)/changes
@echo
@echo "The overview file is in $(BUILDDIR)/changes."
linkcheck:
$(SPHINXBUILD) -b linkcheck $(ALLSPHINXOPTS) $(BUILDDIR)/linkcheck
@echo
@echo "Link check complete; look for any errors in the above output " \
"or in $(BUILDDIR)/linkcheck/output.txt."
doctest:
PSYCOPG2_DSN=$(PSYCOPG2_DSN) \
$(SPHINXBUILD) -b doctest $(ALLSPHINXOPTS) $(BUILDDIR)/doctest
@echo "Testing of doctests in the sources finished, look at the " \
"results in $(BUILDDIR)/doctest/output.txt."

View File

@ -1,136 +0,0 @@
blockquote {
font-style: italic;
}
div.admonition-todo {
background-color: #ffa;
border: 1px solid #ee2;
}
div.dbapi-extension {
background-color: #eef;
border: 1px solid #aaf;
}
code.sql,
tt.sql {
font-size: 1em;
background-color: transparent;
}
a > code.sql,
a > tt.sql {
font-weight: normal;
}
a > code.sql:hover,
a > tt.sql:hover {
text-decoration: underline;
}
dl.faq dt {
font-weight: bold;
}
table.data-types div.line-block {
margin-bottom: 0;
}
/* better theme customisation */
body {
background-color: #216464;
}
header, .related, .document, footer {
background-color: white;
}
header h1 {
font-size: 150%;
margin-bottom: 0;
padding: 0.5rem 10px 0.5rem 10px;
}
h1, h2, h3 {
font-weight: normal;
}
.body h1, .body h2, .body h3 {
color: #074848;
}
h1 {
font-size: 200%;
}
h2 {
font-size: 160%;
}
h3 {
font-size: 140%;
}
footer#pagefooter {
margin-bottom: 1rem;
font-size: 85%;
color: #444;
}
#rellinks, #breadcrumbs {
padding-right: 10px;
padding-left: 10px;
}
.sphinxsidebar {
padding-left: 10px;
}
.bodywrapper {
padding-right: 10px;
}
div.body h1, div.body h2, div.body h3 {
background-color: #f2f2f2;
border-bottom: 1px solid #d0d0d0;
}
div.body p.rubric {
border-bottom: 1px solid #d0d0d0;
}
body .sphinxsidebar .search {
margin-top: 0;
}
html pre {
background-color: #efc;
border: 1px solid #ac9;
border-left: none;
border-right: none;
}
a, a:visited {
color: #0b6868;
}
th {
background-color: #ede;
}
code.xref, a code {
font-weight: bold;
}
code.descname {
font-weight: bold;
font-size: 120%;
}
@media (max-width: 820px) {
body {
background-color: white;
}
}

View File

@ -1,6 +0,0 @@
{# Add a title over the search box #}
{%- if pagename != "search" %}
<h3>Quick search</h3>
{%- include "!searchbox.html" %}
{%- endif %}

View File

@ -1,599 +0,0 @@
More advanced topics
====================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. testsetup:: *
import re
import select
cur.execute("CREATE TABLE atable (apoint point)")
conn.commit()
def wait(conn):
while True:
state = conn.poll()
if state == psycopg2.extensions.POLL_OK:
break
elif state == psycopg2.extensions.POLL_WRITE:
select.select([], [conn.fileno()], [])
elif state == psycopg2.extensions.POLL_READ:
select.select([conn.fileno()], [], [])
else:
raise psycopg2.OperationalError("poll() returned %s" % state)
aconn = psycopg2.connect(database='test', async=1)
wait(aconn)
acurs = aconn.cursor()
.. index::
double: Subclassing; Cursor
double: Subclassing; Connection
.. _subclassing-connection:
.. _subclassing-cursor:
Connection and cursor factories
-------------------------------
Psycopg exposes two new-style classes that can be sub-classed and expanded to
adapt them to the needs of the programmer: `psycopg2.extensions.cursor`
and `psycopg2.extensions.connection`. The `connection` class is
usually sub-classed only to provide an easy way to create customized cursors
but other uses are possible. `cursor` is much more interesting, because
it is the class where query building, execution and result type-casting into
Python variables happens.
The `~psycopg2.extras` module contains several examples of :ref:`connection
and cursor subclasses <cursor-subclasses>`.
.. note::
If you only need a customized cursor class, since Psycopg 2.5 you can use
the `~connection.cursor_factory` parameter of a regular connection instead
of creating a new `!connection` subclass.
.. index::
single: Example; Cursor subclass
An example of cursor subclass performing logging is::
import psycopg2
import psycopg2.extensions
import logging
class LoggingCursor(psycopg2.extensions.cursor):
def execute(self, sql, args=None):
logger = logging.getLogger('sql_debug')
logger.info(self.mogrify(sql, args))
try:
psycopg2.extensions.cursor.execute(self, sql, args)
except Exception, exc:
logger.error("%s: %s" % (exc.__class__.__name__, exc))
raise
conn = psycopg2.connect(DSN)
cur = conn.cursor(cursor_factory=LoggingCursor)
cur.execute("INSERT INTO mytable VALUES (%s, %s, %s);",
(10, 20, 30))
.. index::
single: Objects; Creating new adapters
single: Adaptation; Creating new adapters
single: Data types; Creating new adapters
.. _adapting-new-types:
Adapting new Python types to SQL syntax
---------------------------------------
Any Python class or type can be adapted to an SQL string. Adaptation mechanism
is similar to the Object Adaptation proposed in the :pep:`246` and is exposed
by the `psycopg2.extensions.adapt()` function.
The `~cursor.execute()` method adapts its arguments to the
`~psycopg2.extensions.ISQLQuote` protocol. Objects that conform to this
protocol expose a `!getquoted()` method returning the SQL representation
of the object as a string (the method must return `!bytes` in Python 3).
Optionally the conform object may expose a
`~psycopg2.extensions.ISQLQuote.prepare()` method.
There are two basic ways to have a Python object adapted to SQL:
- the object itself is conform, or knows how to make itself conform. Such
object must expose a `__conform__()` method that will be called with the
protocol object as argument. The object can check that the protocol is
`!ISQLQuote`, in which case it can return `!self` (if the object also
implements `!getquoted()`) or a suitable wrapper object. This option is
viable if you are the author of the object and if the object is specifically
designed for the database (i.e. having Psycopg as a dependency and polluting
its interface with the required methods doesn't bother you). For a simple
example you can take a look at the source code for the
`psycopg2.extras.Inet` object.
- If implementing the `!ISQLQuote` interface directly in the object is not an
option (maybe because the object to adapt comes from a third party library),
you can use an *adaptation function*, taking the object to be adapted as
argument and returning a conforming object. The adapter must be
registered via the `~psycopg2.extensions.register_adapter()` function. A
simple example wrapper is `!psycopg2.extras.UUID_adapter` used by the
`~psycopg2.extras.register_uuid()` function.
A convenient object to write adapters is the `~psycopg2.extensions.AsIs`
wrapper, whose `!getquoted()` result is simply the `!str()`\ ing conversion of
the wrapped object.
.. index::
single: Example; Types adaptation
Example: mapping of a `!Point` class into the |point|_ PostgreSQL
geometric type:
.. doctest::
>>> from psycopg2.extensions import adapt, register_adapter, AsIs
>>> class Point(object):
... def __init__(self, x, y):
... self.x = x
... self.y = y
>>> def adapt_point(point):
... x = adapt(point.x).getquoted()
... y = adapt(point.y).getquoted()
... return AsIs("'(%s, %s)'" % (x, y))
>>> register_adapter(Point, adapt_point)
>>> cur.execute("INSERT INTO atable (apoint) VALUES (%s)",
... (Point(1.23, 4.56),))
.. |point| replace:: :sql:`point`
.. _point: https://www.postgresql.org/docs/current/static/datatype-geometric.html#DATATYPE-GEOMETRIC
The above function call results in the SQL command::
INSERT INTO atable (apoint) VALUES ('(1.23, 4.56)');
.. index:: Type casting
.. _type-casting-from-sql-to-python:
Type casting of SQL types into Python objects
---------------------------------------------
PostgreSQL objects read from the database can be adapted to Python objects
through an user-defined adapting function. An adapter function takes two
arguments: the object string representation as returned by PostgreSQL and the
cursor currently being read, and should return a new Python object. For
example, the following function parses the PostgreSQL :sql:`point`
representation into the previously defined `!Point` class:
>>> def cast_point(value, cur):
... if value is None:
... return None
...
... # Convert from (f1, f2) syntax using a regular expression.
... m = re.match(r"\(([^)]+),([^)]+)\)", value)
... if m:
... return Point(float(m.group(1)), float(m.group(2)))
... else:
... raise InterfaceError("bad point representation: %r" % value)
In order to create a mapping from a PostgreSQL type (either standard or
user-defined), its OID must be known. It can be retrieved either by the second
column of the `cursor.description`:
>>> cur.execute("SELECT NULL::point")
>>> point_oid = cur.description[0][1]
>>> point_oid
600
or by querying the system catalog for the type name and namespace (the
namespace for system objects is :sql:`pg_catalog`):
>>> cur.execute("""
... SELECT pg_type.oid
... FROM pg_type JOIN pg_namespace
... ON typnamespace = pg_namespace.oid
... WHERE typname = %(typename)s
... AND nspname = %(namespace)s""",
... {'typename': 'point', 'namespace': 'pg_catalog'})
>>> point_oid = cur.fetchone()[0]
>>> point_oid
600
After you know the object OID, you can create and register the new type:
>>> POINT = psycopg2.extensions.new_type((point_oid,), "POINT", cast_point)
>>> psycopg2.extensions.register_type(POINT)
The `~psycopg2.extensions.new_type()` function binds the object OIDs
(more than one can be specified) to the adapter function.
`~psycopg2.extensions.register_type()` completes the spell. Conversion
is automatically performed when a column whose type is a registered OID is
read:
>>> cur.execute("SELECT '(10.2,20.3)'::point")
>>> point = cur.fetchone()[0]
>>> print(type(point), point.x, point.y)
<class 'Point'> 10.2 20.3
A typecaster created by `!new_type()` can be also used with
`~psycopg2.extensions.new_array_type()` to create a typecaster converting a
PostgreSQL array into a Python list.
.. index::
pair: Asynchronous; Notifications
pair: LISTEN; SQL command
pair: NOTIFY; SQL command
.. _async-notify:
Asynchronous notifications
--------------------------
Psycopg allows asynchronous interaction with other database sessions using the
facilities offered by PostgreSQL commands |LISTEN|_ and |NOTIFY|_. Please
refer to the PostgreSQL documentation for examples about how to use this form of
communication.
Notifications are instances of the `~psycopg2.extensions.Notify` object made
available upon reception in the `connection.notifies` list. Notifications can
be sent from Python code simply executing a :sql:`NOTIFY` command in an
`~cursor.execute()` call.
Because of the way sessions interact with notifications (see |NOTIFY|_
documentation), you should keep the connection in `~connection.autocommit`
mode if you wish to receive or send notifications in a timely manner.
.. |LISTEN| replace:: :sql:`LISTEN`
.. _LISTEN: https://www.postgresql.org/docs/current/static/sql-listen.html
.. |NOTIFY| replace:: :sql:`NOTIFY`
.. _NOTIFY: https://www.postgresql.org/docs/current/static/sql-notify.html
Notifications are received after every query execution. If the user is
interested in receiving notifications but not in performing any query, the
`~connection.poll()` method can be used to check for new messages without
wasting resources.
A simple application could poll the connection from time to time to check if
something new has arrived. A better strategy is to use some I/O completion
function such as :py:func:`~select.select` to sleep until awakened by the kernel when there is
some data to read on the connection, thereby using no CPU unless there is
something to read::
import select
import psycopg2
import psycopg2.extensions
conn = psycopg2.connect(DSN)
conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
curs = conn.cursor()
curs.execute("LISTEN test;")
print("Waiting for notifications on channel 'test'")
while True:
if select.select([conn],[],[],5) == ([],[],[]):
print("Timeout")
else:
conn.poll()
while conn.notifies:
notify = conn.notifies.pop(0)
print("Got NOTIFY:", notify.pid, notify.channel, notify.payload)
Running the script and executing a command such as :sql:`NOTIFY test, 'hello'`
in a separate :program:`psql` shell, the output may look similar to:
.. code-block:: none
Waiting for notifications on channel 'test'
Timeout
Timeout
Got NOTIFY: 6535 test hello
Timeout
...
Note that the payload is only available from PostgreSQL 9.0: notifications
received from a previous version server will have the
`~psycopg2.extensions.Notify.payload` attribute set to the empty string.
.. versionchanged:: 2.3
Added `~psycopg2.extensions.Notify` object and handling notification
payload.
.. versionchanged:: 2.7
The `~connection.notifies` attribute is writable: it is possible to
replace it with any object exposing an `!append()` method. An useful
example would be to use a `~collections.deque` object.
.. index::
double: Asynchronous; Connection
.. _async-support:
Asynchronous support
--------------------
.. versionadded:: 2.2
Psycopg can issue asynchronous queries to a PostgreSQL database. An asynchronous
communication style is established passing the parameter *async*\=1 to the
`~psycopg2.connect()` function: the returned connection will work in
*asynchronous mode*.
In asynchronous mode, a Psycopg connection will rely on the caller to poll the
socket file descriptor, checking if it is ready to accept data or if a query
result has been transferred and is ready to be read on the client. The caller
can use the method `~connection.fileno()` to get the connection file
descriptor and `~connection.poll()` to make communication proceed according to
the current connection state.
The following is an example loop using methods `!fileno()` and `!poll()`
together with the Python :py:func:`~select.select` function in order to carry on
asynchronous operations with Psycopg::
def wait(conn):
while True:
state = conn.poll()
if state == psycopg2.extensions.POLL_OK:
break
elif state == psycopg2.extensions.POLL_WRITE:
select.select([], [conn.fileno()], [])
elif state == psycopg2.extensions.POLL_READ:
select.select([conn.fileno()], [], [])
else:
raise psycopg2.OperationalError("poll() returned %s" % state)
The above loop of course would block an entire application: in a real
asynchronous framework, `!select()` would be called on many file descriptors
waiting for any of them to be ready. Nonetheless the function can be used to
connect to a PostgreSQL server only using nonblocking commands and the
connection obtained can be used to perform further nonblocking queries. After
`!poll()` has returned `~psycopg2.extensions.POLL_OK`, and thus `!wait()` has
returned, the connection can be safely used:
>>> aconn = psycopg2.connect(database='test', async=1)
>>> wait(aconn)
>>> acurs = aconn.cursor()
Note that there are a few other requirements to be met in order to have a
completely non-blocking connection attempt: see the libpq documentation for
|PQconnectStart|_.
.. |PQconnectStart| replace:: `!PQconnectStart()`
.. _PQconnectStart: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PQCONNECTSTARTPARAMS
The same loop should be also used to perform nonblocking queries: after
sending a query via `~cursor.execute()` or `~cursor.callproc()`, call
`!poll()` on the connection available from `cursor.connection` until it
returns `!POLL_OK`, at which point the query has been completely sent to the
server and, if it produced data, the results have been transferred to the
client and available using the regular cursor methods:
>>> acurs.execute("SELECT pg_sleep(5); SELECT 42;")
>>> wait(acurs.connection)
>>> acurs.fetchone()[0]
42
When an asynchronous query is being executed, `connection.isexecuting()` returns
`!True`. Two cursors can't execute concurrent queries on the same asynchronous
connection.
There are several limitations in using asynchronous connections: the
connection is always in `~connection.autocommit` mode and it is not
possible to change it. So a
transaction is not implicitly started at the first query and is not possible
to use methods `~connection.commit()` and `~connection.rollback()`: you can
manually control transactions using `~cursor.execute()` to send database
commands such as :sql:`BEGIN`, :sql:`COMMIT` and :sql:`ROLLBACK`. Similarly
`~connection.set_session()` can't be used but it is still possible to invoke the
:sql:`SET` command with the proper :sql:`default_transaction_...` parameter.
With asynchronous connections it is also not possible to use
`~connection.set_client_encoding()`, `~cursor.executemany()`, :ref:`large
objects <large-objects>`, :ref:`named cursors <server-side-cursors>`.
:ref:`COPY commands <copy>` are not supported either in asynchronous mode, but
this will be probably implemented in a future release.
.. index::
single: Greenlet
single: Coroutine
single: Eventlet
single: gevent
single: Wait callback
.. _green-support:
Support for coroutine libraries
-------------------------------
.. versionadded:: 2.2
Psycopg can be used together with coroutine_\-based libraries and participate
in cooperative multithreading.
Coroutine-based libraries (such as Eventlet_ or gevent_) can usually patch the
Python standard library in order to enable a coroutine switch in the presence of
blocking I/O: the process is usually referred as making the system *green*, in
reference to the `green threads`_.
Because Psycopg is a C extension module, it is not possible for coroutine
libraries to patch it: Psycopg instead enables cooperative multithreading by
allowing the registration of a *wait callback* using the
`psycopg2.extensions.set_wait_callback()` function. When a wait callback is
registered, Psycopg will use `libpq non-blocking calls`__ instead of the regular
blocking ones, and will delegate to the callback the responsibility to wait
for the socket to become readable or writable.
Working this way, the caller does not have the complete freedom to schedule the
socket check whenever they want as with an :ref:`asynchronous connection
<async-support>`, but has the advantage of maintaining a complete |DBAPI|
semantics: from the point of view of the end user, all Psycopg functions and
objects will work transparently in the coroutine environment (blocking the
calling green thread and giving other green threads the possibility to be
scheduled), allowing non modified code and third party libraries (such as
SQLAlchemy_) to be used in coroutine-based programs.
.. warning::
Psycopg connections are not *green thread safe* and can't be used
concurrently by different green threads. Trying to execute more than one
command at time using one cursor per thread will result in an error (or a
deadlock on versions before 2.4.2).
Therefore, programmers are advised to either avoid sharing connections
between coroutines or to use a library-friendly lock to synchronize shared
connections, e.g. for pooling.
Coroutine libraries authors should provide a callback implementation (and
possibly a method to register it) to make Psycopg as green as they want. An
example callback (using `!select()` to block) is provided as
`psycopg2.extras.wait_select()`: it boils down to something similar to::
def wait_select(conn):
while True:
state = conn.poll()
if state == extensions.POLL_OK:
break
elif state == extensions.POLL_READ:
select.select([conn.fileno()], [], [])
elif state == extensions.POLL_WRITE:
select.select([], [conn.fileno()], [])
else:
raise OperationalError("bad state from poll: %s" % state)
Providing callback functions for the single coroutine libraries is out of
psycopg2 scope, as the callback can be tied to the libraries' implementation
details. You can check the `psycogreen`_ project for further informations and
resources about the topic.
.. _coroutine: https://en.wikipedia.org/wiki/Coroutine
.. _greenlet: https://pypi.org/project/greenlet/
.. _green threads: https://en.wikipedia.org/wiki/Green_threads
.. _Eventlet: https://eventlet.net/
.. _gevent: http://www.gevent.org/
.. _SQLAlchemy: https://www.sqlalchemy.org/
.. _psycogreen: https://github.com/psycopg/psycogreen/
.. __: https://www.postgresql.org/docs/current/static/libpq-async.html
.. warning::
:ref:`COPY commands <copy>` are currently not supported when a wait callback
is registered, but they will be probably implemented in a future release.
:ref:`Large objects <large-objects>` are not supported either: they are
not compatible with asynchronous connections.
.. testcode::
:hide:
aconn.close()
conn.rollback()
cur.execute("DROP TABLE atable")
conn.commit()
cur.close()
conn.close()
.. index::
single: Replication
.. _replication-support:
Replication protocol support
----------------------------
.. versionadded:: 2.7
Modern PostgreSQL servers (version 9.0 and above) support replication. The
replication protocol is built on top of the client-server protocol and can be
operated using ``libpq``, as such it can be also operated by ``psycopg2``.
The replication protocol can be operated on both synchronous and
:ref:`asynchronous <async-support>` connections.
Server version 9.4 adds a new feature called *Logical Replication*.
.. seealso::
- PostgreSQL `Streaming Replication Protocol`__
.. __: https://www.postgresql.org/docs/current/static/protocol-replication.html
Logical replication Quick-Start
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
You must be using PostgreSQL server version 9.4 or above to run this quick
start.
Make sure that replication connections are permitted for user ``postgres`` in
``pg_hba.conf`` and reload the server configuration. You also need to set
``wal_level=logical`` and ``max_wal_senders``, ``max_replication_slots`` to
value greater than zero in ``postgresql.conf`` (these changes require a server
restart). Create a database ``psycopg2_test``.
Then run the following code to quickly try the replication support out. This
is not production code -- it's only intended as a simple demo of logical
replication::
from __future__ import print_function
import sys
import psycopg2
import psycopg2.extras
conn = psycopg2.connect('dbname=psycopg2_test user=postgres',
connection_factory=psycopg2.extras.LogicalReplicationConnection)
cur = conn.cursor()
try:
# test_decoding produces textual output
cur.start_replication(slot_name='pytest', decode=True)
except psycopg2.ProgrammingError:
cur.create_replication_slot('pytest', output_plugin='test_decoding')
cur.start_replication(slot_name='pytest', decode=True)
class DemoConsumer(object):
def __call__(self, msg):
print(msg.payload)
msg.cursor.send_feedback(flush_lsn=msg.data_start)
democonsumer = DemoConsumer()
print("Starting streaming, press Control-C to end...", file=sys.stderr)
try:
cur.consume_stream(democonsumer)
except KeyboardInterrupt:
cur.close()
conn.close()
print("The slot 'pytest' still exists. Drop it with "
"SELECT pg_drop_replication_slot('pytest'); if no longer needed.",
file=sys.stderr)
print("WARNING: Transaction logs will accumulate in pg_xlog "
"until the slot is dropped.", file=sys.stderr)
You can now make changes to the ``psycopg2_test`` database using a normal
psycopg2 session, ``psql``, etc. and see the logical decoding stream printed
by this demo client.
This will continue running until terminated with ``Control-C``.
For the details see :ref:`replication-objects`.

View File

@ -1,289 +0,0 @@
#
# Psycopg documentation build configuration file, created by
# sphinx-quickstart on Sun Feb 7 13:48:41 2010.
#
# This file is execfile()d with the current directory set to its containing dir.
#
# Note that not all possible configuration values are present in this
# autogenerated file.
#
# All configuration values have a default; values that are commented out
# serve to show the default.
import os
import sys
from better import better_theme_path
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
sys.path.append(os.path.abspath('tools/lib'))
# -- General configuration -----------------------------------------------------
# Add any Sphinx extension module names here, as strings. They can be extensions
# coming with Sphinx (named 'sphinx.ext.*') or your custom ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.todo',
'sphinx.ext.ifconfig',
'sphinx.ext.doctest',
'sphinx.ext.intersphinx',
]
# Specific extensions for Psycopg documentation.
extensions += ['dbapi_extension', 'sql_role', 'ticket_role']
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix of source filenames.
source_suffix = '.rst'
# The encoding of source files.
# source_encoding = 'utf-8'
# The master toctree document.
master_doc = 'index'
# General information about the project.
project = 'Psycopg'
copyright = (
'2001-2021, Federico Di Gregorio, Daniele Varrazzo, The Psycopg Team'
)
# The version info for the project you're documenting, acts as replacement for
# |version| and |release|, also used in various other places throughout the
# built documents.
#
# The short X.Y version.
version = '2.0'
# The full version, including alpha/beta/rc tags.
try:
import psycopg2
except ImportError:
print("WARNING: couldn't import psycopg to read version.")
release = version
else:
release = psycopg2.__version__.split()[0]
version = '.'.join(release.split('.')[:2])
intersphinx_mapping = {'py': ('https://docs.python.org/3', None)}
# Pattern to generate links to the bug tracker
ticket_url = 'https://github.com/psycopg/psycopg2/issues/%s'
ticket_remap_until = 25
ticket_remap_offset = 230
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
# language = None
# There are two options for replacing |today|: either, you set today to some
# non-false value, then it is used:
# today = ''
# Else, today_fmt is used as the format for a strftime call.
# today_fmt = '%B %d, %Y'
# List of documents that shouldn't be included in the build.
# unused_docs = []
# List of directories, relative to source directory, that shouldn't be searched
# for source files.
exclude_trees = ['_build', 'html']
# The reST default role (used for this markup: `text`) to use for all documents.
default_role = 'obj'
# If true, '()' will be appended to :func: etc. cross-reference text.
# add_function_parentheses = True
# If true, the current module name will be prepended to all description
# unit titles (such as .. function::).
# add_module_names = True
# If true, sectionauthor and moduleauthor directives will be shown in the
# output. They are ignored by default.
# show_authors = False
# Using 'python' instead of the default gives warnings if parsing an example
# fails, instead of defaulting to none
highlight_language = 'python'
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# A list of ignored prefixes for module index sorting.
# modindex_common_prefix = []
# Include TODO items in the documentation
todo_include_todos = False
rst_epilog = """
.. |DBAPI| replace:: DB API 2.0
.. _DBAPI: https://www.python.org/dev/peps/pep-0249/
.. _transaction isolation level:
https://www.postgresql.org/docs/current/static/transaction-iso.html
.. |MVCC| replace:: :abbr:`MVCC (Multiversion concurrency control)`
"""
# -- Options for HTML output ---------------------------------------------------
# The theme to use for HTML and HTML Help pages. Major themes that come with
# Sphinx are currently 'default' and 'sphinxdoc'.
html_theme = 'better'
# The stylesheet to use with HTML output: this will include the original one
# adding a few classes.
# html_style = 'psycopg.css'
# Hide the sphinx footer
html_show_sphinx = False
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
html_theme_options = {
'linktotheme': False,
'cssfiles': ['_static/psycopg.css'],
}
# Add any paths that contain custom themes here, relative to this directory.
html_theme_path = [better_theme_path]
# The name for this set of Sphinx documents. If None, it defaults to
# "<project> v<release> documentation".
# html_title = None
# A shorter title for the navigation bar. Default is the same as html_title.
html_short_title = 'Home'
# The name of an image file (relative to this directory) to place at the top
# of the sidebar.
# html_logo = None
# The name of an image file (within the static path) to use as favicon of the
# docs. This file should be a Windows icon file (.ico) being 16x16 or 32x32
# pixels large.
# html_favicon = None
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# If not '', a 'Last updated on:' timestamp is inserted at every page bottom,
# using the given strftime format.
# html_last_updated_fmt = '%b %d, %Y'
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
# html_use_smartypants = True
# Custom sidebar templates, maps document names to template names.
# no need for the prev/next topic link using better theme: they are on top
html_sidebars = {
'**': ['localtoc.html', 'searchbox.html'],
}
# Additional templates that should be rendered to pages, maps page names to
# template names.
# html_additional_pages = {}
# If false, no module index is generated.
# html_use_modindex = True
# If false, no index is generated.
# html_use_index = True
# If true, the index is split into individual pages for each letter.
# html_split_index = False
# If true, links to the reST sources are added to the pages.
# html_show_sourcelink = True
# If true, an OpenSearch description file will be output, and all pages will
# contain a <link> tag referring to it. The value of this option must be the
# base URL from which the finished HTML is served.
# html_use_opensearch = ''
# If nonempty, this is the file name suffix for HTML files (e.g. ".xhtml").
# html_file_suffix = ''
# Output file base name for HTML help builder.
htmlhelp_basename = 'psycopgdoc'
# -- Options for LaTeX output --------------------------------------------------
# The paper size ('letter' or 'a4').
# latex_paper_size = 'letter'
# The font size ('10pt', '11pt' or '12pt').
# latex_font_size = '10pt'
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title, author, documentclass [howto/manual]).
latex_documents = [
(
'index',
'psycopg.tex',
'Psycopg Documentation',
'Federico Di Gregorio',
'manual',
)
]
# The name of an image file (relative to this directory) to place at the top of
# the title page.
# latex_logo = None
# For "manual" documents, if this is true, then toplevel headings are parts,
# not chapters.
# latex_use_parts = False
# Additional stuff for the LaTeX preamble.
# latex_preamble = ''
# Documents to append as an appendix to all manuals.
# latex_appendices = []
# If false, no module index is generated.
# latex_use_modindex = True
toc_object_entries = False
doctest_global_setup = """
import os
import psycopg2
def test_connect():
try:
dsn = os.environ['PSYCOPG2_DSN']
except KeyError:
assert False, "You need to set the environment variable PSYCOPG2_DSN" \
" in order to test the documentation!"
return psycopg2.connect(dsn)
conn = test_connect()
cur = conn.cursor()
def drop_test_table(name):
cur.execute("SAVEPOINT drop_test_table;")
try:
cur.execute("DROP TABLE %s;" % name)
except:
cur.execute("ROLLBACK TO SAVEPOINT drop_test_table;")
conn.commit()
def create_test_table():
drop_test_table('test')
cur.execute("CREATE TABLE test (id SERIAL PRIMARY KEY, num INT, data TEXT)")
conn.commit()
"""

View File

@ -1,916 +0,0 @@
The ``connection`` class
========================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. testsetup::
from pprint import pprint
import psycopg2.extensions
drop_test_table('foo')
.. class:: connection
Handles the connection to a PostgreSQL database instance. It encapsulates
a database session.
Connections are created using the factory function
`~psycopg2.connect()`.
Connections are thread safe and can be shared among many threads. See
:ref:`thread-safety` for details.
Connections can be used as context managers. Note that a context wraps a
transaction: if the context exits with success the transaction is
committed, if it exits with an exception the transaction is rolled back.
Note that the connection is not closed by the context and it can be used
for several contexts.
.. code:: python
conn = psycopg2.connect(DSN)
with conn:
with conn.cursor() as curs:
curs.execute(SQL1)
with conn:
with conn.cursor() as curs:
curs.execute(SQL2)
# leaving contexts doesn't close the connection
conn.close()
.. method:: cursor(name=None, cursor_factory=None, scrollable=None, withhold=False)
Return a new `cursor` object using the connection.
If *name* is specified, the returned cursor will be a :ref:`server
side cursor <server-side-cursors>` (also known as *named cursor*).
Otherwise it will be a regular *client side* cursor. By default a
named cursor is declared without :sql:`SCROLL` option and
:sql:`WITHOUT HOLD`: set the argument or property `~cursor.scrollable`
to `!True`/`!False` and or `~cursor.withhold` to `!True` to change the
declaration.
The name can be a string not valid as a PostgreSQL identifier: for
example it may start with a digit and contain non-alphanumeric
characters and quotes.
.. versionchanged:: 2.4
previously only valid PostgreSQL identifiers were accepted as
cursor name.
The *cursor_factory* argument can be used to create non-standard
cursors. The class returned must be a subclass of
`psycopg2.extensions.cursor`. See :ref:`subclassing-cursor` for
details. A default factory for the connection can also be specified
using the `~connection.cursor_factory` attribute.
.. versionchanged:: 2.4.3 added the *withhold* argument.
.. versionchanged:: 2.5 added the *scrollable* argument.
.. extension::
All the function arguments are Psycopg extensions to the |DBAPI|.
.. index::
pair: Transaction; Commit
.. method:: commit()
Commit any pending transaction to the database.
By default, Psycopg opens a transaction before executing the first
command: if `!commit()` is not called, the effect of any data
manipulation will be lost.
The connection can be also set in "autocommit" mode: no transaction is
automatically open, commands have immediate effect. See
:ref:`transactions-control` for details.
.. versionchanged:: 2.5 if the connection is used in a ``with``
statement, the method is automatically called if no exception is
raised in the ``with`` block.
.. index::
pair: Transaction; Rollback
.. method:: rollback()
Roll back to the start of any pending transaction. Closing a
connection without committing the changes first will cause an implicit
rollback to be performed.
.. versionchanged:: 2.5 if the connection is used in a ``with``
statement, the method is automatically called if an exception is
raised in the ``with`` block.
.. method:: close()
Close the connection now (rather than whenever `del` is executed).
The connection will be unusable from this point forward; an
`~psycopg2.InterfaceError` will be raised if any operation is
attempted with the connection. The same applies to all cursor objects
trying to use the connection. Note that closing a connection without
committing the changes first will cause any pending change to be
discarded as if a :sql:`ROLLBACK` was performed (unless a different
isolation level has been selected: see
`~connection.set_isolation_level()`).
.. index::
single: PgBouncer; unclean server
.. versionchanged:: 2.2
previously an explicit :sql:`ROLLBACK` was issued by Psycopg on
`!close()`. The command could have been sent to the backend at an
inappropriate time, so Psycopg currently relies on the backend to
implicitly discard uncommitted changes. Some middleware are known
to behave incorrectly though when the connection is closed during
a transaction (when `~connection.status` is
`~psycopg2.extensions.STATUS_IN_TRANSACTION`), e.g. PgBouncer_
reports an ``unclean server`` and discards the connection. To
avoid this problem you can ensure to terminate the transaction
with a `~connection.commit()`/`~connection.rollback()` before
closing.
.. _PgBouncer: http://www.pgbouncer.org/
.. index::
single: Exceptions; In the connection class
.. rubric:: Exceptions as connection class attributes
The `!connection` also exposes as attributes the same exceptions
available in the `psycopg2` module. See :ref:`dbapi-exceptions`.
.. index::
single: Two-phase commit; methods
.. rubric:: Two-phase commit support methods
.. versionadded:: 2.3
.. seealso:: :ref:`tpc` for an introductory explanation of these methods.
Note that PostgreSQL supports two-phase commit since release 8.1: these
methods raise `~psycopg2.NotSupportedError` if used with an older version
server.
.. _tpc_methods:
.. method:: xid(format_id, gtrid, bqual)
Returns a `~psycopg2.extensions.Xid` instance to be passed to the
`!tpc_*()` methods of this connection. The argument types and
constraints are explained in :ref:`tpc`.
The values passed to the method will be available on the returned
object as the members `~psycopg2.extensions.Xid.format_id`,
`~psycopg2.extensions.Xid.gtrid`, `~psycopg2.extensions.Xid.bqual`.
The object also allows accessing to these members and unpacking as a
3-items tuple.
.. method:: tpc_begin(xid)
Begins a TPC transaction with the given transaction ID *xid*.
This method should be called outside of a transaction (i.e. nothing
may have executed since the last `~connection.commit()` or
`~connection.rollback()` and `connection.status` is
`~psycopg2.extensions.STATUS_READY`).
Furthermore, it is an error to call `!commit()` or `!rollback()`
within the TPC transaction: in this case a `~psycopg2.ProgrammingError`
is raised.
The *xid* may be either an object returned by the `~connection.xid()`
method or a plain string: the latter allows to create a transaction
using the provided string as PostgreSQL transaction id. See also
`~connection.tpc_recover()`.
.. index::
pair: Transaction; Prepare
.. method:: tpc_prepare()
Performs the first phase of a transaction started with
`~connection.tpc_begin()`. A `~psycopg2.ProgrammingError` is raised if
this method is used outside of a TPC transaction.
After calling `!tpc_prepare()`, no statements can be executed until
`~connection.tpc_commit()` or `~connection.tpc_rollback()` will be
called. The `~connection.reset()` method can be used to restore the
status of the connection to `~psycopg2.extensions.STATUS_READY`: the
transaction will remain prepared in the database and will be
possible to finish it with `!tpc_commit(xid)` and
`!tpc_rollback(xid)`.
.. seealso:: the |PREPARE TRANSACTION|_ PostgreSQL command.
.. |PREPARE TRANSACTION| replace:: :sql:`PREPARE TRANSACTION`
.. _PREPARE TRANSACTION: https://www.postgresql.org/docs/current/static/sql-prepare-transaction.html
.. index::
pair: Commit; Prepared
.. method:: tpc_commit([xid])
When called with no arguments, `!tpc_commit()` commits a TPC
transaction previously prepared with `~connection.tpc_prepare()`.
If `!tpc_commit()` is called prior to `!tpc_prepare()`, a single phase
commit is performed. A transaction manager may choose to do this if
only a single resource is participating in the global transaction.
When called with a transaction ID *xid*, the database commits
the given transaction. If an invalid transaction ID is
provided, a `~psycopg2.ProgrammingError` will be raised. This form
should be called outside of a transaction, and is intended for use in
recovery.
On return, the TPC transaction is ended.
.. seealso:: the |COMMIT PREPARED|_ PostgreSQL command.
.. |COMMIT PREPARED| replace:: :sql:`COMMIT PREPARED`
.. _COMMIT PREPARED: https://www.postgresql.org/docs/current/static/sql-commit-prepared.html
.. index::
pair: Rollback; Prepared
.. method:: tpc_rollback([xid])
When called with no arguments, `!tpc_rollback()` rolls back a TPC
transaction. It may be called before or after
`~connection.tpc_prepare()`.
When called with a transaction ID *xid*, it rolls back the given
transaction. If an invalid transaction ID is provided, a
`~psycopg2.ProgrammingError` is raised. This form should be called
outside of a transaction, and is intended for use in recovery.
On return, the TPC transaction is ended.
.. seealso:: the |ROLLBACK PREPARED|_ PostgreSQL command.
.. |ROLLBACK PREPARED| replace:: :sql:`ROLLBACK PREPARED`
.. _ROLLBACK PREPARED: https://www.postgresql.org/docs/current/static/sql-rollback-prepared.html
.. index::
pair: Transaction; Recover
.. method:: tpc_recover()
Returns a list of `~psycopg2.extensions.Xid` representing pending
transactions, suitable for use with `tpc_commit()` or
`tpc_rollback()`.
If a transaction was not initiated by Psycopg, the returned Xids will
have attributes `~psycopg2.extensions.Xid.format_id` and
`~psycopg2.extensions.Xid.bqual` set to `!None` and the
`~psycopg2.extensions.Xid.gtrid` set to the PostgreSQL transaction ID: such Xids are still
usable for recovery. Psycopg uses the same algorithm of the
`PostgreSQL JDBC driver`__ to encode a XA triple in a string, so
transactions initiated by a program using such driver should be
unpacked correctly.
.. __: https://jdbc.postgresql.org/
Xids returned by `!tpc_recover()` also have extra attributes
`~psycopg2.extensions.Xid.prepared`, `~psycopg2.extensions.Xid.owner`,
`~psycopg2.extensions.Xid.database` populated with the values read
from the server.
.. seealso:: the |pg_prepared_xacts|_ system view.
.. |pg_prepared_xacts| replace:: `pg_prepared_xacts`
.. _pg_prepared_xacts: https://www.postgresql.org/docs/current/static/view-pg-prepared-xacts.html
.. extension::
The above methods are the only ones defined by the |DBAPI| protocol.
The Psycopg connection objects exports the following additional
methods and attributes.
.. attribute:: closed
Read-only integer attribute: 0 if the connection is open, nonzero if
it is closed or broken.
.. method:: cancel
Cancel the current database operation.
The method interrupts the processing of the current operation. If no
query is being executed, it does nothing. You can call this function
from a different thread than the one currently executing a database
operation, for instance if you want to cancel a long running query if a
button is pushed in the UI. Interrupting query execution will cause the
cancelled method to raise a
`~psycopg2.extensions.QueryCanceledError`. Note that the termination
of the query is not guaranteed to succeed: see the documentation for
|PQcancel|_.
.. |PQcancel| replace:: `!PQcancel()`
.. _PQcancel: https://www.postgresql.org/docs/current/static/libpq-cancel.html#LIBPQ-PQCANCEL
.. versionadded:: 2.3
.. method:: reset
Reset the connection to the default.
The method rolls back an eventual pending transaction and executes the
PostgreSQL |RESET|_ and |SET SESSION AUTHORIZATION|__ to revert the
session to the default values. A two-phase commit transaction prepared
using `~connection.tpc_prepare()` will remain in the database
available for recover.
.. |RESET| replace:: :sql:`RESET`
.. _RESET: https://www.postgresql.org/docs/current/static/sql-reset.html
.. |SET SESSION AUTHORIZATION| replace:: :sql:`SET SESSION AUTHORIZATION`
.. __: https://www.postgresql.org/docs/current/static/sql-set-session-authorization.html
.. versionadded:: 2.0.12
.. attribute:: dsn
Read-only string containing the connection string used by the
connection.
If a password was specified in the connection string it will be
obscured.
.. rubric:: Transaction control methods and attributes.
.. index::
pair: Transaction; Autocommit
pair: Transaction; Isolation level
.. method:: set_session(isolation_level=None, readonly=None, deferrable=None, autocommit=None)
Set one or more parameters for the next transactions or statements in
the current session.
:param isolation_level: set the `isolation level`_ for the next
transactions/statements. The value can be one of the literal
values ``READ UNCOMMITTED``, ``READ COMMITTED``, ``REPEATABLE
READ``, ``SERIALIZABLE`` or the equivalent :ref:`constant
<isolation-level-constants>` defined in the `~psycopg2.extensions`
module.
:param readonly: if `!True`, set the connection to read only;
read/write if `!False`.
:param deferrable: if `!True`, set the connection to deferrable;
non deferrable if `!False`. Only available from PostgreSQL 9.1.
:param autocommit: switch the connection to autocommit mode: not a
PostgreSQL session setting but an alias for setting the
`autocommit` attribute.
.. _isolation level:
https://www.postgresql.org/docs/current/static/transaction-iso.html
Arguments set to `!None` (the default for all) will not be changed.
The parameters *isolation_level*, *readonly* and *deferrable* also
accept the string ``DEFAULT`` as a value: the effect is to reset the
parameter to the server default. Defaults are defined by the server
configuration: see values for |default_transaction_isolation|__,
|default_transaction_read_only|__, |default_transaction_deferrable|__.
.. |default_transaction_isolation| replace:: :sql:`default_transaction_isolation`
.. __: https://www.postgresql.org/docs/current/static/runtime-config-client.html#GUC-DEFAULT-TRANSACTION-ISOLATION
.. |default_transaction_read_only| replace:: :sql:`default_transaction_read_only`
.. __: https://www.postgresql.org/docs/current/static/runtime-config-client.html#GUC-DEFAULT-TRANSACTION-READ-ONLY
.. |default_transaction_deferrable| replace:: :sql:`default_transaction_deferrable`
.. __: https://www.postgresql.org/docs/current/static/runtime-config-client.html#GUC-DEFAULT-TRANSACTION-DEFERRABLE
The function must be invoked with no transaction in progress.
.. seealso:: |SET TRANSACTION|_ for further details about the behaviour
of the transaction parameters in the server.
.. |SET TRANSACTION| replace:: :sql:`SET TRANSACTION`
.. _SET TRANSACTION: https://www.postgresql.org/docs/current/static/sql-set-transaction.html
.. versionadded:: 2.4.2
.. versionchanged:: 2.7
Before this version, the function would have set
:sql:`default_transaction_*` attribute in the current session;
this implementation has the problem of not playing well with
external connection pooling working at transaction level and not
resetting the state of the session: changing the default
transaction would pollute the connections in the pool and create
problems to other applications using the same pool.
Starting from 2.7, if the connection is not autocommit, the
transaction characteristics are issued together with :sql:`BEGIN`
and will leave the :sql:`default_transaction_*` settings untouched.
For example::
conn.set_session(readonly=True)
will not change :sql:`default_transaction_read_only`, but
following transaction will start with a :sql:`BEGIN READ ONLY`.
Conversely, using::
conn.set_session(readonly=True, autocommit=True)
will set :sql:`default_transaction_read_only` to :sql:`on` and
rely on the server to apply the read only state to whatever
transaction, implicit or explicit, is executed in the connection.
.. attribute:: autocommit
Read/write attribute: if `!True`, no transaction is handled by the
driver and every statement sent to the backend has immediate effect;
if `!False` a new transaction is started at the first command
execution: the methods `commit()` or `rollback()` must be manually
invoked to terminate the transaction.
The autocommit mode is useful to execute commands requiring to be run
outside a transaction, such as :sql:`CREATE DATABASE` or
:sql:`VACUUM`.
The default is `!False` (manual commit) as per DBAPI specification.
.. warning::
By default, any query execution, including a simple :sql:`SELECT`
will start a transaction: for long-running programs, if no further
action is taken, the session will remain "idle in transaction", an
undesirable condition for several reasons (locks are held by
the session, tables bloat...). For long lived scripts, either
ensure to terminate a transaction as soon as possible or use an
autocommit connection.
.. versionadded:: 2.4.2
.. attribute:: isolation_level
Return or set the `transaction isolation level`_ for the current
session. The value is one of the :ref:`isolation-level-constants`
defined in the `psycopg2.extensions` module. On set it is also
possible to use one of the literal values ``READ UNCOMMITTED``, ``READ
COMMITTED``, ``REPEATABLE READ``, ``SERIALIZABLE``, ``DEFAULT``.
.. versionchanged:: 2.7
the property is writable.
.. versionchanged:: 2.7
the default value for `!isolation_level` is
`~psycopg2.extensions.ISOLATION_LEVEL_DEFAULT`; previously the
property would have queried the server and returned the real value
applied. To know this value you can run a query such as :sql:`show
transaction_isolation`. Usually the default value is `READ
COMMITTED`, but this may be changed in the server configuration.
This value is now entirely separate from the `autocommit`
property: in previous version, if `!autocommit` was set to `!True`
this property would have returned
`~psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT`; it will now
return the server isolation level.
.. attribute:: readonly
Return or set the read-only status for the current session. Available
values are `!True` (new transactions will be in read-only mode),
`!False` (new transactions will be writable), `!None` (use the default
configured for the server by :sql:`default_transaction_read_only`).
.. versionadded:: 2.7
.. attribute:: deferrable
Return or set the `deferrable status`__ for the current session.
Available values are `!True` (new transactions will be in deferrable
mode), `!False` (new transactions will be in non deferrable mode),
`!None` (use the default configured for the server by
:sql:`default_transaction_deferrable`).
.. __: `SET TRANSACTION`_
.. versionadded:: 2.7
.. method:: set_isolation_level(level)
.. note::
This is a legacy method mixing `~conn.isolation_level` and
`~conn.autocommit`. Using the respective properties is a better
option.
Set the `transaction isolation level`_ for the current session.
The level defines the different phenomena that can happen in the
database between concurrent transactions.
The value set is an integer: symbolic constants are defined in
the module `psycopg2.extensions`: see
:ref:`isolation-level-constants` for the available values.
The default level is `~psycopg2.extensions.ISOLATION_LEVEL_DEFAULT`:
at this level a transaction is automatically started the first time a
database command is executed. If you want an *autocommit* mode,
switch to `~psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT` before
executing any command::
>>> conn.set_isolation_level(psycopg2.extensions.ISOLATION_LEVEL_AUTOCOMMIT)
See also :ref:`transactions-control`.
.. index::
pair: Client; Encoding
.. attribute:: encoding
.. method:: set_client_encoding(enc)
Read or set the client encoding for the current session. The default
is the encoding defined by the database. It should be one of the
`characters set supported by PostgreSQL`__
.. __: https://www.postgresql.org/docs/current/static/multibyte.html
.. index::
pair: Client; Logging
.. attribute:: notices
A list containing all the database messages sent to the client during
the session.
.. doctest::
:options: +NORMALIZE_WHITESPACE
>>> cur.execute("CREATE TABLE foo (id serial PRIMARY KEY);")
>>> pprint(conn.notices)
['NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index "foo_pkey" for table "foo"\n',
'NOTICE: CREATE TABLE will create implicit sequence "foo_id_seq" for serial column "foo.id"\n']
.. versionchanged:: 2.7
The `!notices` attribute is writable: the user may replace it
with any Python object exposing an `!append()` method. If
appending raises an exception the notice is silently
dropped.
To avoid a leak in case excessive notices are generated, only the last
50 messages are kept. This check is only in place if the `!notices`
attribute is a list: if any other object is used it will be up to the
user to guard from leakage.
You can configure what messages to receive using `PostgreSQL logging
configuration parameters`__ such as ``log_statement``,
``client_min_messages``, ``log_min_duration_statement`` etc.
.. __: https://www.postgresql.org/docs/current/static/runtime-config-logging.html
.. attribute:: notifies
List of `~psycopg2.extensions.Notify` objects containing asynchronous
notifications received by the session.
For other details see :ref:`async-notify`.
.. versionchanged:: 2.3
Notifications are instances of the `!Notify` object. Previously the
list was composed by 2 items tuples :samp:`({pid},{channel})` and
the payload was not accessible. To keep backward compatibility,
`!Notify` objects can still be accessed as 2 items tuples.
.. versionchanged:: 2.7
The `!notifies` attribute is writable: the user may replace it
with any Python object exposing an `!append()` method. If
appending raises an exception the notification is silently
dropped.
.. attribute:: cursor_factory
The default cursor factory used by `~connection.cursor()` if the
parameter is not specified.
.. versionadded:: 2.5
.. index::
pair: Connection; Info
.. attribute:: info
A `~psycopg2.extensions.ConnectionInfo` object exposing information
about the native libpq connection.
.. versionadded:: 2.8
.. index::
pair: Connection; Status
.. attribute:: status
A read-only integer representing the status of the connection.
Symbolic constants for the values are defined in the module
`psycopg2.extensions`: see :ref:`connection-status-constants`
for the available values.
The status is undefined for `closed` connections.
.. method:: lobject([oid [, mode [, new_oid [, new_file [, lobject_factory]]]]])
Return a new database large object as a `~psycopg2.extensions.lobject`
instance.
See :ref:`large-objects` for an overview.
:param oid: The OID of the object to read or write. 0 to create
a new large object and and have its OID assigned automatically.
:param mode: Access mode to the object, see below.
:param new_oid: Create a new object using the specified OID. The
function raises `~psycopg2.OperationalError` if the OID is already
in use. Default is 0, meaning assign a new one automatically.
:param new_file: The name of a file to be imported in the database
(using the |lo_import|_ function)
:param lobject_factory: Subclass of
`~psycopg2.extensions.lobject` to be instantiated.
.. |lo_import| replace:: `!lo_import()`
.. _lo_import: https://www.postgresql.org/docs/current/static/lo-interfaces.html#LO-IMPORT
Available values for *mode* are:
======= =========
*mode* meaning
======= =========
``r`` Open for read only
``w`` Open for write only
``rw`` Open for read/write
``n`` Don't open the file
``b`` Don't decode read data (return data as `!str` in Python 2 or `!bytes` in Python 3)
``t`` Decode read data according to `connection.encoding` (return data as `!unicode` in Python 2 or `!str` in Python 3)
======= =========
``b`` and ``t`` can be specified together with a read/write mode. If
neither ``b`` nor ``t`` is specified, the default is ``b`` in Python 2
and ``t`` in Python 3.
.. versionadded:: 2.0.8
.. versionchanged:: 2.4 added ``b`` and ``t`` mode and unicode
support.
.. rubric:: Methods related to asynchronous support
.. versionadded:: 2.2
.. seealso:: :ref:`async-support` and :ref:`green-support`.
.. attribute:: async
async_
Read only attribute: 1 if the connection is asynchronous, 0 otherwise.
.. versionchanged:: 2.7 added the `!async_` alias for Python versions
where `!async` is a keyword.
.. method:: poll()
Used during an asynchronous connection attempt, or when a cursor is
executing a query on an asynchronous connection, make communication
proceed if it wouldn't block.
Return one of the constants defined in :ref:`poll-constants`. If it
returns `~psycopg2.extensions.POLL_OK` then the connection has been
established or the query results are available on the client.
Otherwise wait until the file descriptor returned by `fileno()` is
ready to read or to write, as explained in :ref:`async-support`.
`poll()` should be also used by the function installed by
`~psycopg2.extensions.set_wait_callback()` as explained in
:ref:`green-support`.
`poll()` is also used to receive asynchronous notifications from the
database: see :ref:`async-notify` from further details.
.. method:: fileno()
Return the file descriptor underlying the connection: useful to read
its status during asynchronous communication.
.. method:: isexecuting()
Return `!True` if the connection is executing an asynchronous operation.
.. rubric:: Interoperation with other C API modules
.. attribute:: pgconn_ptr
Return the internal `!PGconn*` as integer. Useful to pass the libpq
raw connection structure to C functions, e.g. via `ctypes`::
>>> import ctypes
>>> import ctypes.util
>>> libpq = ctypes.pydll.LoadLibrary(ctypes.util.find_library('pq'))
>>> libpq.PQserverVersion.argtypes = [ctypes.c_void_p]
>>> libpq.PQserverVersion.restype = ctypes.c_int
>>> libpq.PQserverVersion(conn.pgconn_ptr)
90611
.. versionadded:: 2.8
.. method:: get_native_connection()
Return the internal `!PGconn*` wrapped in a PyCapsule object. This is
only useful for passing the `libpq` raw connection associated to this
connection object to other C-level modules that may have a use for it.
.. seealso:: Python C API `Capsules`__ docs.
.. __: https://docs.python.org/3.1/c-api/capsule.html
.. versionadded:: 2.8
.. rubric:: informative methods of the native connection
.. note::
These methods are better accessed using the `~connection.info`
attributes and may be dropped in future versions.
.. index::
pair: Transaction; Status
.. method:: get_transaction_status()
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.transaction_status`.
Return the current session transaction status as an integer. Symbolic
constants for the values are defined in the module
`psycopg2.extensions`: see :ref:`transaction-status-constants`
for the available values.
.. seealso:: libpq docs for `PQtransactionStatus()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-status.html#LIBPQ-PQTRANSACTIONSTATUS
.. index::
pair: Protocol; Version
.. attribute:: protocol_version
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.protocol_version`.
A read-only integer representing frontend/backend protocol being used.
Currently Psycopg supports only protocol 3, which allows connection
to PostgreSQL server from version 7.4. Psycopg versions previous than
2.3 support both protocols 2 and 3.
.. seealso:: libpq docs for `PQprotocolVersion()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-status.html#LIBPQ-PQPROTOCOLVERSION
.. versionadded:: 2.0.12
.. index::
pair: Server; Version
.. attribute:: server_version
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.server_version`.
A read-only integer representing the backend version.
The number is formed by converting the major, minor, and revision
numbers into two-decimal-digit numbers and appending them together.
For example, version 8.1.5 will be returned as ``80105``.
.. seealso:: libpq docs for `PQserverVersion()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-status.html#LIBPQ-PQSERVERVERSION
.. versionadded:: 2.0.12
.. index::
pair: Backend; PID
.. method:: get_backend_pid()
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.backend_pid`.
Returns the process ID (PID) of the backend server process *you
connected to*. Note that if you use a connection pool service such as
PgBouncer_ this value will not be updated if your connection is
switched to a different backend.
Note that the PID belongs to a process executing on the database
server host, not the local host!
.. seealso:: libpq docs for `PQbackendPID()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-status.html#LIBPQ-PQBACKENDPID
.. versionadded:: 2.0.8
.. index::
pair: Server; Parameters
.. method:: get_parameter_status(parameter)
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.parameter_status()`.
Look up a current parameter setting of the server.
Potential values for ``parameter`` are: ``server_version``,
``server_encoding``, ``client_encoding``, ``is_superuser``,
``session_authorization``, ``DateStyle``, ``TimeZone``,
``integer_datetimes``, and ``standard_conforming_strings``.
If server did not report requested parameter, return `!None`.
.. seealso:: libpq docs for `PQparameterStatus()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-status.html#LIBPQ-PQPARAMETERSTATUS
.. versionadded:: 2.0.12
.. index::
pair: Connection; Parameters
.. method:: get_dsn_parameters()
Also available as `~connection.info`\ `!.`\
`~psycopg2.extensions.ConnectionInfo.dsn_parameters`.
Get the effective dsn parameters for the connection as a dictionary.
The *password* parameter is removed from the result.
Example::
>>> conn.get_dsn_parameters()
{'dbname': 'test', 'user': 'postgres', 'port': '5432', 'sslmode': 'prefer'}
Requires libpq >= 9.3.
.. seealso:: libpq docs for `PQconninfo()`__ for details.
.. __: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PQCONNINFO
.. versionadded:: 2.7
.. testcode::
:hide:
conn.rollback()

View File

@ -1,682 +0,0 @@
The ``cursor`` class
====================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. testsetup:: *
from StringIO import StringIO
import sys
create_test_table()
# initial data
cur.executemany("INSERT INTO test (num, data) VALUES (%s, %s)",
[(100, "abc'def"), (None, 'dada'), (42, 'bar')])
conn.commit()
.. class:: cursor
Allows Python code to execute PostgreSQL command in a database session.
Cursors are created by the `connection.cursor()` method: they are
bound to the connection for the entire lifetime and all the commands are
executed in the context of the database session wrapped by the connection.
Cursors created from the same connection are not isolated, i.e., any
changes done to the database by a cursor are immediately visible by the
other cursors. Cursors created from different connections can or can not
be isolated, depending on the connections' :ref:`isolation level
<transactions-control>`. See also `~connection.rollback()` and
`~connection.commit()` methods.
Cursors are *not* thread safe: a multithread application can create
many cursors from the same connection and should use each cursor from
a single thread. See :ref:`thread-safety` for details.
Cursors can be used as context managers: leaving the context will close
the cursor.
.. code:: python
with conn.cursor() as curs:
curs.execute(SQL)
# the cursor is now closed
.. attribute:: description
Read-only attribute describing the result of a query. It is a
sequence of `~psycopg2.extensions.Column` instances, each one
describing one result column in order. The attribute is `!None` for
operations that do not return rows or if the cursor has not had an
operation invoked via the |execute*|_ methods yet.
For compatibility with the DB-API, every object can be unpacked as a
7-items sequence: the attributes retuned this way are the following.
For further details and other attributes available check the
`~psycopg2.extensions.Column` documentation.
0. `~psycopg2.extensions.Column.name`: the name of the column returned.
1. `~psycopg2.extensions.Column.type_code`: the PostgreSQL OID of the
column.
2. `~psycopg2.extensions.Column.display_size`: the actual length of
the column in bytes.
3. `~psycopg2.extensions.Column.internal_size`: the size in bytes of
the column associated to this column on the server.
4. `~psycopg2.extensions.Column.precision`: total number of
significant digits in columns of type |NUMERIC|. `!None`
for other types.
5. `~psycopg2.extensions.Column.scale`: count of decimal digits in
the fractional part in columns of type |NUMERIC|. `!None`
for other types.
6. `~psycopg2.extensions.Column.null_ok`: always `!None` as not easy
to retrieve from the libpq.
.. versionchanged:: 2.4
if possible, columns descriptions are named tuple instead of
regular tuples.
.. versionchanged:: 2.8
columns descriptions are instances of `!Column`, exposing extra
attributes.
.. |NUMERIC| replace:: :sql:`NUMERIC`
.. method:: close()
Close the cursor now (rather than whenever `del` is executed).
The cursor will be unusable from this point forward; an
`~psycopg2.InterfaceError` will be raised if any operation is
attempted with the cursor.
.. versionchanged:: 2.5 if the cursor is used in a ``with`` statement,
the method is automatically called at the end of the ``with``
block.
.. attribute:: closed
Read-only boolean attribute: specifies if the cursor is closed
(`!True`) or not (`!False`).
.. extension::
The `closed` attribute is a Psycopg extension to the
|DBAPI|.
.. versionadded:: 2.0.7
.. attribute:: connection
Read-only attribute returning a reference to the `connection`
object on which the cursor was created.
.. attribute:: name
Read-only attribute containing the name of the cursor if it was
created as named cursor by `connection.cursor()`, or `!None` if
it is a client side cursor. See :ref:`server-side-cursors`.
.. extension::
The `name` attribute is a Psycopg extension to the |DBAPI|.
.. attribute:: scrollable
Read/write attribute: specifies if a named cursor is declared
:sql:`SCROLL`, hence is capable to scroll backwards (using
`~cursor.scroll()`). If `!True`, the cursor can be scrolled backwards,
if `!False` it is never scrollable. If `!None` (default) the cursor
scroll option is not specified, usually but not always meaning no
backward scroll (see the |declare-notes|__).
.. |declare-notes| replace:: :sql:`DECLARE` notes
.. __: https://www.postgresql.org/docs/current/static/sql-declare.html#SQL-DECLARE-NOTES
.. note::
set the value before calling `~cursor.execute()` or use the
`connection.cursor()` *scrollable* parameter, otherwise the value
will have no effect.
.. versionadded:: 2.5
.. extension::
The `scrollable` attribute is a Psycopg extension to the |DBAPI|.
.. attribute:: withhold
Read/write attribute: specifies if a named cursor lifetime should
extend outside of the current transaction, i.e., it is possible to
fetch from the cursor even after a `connection.commit()` (but not after
a `connection.rollback()`). See :ref:`server-side-cursors`
.. note::
set the value before calling `~cursor.execute()` or use the
`connection.cursor()` *withhold* parameter, otherwise the value
will have no effect.
.. versionadded:: 2.4.3
.. extension::
The `withhold` attribute is a Psycopg extension to the |DBAPI|.
.. |execute*| replace:: `execute*()`
.. _execute*:
.. rubric:: Commands execution methods
.. method:: execute(query, vars=None)
Execute a database operation (query or command).
Parameters may be provided as sequence or mapping and will be bound to
variables in the operation. Variables are specified either with
positional (``%s``) or named (:samp:`%({name})s`) placeholders. See
:ref:`query-parameters`.
The method returns `!None`. If a query was executed, the returned
values can be retrieved using |fetch*|_ methods.
.. method:: executemany(query, vars_list)
Execute a database operation (query or command) against all parameter
tuples or mappings found in the sequence *vars_list*.
The function is mostly useful for commands that update the database:
any result set returned by the query is discarded.
Parameters are bounded to the query using the same rules described in
the `~cursor.execute()` method.
.. code:: python
>>> nums = ((1,), (5,), (10,))
>>> cur.executemany("INSERT INTO test (num) VALUES (%s)", nums)
>>> tuples = ((123, "foo"), (42, "bar"), (23, "baz"))
>>> cur.executemany("INSERT INTO test (num, data) VALUES (%s, %s)", tuples)
.. warning::
In its current implementation this method is not faster than
executing `~cursor.execute()` in a loop. For better performance
you can use the functions described in :ref:`fast-exec`.
.. method:: callproc(procname [, parameters])
Call a stored database procedure with the given name. The sequence of
parameters must contain one entry for each argument that the procedure
expects. Overloaded procedures are supported. Named parameters can be
used by supplying the parameters as a dictionary.
This function is, at present, not DBAPI-compliant. The return value is
supposed to consist of the sequence of parameters with modified output
and input/output parameters. In future versions, the DBAPI-compliant
return value may be implemented, but for now the function returns None.
The procedure may provide a result set as output. This is then made
available through the standard |fetch*|_ methods.
.. versionchanged:: 2.7
added support for named arguments.
.. note::
`!callproc()` can only be used with PostgreSQL functions__, not
with the procedures__ introduced in PostgreSQL 11, which require
the :sql:`CALL` statement to run. Please use a normal
`execute()` to run them.
.. __: https://www.postgresql.org/docs/current/sql-createfunction.html
.. __: https://www.postgresql.org/docs/current/sql-createprocedure.html
.. method:: mogrify(operation [, parameters])
Return a query string after arguments binding. The string returned is
exactly the one that would be sent to the database running the
`~cursor.execute()` method or similar.
The returned string is always a bytes string.
>>> cur.mogrify("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
"INSERT INTO test (num, data) VALUES (42, E'bar')"
.. extension::
The `mogrify()` method is a Psycopg extension to the |DBAPI|.
.. method:: setinputsizes(sizes)
This method is exposed in compliance with the |DBAPI|. It currently
does nothing but it is safe to call it.
.. |fetch*| replace:: `!fetch*()`
.. _fetch*:
.. rubric:: Results retrieval methods
The following methods are used to read data from the database after an
`~cursor.execute()` call.
.. _cursor-iterable:
.. note::
`cursor` objects are iterable, so, instead of calling
explicitly `~cursor.fetchone()` in a loop, the object itself can
be used:
>>> cur.execute("SELECT * FROM test;")
>>> for record in cur:
... print(record)
...
(1, 100, "abc'def")
(2, None, 'dada')
(3, 42, 'bar')
.. versionchanged:: 2.4
iterating over a :ref:`named cursor <server-side-cursors>`
fetches `~cursor.itersize` records at time from the backend.
Previously only one record was fetched per roundtrip, resulting
in a large overhead.
.. method:: fetchone()
Fetch the next row of a query result set, returning a single tuple,
or `!None` when no more data is available:
>>> cur.execute("SELECT * FROM test WHERE id = %s", (3,))
>>> cur.fetchone()
(3, 42, 'bar')
A `~psycopg2.ProgrammingError` is raised if the previous call
to |execute*|_ did not produce any result set or no call was issued
yet.
.. method:: fetchmany([size=cursor.arraysize])
Fetch the next set of rows of a query result, returning a list of
tuples. An empty list is returned when no more rows are available.
The number of rows to fetch per call is specified by the parameter.
If it is not given, the cursor's `~cursor.arraysize` determines
the number of rows to be fetched. The method should try to fetch as
many rows as indicated by the size parameter. If this is not possible
due to the specified number of rows not being available, fewer rows
may be returned:
>>> cur.execute("SELECT * FROM test;")
>>> cur.fetchmany(2)
[(1, 100, "abc'def"), (2, None, 'dada')]
>>> cur.fetchmany(2)
[(3, 42, 'bar')]
>>> cur.fetchmany(2)
[]
A `~psycopg2.ProgrammingError` is raised if the previous call to
|execute*|_ did not produce any result set or no call was issued yet.
Note there are performance considerations involved with the size
parameter. For optimal performance, it is usually best to use the
`~cursor.arraysize` attribute. If the size parameter is used,
then it is best for it to retain the same value from one
`fetchmany()` call to the next.
.. method:: fetchall()
Fetch all (remaining) rows of a query result, returning them as a list
of tuples. An empty list is returned if there is no more record to
fetch.
>>> cur.execute("SELECT * FROM test;")
>>> cur.fetchall()
[(1, 100, "abc'def"), (2, None, 'dada'), (3, 42, 'bar')]
A `~psycopg2.ProgrammingError` is raised if the previous call to
|execute*|_ did not produce any result set or no call was issued yet.
.. method:: scroll(value [, mode='relative'])
Scroll the cursor in the result set to a new position according
to mode.
If `mode` is ``relative`` (default), value is taken as offset to
the current position in the result set, if set to ``absolute``,
value states an absolute target position.
If the scroll operation would leave the result set, a
`~psycopg2.ProgrammingError` is raised and the cursor position is
not changed.
.. note::
According to the |DBAPI|_, the exception raised for a cursor out
of bound should have been `!IndexError`. The best option is
probably to catch both exceptions in your code::
try:
cur.scroll(1000 * 1000)
except (ProgrammingError, IndexError), exc:
deal_with_it(exc)
The method can be used both for client-side cursors and
:ref:`server-side cursors <server-side-cursors>`. Server-side cursors
can usually scroll backwards only if declared `~cursor.scrollable`.
Moving out-of-bound in a server-side cursor doesn't result in an
exception, if the backend doesn't raise any (Postgres doesn't tell us
in a reliable way if we went out of bound).
.. attribute:: arraysize
This read/write attribute specifies the number of rows to fetch at a
time with `~cursor.fetchmany()`. It defaults to 1 meaning to fetch
a single row at a time.
.. attribute:: itersize
Read/write attribute specifying the number of rows to fetch from the
backend at each network roundtrip during :ref:`iteration
<cursor-iterable>` on a :ref:`named cursor <server-side-cursors>`. The
default is 2000.
.. versionadded:: 2.4
.. extension::
The `itersize` attribute is a Psycopg extension to the |DBAPI|.
.. attribute:: rowcount
This read-only attribute specifies the number of rows that the last
|execute*|_ produced (for :abbr:`DQL (Data Query Language)` statements
like :sql:`SELECT`) or affected (for
:abbr:`DML (Data Manipulation Language)` statements like :sql:`UPDATE`
or :sql:`INSERT`).
The attribute is -1 in case no |execute*| has been performed on
the cursor or the row count of the last operation if it can't be
determined by the interface.
.. note::
The |DBAPI|_ interface reserves to redefine the latter case to
have the object return `!None` instead of -1 in future versions
of the specification.
.. attribute:: rownumber
This read-only attribute provides the current 0-based index of the
cursor in the result set or `!None` if the index cannot be
determined.
The index can be seen as index of the cursor in a sequence (the result
set). The next fetch operation will fetch the row indexed by
`rownumber` in that sequence.
.. index:: oid
.. attribute:: lastrowid
This read-only attribute provides the OID of the last row inserted
by the cursor. If the table wasn't created with OID support or the
last operation is not a single record insert, the attribute is set to
`!None`.
.. note::
PostgreSQL currently advices to not create OIDs on the tables and
the default for |CREATE-TABLE|__ is to not support them. The
|INSERT-RETURNING|__ syntax available from PostgreSQL 8.3 allows
more flexibility.
.. |CREATE-TABLE| replace:: :sql:`CREATE TABLE`
.. __: https://www.postgresql.org/docs/current/static/sql-createtable.html
.. |INSERT-RETURNING| replace:: :sql:`INSERT ... RETURNING`
.. __: https://www.postgresql.org/docs/current/static/sql-insert.html
.. attribute:: query
Read-only attribute containing the body of the last query sent to the
backend (including bound arguments) as bytes string. `!None` if no
query has been executed yet:
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
>>> cur.query
"INSERT INTO test (num, data) VALUES (42, E'bar')"
.. extension::
The `query` attribute is a Psycopg extension to the |DBAPI|.
.. attribute:: statusmessage
Read-only attribute containing the message returned by the last
command:
>>> cur.execute("INSERT INTO test (num, data) VALUES (%s, %s)", (42, 'bar'))
>>> cur.statusmessage
'INSERT 0 1'
.. extension::
The `statusmessage` attribute is a Psycopg extension to the
|DBAPI|.
.. method:: cast(oid, s)
Convert a value from the PostgreSQL string representation to a Python
object.
Use the most specific of the typecasters registered by
`~psycopg2.extensions.register_type()`.
.. versionadded:: 2.4
.. extension::
The `cast()` method is a Psycopg extension to the |DBAPI|.
.. attribute:: tzinfo_factory
The time zone factory used to handle data types such as
:sql:`TIMESTAMP WITH TIME ZONE`. It should be a `~datetime.tzinfo`
object. Default is `datetime.timezone`.
.. versionchanged:: 2.9
previosly the default factory was `psycopg2.tz.FixedOffsetTimezone`.
.. method:: nextset()
This method is not supported (PostgreSQL does not have multiple data
sets) and will raise a `~psycopg2.NotSupportedError` exception.
.. method:: setoutputsize(size [, column])
This method is exposed in compliance with the |DBAPI|. It currently
does nothing but it is safe to call it.
.. rubric:: COPY-related methods
Efficiently copy data from file-like objects to the database and back. See
:ref:`copy` for an overview.
.. extension::
The :sql:`COPY` command is a PostgreSQL extension to the SQL standard.
As such, its support is a Psycopg extension to the |DBAPI|.
.. method:: copy_from(file, table, sep='\\t', null='\\\\N', size=8192, columns=None)
Read data *from* the file-like object *file* appending them to
the table named *table*.
:param file: file-like object to read data from. It must have both
`!read()` and `!readline()` methods.
:param table: name of the table to copy data into.
:param sep: columns separator expected in the file. Defaults to a tab.
:param null: textual representation of :sql:`NULL` in the file.
The default is the two characters string ``\N``.
:param size: size of the buffer used to read from the file.
:param columns: iterable with name of the columns to import.
The length and types should match the content of the file to read.
If not specified, it is assumed that the entire table matches the
file structure.
Example::
>>> f = StringIO("42\tfoo\n74\tbar\n")
>>> cur.copy_from(f, 'test', columns=('num', 'data'))
>>> cur.execute("select * from test where id > 5;")
>>> cur.fetchall()
[(6, 42, 'foo'), (7, 74, 'bar')]
.. versionchanged:: 2.0.6
added the *columns* parameter.
.. versionchanged:: 2.4
data read from files implementing the `io.TextIOBase` interface
are encoded in the connection `~connection.encoding` when sent to
the backend.
.. versionchanged:: 2.9
the table and fields names are now quoted. If you need to specify
a schema-qualified table please use `copy_expert()`.
.. method:: copy_to(file, table, sep='\\t', null='\\\\N', columns=None)
Write the content of the table named *table* *to* the file-like
object *file*. See :ref:`copy` for an overview.
:param file: file-like object to write data into. It must have a
`!write()` method.
:param table: name of the table to copy data from.
:param sep: columns separator expected in the file. Defaults to a tab.
:param null: textual representation of :sql:`NULL` in the file.
The default is the two characters string ``\N``.
:param columns: iterable with name of the columns to export.
If not specified, export all the columns.
Example::
>>> cur.copy_to(sys.stdout, 'test', sep="|")
1|100|abc'def
2|\N|dada
...
.. versionchanged:: 2.0.6
added the *columns* parameter.
.. versionchanged:: 2.4
data sent to files implementing the `io.TextIOBase` interface
are decoded in the connection `~connection.encoding` when read
from the backend.
.. versionchanged:: 2.9
the table and fields names are now quoted. If you need to specify
a schema-qualified table please use `copy_expert()`.
.. method:: copy_expert(sql, file, size=8192)
Submit a user-composed :sql:`COPY` statement. The method is useful to
handle all the parameters that PostgreSQL makes available (see
|COPY|__ command documentation).
:param sql: the :sql:`COPY` statement to execute.
:param file: a file-like object to read or write (according to *sql*).
:param size: size of the read buffer to be used in :sql:`COPY FROM`.
The *sql* statement should be in the form :samp:`COPY {table} TO
STDOUT` to export :samp:`{table}` to the *file* object passed as
argument or :samp:`COPY {table} FROM STDIN` to import the content of
the *file* object into :samp:`{table}`. If you need to compose a
:sql:`COPY` statement dynamically (because table, fields, or query
parameters are in Python variables) you may use the objects provided
by the `psycopg2.sql` module.
*file* must be a readable file-like object (as required by
`~cursor.copy_from()`) for *sql* statement :sql:`COPY ... FROM STDIN`
or a writable one (as required by `~cursor.copy_to()`) for :sql:`COPY
... TO STDOUT`.
Example:
>>> cur.copy_expert("COPY test TO STDOUT WITH CSV HEADER", sys.stdout)
id,num,data
1,100,abc'def
2,,dada
...
.. |COPY| replace:: :sql:`COPY`
.. __: https://www.postgresql.org/docs/current/static/sql-copy.html
.. versionadded:: 2.0.6
.. versionchanged:: 2.4
files implementing the `io.TextIOBase` interface are dealt with
using Unicode data instead of bytes.
.. rubric:: Interoperation with other C API modules
.. attribute:: pgresult_ptr
Return the cursor's internal `!PGresult*` as integer. Useful to pass
the libpq raw result structure to C functions, e.g. via `ctypes`::
>>> import ctypes
>>> libpq = ctypes.pydll.LoadLibrary(ctypes.util.find_library('pq'))
>>> libpq.PQcmdStatus.argtypes = [ctypes.c_void_p]
>>> libpq.PQcmdStatus.restype = ctypes.c_char_p
>>> curs.execute("select 'x'")
>>> libpq.PQcmdStatus(curs.pgresult_ptr)
b'SELECT 1'
.. versionadded:: 2.8
.. testcode::
:hide:
conn.rollback()

View File

@ -1,76 +0,0 @@
`psycopg2.errorcodes` -- Error codes defined by PostgreSQL
===============================================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. index::
single: Error; Codes
.. module:: psycopg2.errorcodes
.. testsetup:: *
from psycopg2 import errorcodes
.. versionadded:: 2.0.6
This module contains symbolic names for all PostgreSQL error codes and error
classes codes. Subclasses of `~psycopg2.Error` make the PostgreSQL error
code available in the `~psycopg2.Error.pgcode` attribute.
From PostgreSQL documentation:
All messages emitted by the PostgreSQL server are assigned five-character
error codes that follow the SQL standard's conventions for :sql:`SQLSTATE`
codes. Applications that need to know which error condition has occurred
should usually test the error code, rather than looking at the textual
error message. The error codes are less likely to change across
PostgreSQL releases, and also are not subject to change due to
localization of error messages. Note that some, but not all, of the error
codes produced by PostgreSQL are defined by the SQL standard; some
additional error codes for conditions not defined by the standard have
been invented or borrowed from other databases.
According to the standard, the first two characters of an error code
denote a class of errors, while the last three characters indicate a
specific condition within that class. Thus, an application that does not
recognize the specific error code can still be able to infer what to do
from the error class.
.. seealso:: `PostgreSQL Error Codes table`__
.. __: https://www.postgresql.org/docs/current/static/errcodes-appendix.html#ERRCODES-TABLE
An example of the available constants defined in the module:
>>> errorcodes.CLASS_SYNTAX_ERROR_OR_ACCESS_RULE_VIOLATION
'42'
>>> errorcodes.UNDEFINED_TABLE
'42P01'
Constants representing all the error values defined by PostgreSQL versions
between 8.1 and 15 are included in the module.
.. autofunction:: lookup(code)
.. doctest::
>>> try:
... cur.execute("SELECT ouch FROM aargh;")
... except Exception as e:
... pass
...
>>> errorcodes.lookup(e.pgcode[:2])
'CLASS_SYNTAX_ERROR_OR_ACCESS_RULE_VIOLATION'
>>> errorcodes.lookup(e.pgcode)
'UNDEFINED_TABLE'
.. versionadded:: 2.0.14
.. testcode::
:hide:
conn.rollback()

View File

@ -1,89 +0,0 @@
`psycopg2.errors` -- Exception classes mapping PostgreSQL errors
================================================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. index::
single: Error; Class
.. module:: psycopg2.errors
.. versionadded:: 2.8
.. versionchanged:: 2.8.4 added errors introduced in PostgreSQL 12
.. versionchanged:: 2.8.6 added errors introduced in PostgreSQL 13
.. versionchanged:: 2.9.2 added errors introduced in PostgreSQL 14
.. versionchanged:: 2.9.4 added errors introduced in PostgreSQL 15
.. versionchanged:: 2.9.10 added errors introduced in PostgreSQL 17
This module exposes the classes psycopg raises upon receiving an error from
the database with a :sql:`SQLSTATE` value attached (available in the
`~psycopg2.Error.pgcode` attribute). The content of the module is generated
from the PostgreSQL source code and includes classes for every error defined
by PostgreSQL in versions between 9.1 and 15.
Every class in the module is named after what referred as "condition name" `in
the documentation`__, converted to CamelCase: e.g. the error 22012,
``division_by_zero`` is exposed by this module as the class `!DivisionByZero`.
.. __: https://www.postgresql.org/docs/current/static/errcodes-appendix.html#ERRCODES-TABLE
Every exception class is a subclass of one of the :ref:`standard DB-API
exception <dbapi-exceptions>` and expose the `~psycopg2.Error` interface.
Each class' superclass is what used to be raised by psycopg in versions before
the introduction of this module, so everything should be compatible with
previously written code catching one the DB-API class: if your code used to
catch `!IntegrityError` to detect a duplicate entry, it will keep on working
even if a more specialised subclass such as `UniqueViolation` is raised.
The new classes allow a more idiomatic way to check and process a specific
error among the many the database may return. For instance, in order to check
that a table is locked, the following code could have been used previously:
.. code-block:: python
try:
cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.OperationalError as e:
if e.pgcode == psycopg2.errorcodes.LOCK_NOT_AVAILABLE:
locked = True
else:
raise
While this method is still available, the specialised class allows for a more
idiomatic error handler:
.. code-block:: python
try:
cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.errors.LockNotAvailable:
locked = True
.. autofunction:: lookup
.. code-block:: python
try:
cur.execute("LOCK TABLE mytable IN ACCESS EXCLUSIVE MODE NOWAIT")
except psycopg2.errors.lookup("55P03"):
locked = True
SQLSTATE exception classes
--------------------------
The following table contains the list of all the SQLSTATE classes exposed by
the module.
Note that, for completeness, the module also exposes all the
:ref:`DB-API-defined exceptions <dbapi-exceptions>` and :ref:`a few
psycopg-specific ones <extension-exceptions>` exposed by the `!extensions`
module, which are not listed here.
.. include:: sqlstate_errors.rst

File diff suppressed because it is too large Load Diff

File diff suppressed because it is too large Load Diff

View File

@ -1,382 +0,0 @@
Frequently Asked Questions
==========================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Here are a few gotchas you may encounter using `psycopg2`. Feel free to
suggest new entries!
Meta
----
.. _faq-question:
.. cssclass:: faq
How do I ask a question?
- Have you first checked if your question is answered already in the
documentation?
- If your question is about installing psycopg, have you checked the
:ref:`install FAQ <faq-compile>` and the :ref:`install docs
<installation>`?
- Have you googled for your error message?
- If you haven't found an answer yet, please write to the `Mailing List`_.
- If you haven't found a bug, DO NOT write to the bug tracker to ask
questions. You will only get piro grumpy.
.. _mailing list: https://www.postgresql.org/list/psycopg/
.. _faq-transactions:
Problems with transactions handling
-----------------------------------
.. _faq-idle-in-transaction:
.. cssclass:: faq
Why does `!psycopg2` leave database sessions "idle in transaction"?
Psycopg normally starts a new transaction the first time a query is
executed, e.g. calling `cursor.execute()`, even if the command is a
:sql:`SELECT`. The transaction is not closed until an explicit
`~connection.commit()` or `~connection.rollback()`.
If you are writing a long-living program, you should probably make sure to
call one of the transaction closing methods before leaving the connection
unused for a long time (which may also be a few seconds, depending on the
concurrency level in your database). Alternatively you can use a
connection in `~connection.autocommit` mode to avoid a new transaction to
be started at the first command.
.. _faq-transaction-aborted:
.. cssclass:: faq
I receive the error *current transaction is aborted, commands ignored until end of transaction block* and can't do anything else!
There was a problem *in the previous* command to the database, which
resulted in an error. The database will not recover automatically from
this condition: you must run a `~connection.rollback()` before sending
new commands to the session (if this seems too harsh, remember that
PostgreSQL supports nested transactions using the |SAVEPOINT|_ command).
.. |SAVEPOINT| replace:: :sql:`SAVEPOINT`
.. _SAVEPOINT: https://www.postgresql.org/docs/current/static/sql-savepoint.html
.. _faq-transaction-aborted-multiprocess:
.. cssclass:: faq
Why do I get the error *current transaction is aborted, commands ignored until end of transaction block* when I use `!multiprocessing` (or any other forking system) and not when use `!threading`?
Psycopg's connections can't be shared across processes (but are thread
safe). If you are forking the Python process make sure to create a new
connection in each forked child. See :ref:`thread-safety` for further
informations.
.. _faq-types:
Problems with type conversions
------------------------------
.. _faq-cant-adapt:
.. cssclass:: faq
Why does `!cursor.execute()` raise the exception *can't adapt*?
Psycopg converts Python objects in a SQL string representation by looking
at the object class. The exception is raised when you are trying to pass
as query parameter an object for which there is no adapter registered for
its class. See :ref:`adapting-new-types` for informations.
.. _faq-number-required:
.. cssclass:: faq
I can't pass an integer or a float parameter to my query: it says *a number is required*, but *it is* a number!
In your query string, you always have to use ``%s`` placeholders,
even when passing a number. All Python objects are converted by Psycopg
in their SQL representation, so they get passed to the query as strings.
See :ref:`query-parameters`. ::
>>> cur.execute("INSERT INTO numbers VALUES (%d)", (42,)) # WRONG
>>> cur.execute("INSERT INTO numbers VALUES (%s)", (42,)) # correct
.. _faq-not-all-arguments-converted:
.. cssclass:: faq
I try to execute a query but it fails with the error *not all arguments converted during string formatting* (or *object does not support indexing*). Why?
Psycopg always require positional arguments to be passed as a sequence, even
when the query takes a single parameter. And remember that to make a
single item tuple in Python you need a comma! See :ref:`query-parameters`.
::
>>> cur.execute("INSERT INTO foo VALUES (%s)", "bar") # WRONG
>>> cur.execute("INSERT INTO foo VALUES (%s)", ("bar")) # WRONG
>>> cur.execute("INSERT INTO foo VALUES (%s)", ("bar",)) # correct
>>> cur.execute("INSERT INTO foo VALUES (%s)", ["bar"]) # correct
.. _faq-unicode:
.. cssclass:: faq
My database is Unicode, but I receive all the strings as UTF-8 `!str`. Can I receive `!unicode` objects instead?
The following magic formula will do the trick::
psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
psycopg2.extensions.register_type(psycopg2.extensions.UNICODEARRAY)
See :ref:`unicode-handling` for the gory details.
.. _faq-bytes:
.. cssclass:: faq
My database is in mixed encoding. My program was working on Python 2 but Python 3 fails decoding the strings. How do I avoid decoding?
From psycopg 2.8 you can use the following adapters to always return bytes
from strings::
psycopg2.extensions.register_type(psycopg2.extensions.BYTES)
psycopg2.extensions.register_type(psycopg2.extensions.BYTESARRAY)
See :ref:`unicode-handling` for an example.
.. _faq-float:
.. cssclass:: faq
Psycopg converts :sql:`decimal`\/\ :sql:`numeric` database types into Python `!Decimal` objects. Can I have `!float` instead?
You can register a customized adapter for PostgreSQL decimal type::
DEC2FLOAT = psycopg2.extensions.new_type(
psycopg2.extensions.DECIMAL.values,
'DEC2FLOAT',
lambda value, curs: float(value) if value is not None else None)
psycopg2.extensions.register_type(DEC2FLOAT)
See :ref:`type-casting-from-sql-to-python` to read the relevant
documentation. If you find `!psycopg2.extensions.DECIMAL` not available, use
`!psycopg2._psycopg.DECIMAL` instead.
.. _faq-json-adapt:
.. cssclass:: faq
Psycopg automatically converts PostgreSQL :sql:`json` data into Python objects. How can I receive strings instead?
The easiest way to avoid JSON parsing is to register a no-op function with
`~psycopg2.extras.register_default_json()`::
psycopg2.extras.register_default_json(loads=lambda x: x)
See :ref:`adapt-json` for further details.
.. _faq-jsonb-adapt:
.. cssclass:: faq
Psycopg converts :sql:`json` values into Python objects but :sql:`jsonb` values are returned as strings. Can :sql:`jsonb` be converted automatically?
Automatic conversion of :sql:`jsonb` values is supported from Psycopg
release 2.5.4. For previous versions you can register the :sql:`json`
typecaster on the :sql:`jsonb` oids (which are known and not supposed to
change in future PostgreSQL versions)::
psycopg2.extras.register_json(oid=3802, array_oid=3807, globally=True)
See :ref:`adapt-json` for further details.
.. _faq-identifier:
.. cssclass:: faq
How can I pass field/table names to a query?
The arguments in the `~cursor.execute()` methods can only represent data
to pass to the query: they cannot represent a table or field name::
# This doesn't work
cur.execute("insert into %s values (%s)", ["my_table", 42])
If you want to build a query dynamically you can use the objects exposed
by the `psycopg2.sql` module::
cur.execute(
sql.SQL("insert into %s values (%%s)") % [sql.Identifier("my_table")],
[42])
.. _faq-bytea-9.0:
.. cssclass:: faq
Transferring binary data from PostgreSQL 9.0 doesn't work.
PostgreSQL 9.0 uses by default `the "hex" format`__ to transfer
:sql:`bytea` data: the format can't be parsed by the libpq 8.4 and
earlier. The problem is solved in Psycopg 2.4.1, that uses its own parser
for the :sql:`bytea` format. For previous Psycopg releases, three options
to solve the problem are:
- set the bytea_output__ parameter to ``escape`` in the server;
- execute the database command ``SET bytea_output TO escape;`` in the
session before reading binary data;
- upgrade the libpq library on the client to at least 9.0.
.. __: https://www.postgresql.org/docs/current/static/datatype-binary.html
.. __: https://www.postgresql.org/docs/current/static/runtime-config-client.html#GUC-BYTEA-OUTPUT
.. _faq-array:
.. cssclass:: faq
Arrays of *TYPE* are not casted to list.
Arrays are only casted to list when their oid is known, and an array
typecaster is registered for them. If there is no typecaster, the array is
returned unparsed from PostgreSQL (e.g. ``{a,b,c}``). It is easy to create
a generic arrays typecaster, returning a list of array: an example is
provided in the `~psycopg2.extensions.new_array_type()` documentation.
.. _faq-best-practices:
Best practices
--------------
.. _faq-reuse-cursors:
.. cssclass:: faq
When should I save and re-use a cursor as opposed to creating a new one as needed?
Cursors are lightweight objects and creating lots of them should not pose
any kind of problem. But note that cursors used to fetch result sets will
cache the data and use memory in proportion to the result set size. Our
suggestion is to almost always create a new cursor and dispose old ones as
soon as the data is not required anymore (call `~cursor.close()` on
them.) The only exception are tight loops where one usually use the same
cursor for a whole bunch of :sql:`INSERT`\s or :sql:`UPDATE`\s.
.. _faq-reuse-connections:
.. cssclass:: faq
When should I save and re-use a connection as opposed to creating a new one as needed?
Creating a connection can be slow (think of SSL over TCP) so the best
practice is to create a single connection and keep it open as long as
required. It is also good practice to rollback or commit frequently (even
after a single :sql:`SELECT` statement) to make sure the backend is never
left "idle in transaction". See also `psycopg2.pool` for lightweight
connection pooling.
.. _faq-named-cursors:
.. cssclass:: faq
What are the advantages or disadvantages of using named cursors?
The only disadvantages is that they use up resources on the server and
that there is a little overhead because at least two queries (one to
create the cursor and one to fetch the initial result set) are issued to
the backend. The advantage is that data is fetched one chunk at a time:
using small `~cursor.fetchmany()` values it is possible to use very
little memory on the client and to skip or discard parts of the result set.
.. _faq-interrupt-query:
.. cssclass:: faq
How do I interrupt a long-running query in an interactive shell?
Normally the interactive shell becomes unresponsive to :kbd:`Ctrl-C` when
running a query. Using a connection in green mode allows Python to
receive and handle the interrupt, although it may leave the connection
broken, if the async callback doesn't handle the `!KeyboardInterrupt`
correctly.
Starting from psycopg 2.6.2, the `~psycopg2.extras.wait_select` callback
can handle a :kbd:`Ctrl-C` correctly. For previous versions, you can use
`this implementation`__.
.. __: https://www.psycopg.org/articles/2014/07/20/cancelling-postgresql-statements-python/
.. code-block:: pycon
>>> psycopg2.extensions.set_wait_callback(psycopg2.extras.wait_select)
>>> cnn = psycopg2.connect('')
>>> cur = cnn.cursor()
>>> cur.execute("select pg_sleep(10)")
^C
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
QueryCanceledError: canceling statement due to user request
>>> cnn.rollback()
>>> # You can use the connection and cursor again from here
.. _faq-compile:
Problems compiling and installing psycopg2
------------------------------------------
.. _faq-wheels:
.. cssclass:: faq
Psycopg 2.8 fails to install, Psycopg 2.7 was working fine.
With Psycopg 2.7 you were installing binary packages, but they have proven
unreliable so now you have to install them explicitly using the
``psycopg2-binary`` package. See :ref:`binary-packages` for all the
details.
.. _faq-python-h:
.. cssclass:: faq
I can't compile `!psycopg2`: the compiler says *error: Python.h: No such file or directory*. What am I missing?
You need to install a Python development package: it is usually called
``python-dev`` or ``python3-dev`` according to your Python version.
.. _faq-libpq-fe-h:
.. cssclass:: faq
I can't compile `!psycopg2`: the compiler says *error: libpq-fe.h: No such file or directory*. What am I missing?
You need to install the development version of the libpq: the package is
usually called ``libpq-dev``.
.. _faq-lo_truncate:
.. cssclass:: faq
`!psycopg2` raises `!ImportError` with message *_psycopg.so: undefined symbol: lo_truncate* when imported.
This means that Psycopg was compiled with |lo_truncate|_ support (*i.e.*
the libpq used at compile time was version >= 8.3) but at runtime an older
libpq dynamic library is found.
Fast-forward several years, if the message reports *undefined symbol:
lo_truncate64* it means that Psycopg was built with large objects 64 bits
API support (*i.e.* the libpq used at compile time was at least 9.3) but
at runtime an older libpq dynamic library is found.
You can use:
.. code-block:: shell
$ ldd /path/to/packages/psycopg2/_psycopg.so | grep libpq
to find what is the libpq dynamic library used at runtime.
You can avoid the problem by using the same version of the
:program:`pg_config` at install time and the libpq at runtime.
.. |lo_truncate| replace:: `!lo_truncate()`
.. _lo_truncate: https://www.postgresql.org/docs/current/static/lo-interfaces.html#LO-TRUNCATE
.. _faq-import-mod_wsgi:
.. cssclass:: faq
Psycopg raises *ImportError: cannot import name tz* on import in mod_wsgi / ASP, but it works fine otherwise.
If `!psycopg2` is installed in an egg_ (e.g. because installed by
:program:`easy_install`), the user running the program may be unable to
write in the `eggs cache`__. Set the env variable
:envvar:`PYTHON_EGG_CACHE` to a writable directory. With modwsgi you can
use the WSGIPythonEggs__ directive.
.. _egg: http://peak.telecommunity.com/DevCenter/PythonEggs
.. __: https://stackoverflow.com/questions/2192323/what-is-the-python-egg-cache-python-egg-cache
.. __: https://modwsgi.readthedocs.io/en/develop/configuration-directives/WSGIPythonEggs.html

View File

@ -1,70 +0,0 @@
=================================================
Psycopg -- PostgreSQL database adapter for Python
=================================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Psycopg_ is the most popular PostgreSQL_ database adapter for the Python_
programming language. Its main features are the complete implementation of
the Python |DBAPI|_ specification and the thread safety (several threads can
share the same connection). It was designed for heavily multi-threaded
applications that create and destroy lots of cursors and make a large number
of concurrent :sql:`INSERT`\s or :sql:`UPDATE`\s.
Psycopg 2 is mostly implemented in C as a libpq_ wrapper, resulting in being
both efficient and secure. It features client-side and :ref:`server-side
<server-side-cursors>` cursors, :ref:`asynchronous communication
<async-support>` and :ref:`notifications <async-notify>`, :ref:`COPY <copy>`
support. Many Python types are supported out-of-the-box and :ref:`adapted to
matching PostgreSQL data types <python-types-adaptation>`; adaptation can be
extended and customized thanks to a flexible :ref:`objects adaptation system
<adapting-new-types>`.
Psycopg 2 is both Unicode and Python 3 friendly.
.. _Psycopg: https://psycopg.org/
.. _PostgreSQL: https://www.postgresql.org/
.. _Python: https://www.python.org/
.. _libpq: https://www.postgresql.org/docs/current/static/libpq.html
.. rubric:: Contents
.. toctree::
:maxdepth: 2
install
usage
module
connection
cursor
advanced
extensions
extras
errors
sql
tz
pool
errorcodes
faq
news
license
.. ifconfig:: builder != 'text'
.. rubric:: Indices and tables
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
.. ifconfig:: todo_include_todos
.. note::
**To Do items in the documentation**
.. todolist::

View File

@ -1,364 +0,0 @@
.. _installation:
Installation
============
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
Psycopg is a PostgreSQL_ adapter for the Python_ programming language. It is a
wrapper for the libpq_, the official PostgreSQL client library.
.. _PostgreSQL: https://www.postgresql.org/
.. _Python: https://www.python.org/
.. index::
single: Install; from PyPI
single: Install; wheel
single: Wheel
.. _binary-packages:
Quick Install
-------------
For most operating systems, the quickest way to install Psycopg is using the
wheel_ package available on PyPI_:
.. code-block:: console
$ pip install psycopg2-binary
This will install a pre-compiled binary version of the module which does not
require the build or runtime prerequisites described below. Make sure to use
an up-to-date version of :program:`pip` (you can upgrade it using something
like ``pip install -U pip``).
You may then import the ``psycopg2`` package, as usual:
.. code-block:: python
import psycopg2
# Connect to your postgres DB
conn = psycopg2.connect("dbname=test user=postgres")
# Open a cursor to perform database operations
cur = conn.cursor()
# Execute a query
cur.execute("SELECT * FROM my_data")
# Retrieve query results
records = cur.fetchall()
.. _PyPI: https://pypi.org/project/psycopg2-binary/
.. _wheel: https://pythonwheels.com/
psycopg vs psycopg-binary
^^^^^^^^^^^^^^^^^^^^^^^^^
The ``psycopg2-binary`` package is meant for beginners to start playing
with Python and PostgreSQL without the need to meet the build
requirements.
If you are the maintainer of a published package depending on `!psycopg2`
you shouldn't use ``psycopg2-binary`` as a module dependency. **For
production use you are advised to use the source distribution.**
The binary packages come with their own versions of a few C libraries,
among which ``libpq`` and ``libssl``, which will be used regardless of other
libraries available on the client: upgrading the system libraries will not
upgrade the libraries used by `!psycopg2`. Please build `!psycopg2` from
source if you want to maintain binary upgradeability.
.. warning::
The `!psycopg2` wheel package comes packaged, among the others, with its
own ``libssl`` binary. This may create conflicts with other extension
modules binding with ``libssl`` as well, for instance with the Python
`ssl` module: in some cases, under concurrency, the interaction between
the two libraries may result in a segfault. In case of doubts you are
advised to use a package built from source.
.. index::
single: Install; disable wheel
single: Wheel; disable
.. _disable-wheel:
Change in binary packages between Psycopg 2.7 and 2.8
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
In version 2.7.x, :command:`pip install psycopg2` would have tried to install
automatically the binary package of Psycopg. Because of concurrency problems
binary packages have displayed, ``psycopg2-binary`` has become a separate
package, and from 2.8 it has become the only way to install the binary
package.
If you are using Psycopg 2.7 and you want to disable the use of wheel binary
packages, relying on the system libraries available on your client, you
can use the :command:`pip` |--no-binary option|__, e.g.:
.. code-block:: console
$ pip install --no-binary :all: psycopg2
.. |--no-binary option| replace:: ``--no-binary`` option
.. __: https://pip.pypa.io/en/stable/reference/pip_install/#install-no-binary
which can be specified in your :file:`requirements.txt` files too, e.g. use:
.. code-block:: none
psycopg2>=2.7,<2.8 --no-binary psycopg2
to use the last bugfix release of the `!psycopg2` 2.7 package, specifying to
always compile it from source. Of course in this case you will have to meet
the :ref:`build prerequisites <build-prerequisites>`.
.. index::
single: Prerequisites
Prerequisites
-------------
The current `!psycopg2` implementation supports:
..
NOTE: keep consistent with setup.py and the /features/ page.
- Python versions from 3.8 to 3.13
- PostgreSQL server versions from 7.4 to 17
- PostgreSQL client library version from 9.1
.. note::
Not all the psycopg2 versions support all the supported Python versions.
Please see the :ref:`release notes <news>` to verify when the support for
a new Python version was added and when the support for an old Python
version was removed.
.. _build-prerequisites:
Build prerequisites
^^^^^^^^^^^^^^^^^^^
The build prerequisites are to be met in order to install Psycopg from source
code, from a source distribution package, GitHub_ or from PyPI.
.. _GitHub: https://github.com/psycopg/psycopg2
Psycopg is a C wrapper around the libpq_ PostgreSQL client library. To install
it from sources you will need:
- A C compiler.
- The Python header files. They are usually installed in a package such as
**python-dev** or **python3-dev**. A message such as *error: Python.h: No
such file or directory* is an indication that the Python headers are
missing.
- The libpq header files. They are usually installed in a package such as
**libpq-dev**. If you get an *error: libpq-fe.h: No such file or directory*
you are missing them.
- The :program:`pg_config` program: it is usually installed by the
**libpq-dev** package but sometimes it is not in a :envvar:`PATH` directory.
Having it in the :envvar:`PATH` greatly streamlines the installation, so try
running ``pg_config --version``: if it returns an error or an unexpected
version number then locate the directory containing the :program:`pg_config`
shipped with the right libpq version (usually
``/usr/lib/postgresql/X.Y/bin/``) and add it to the :envvar:`PATH`:
.. code-block:: console
$ export PATH=/usr/lib/postgresql/X.Y/bin/:$PATH
You only need :program:`pg_config` to compile `!psycopg2`, not for its
regular usage.
Once everything is in place it's just a matter of running the standard:
.. code-block:: console
$ pip install psycopg2
or, from the directory containing the source code:
.. code-block:: console
$ python setup.py build
$ python setup.py install
Runtime requirements
^^^^^^^^^^^^^^^^^^^^
Unless you compile `!psycopg2` as a static library, or you install it from a
self-contained wheel package, it will need the libpq_ library at runtime
(usually distributed in a ``libpq.so`` or ``libpq.dll`` file). `!psycopg2`
relies on the host OS to find the library if the library is installed in a
standard location there is usually no problem; if the library is in a
non-standard location you will have to tell Psycopg how to find it,
which is OS-dependent (for instance setting a suitable
:envvar:`LD_LIBRARY_PATH` on Linux).
.. note::
The libpq header files used to compile `!psycopg2` should match the
version of the library linked at runtime. If you get errors about missing
or mismatching libraries when importing `!psycopg2` check (e.g. using
:program:`ldd`) if the module ``psycopg2/_psycopg.so`` is linked to the
right ``libpq.so``.
.. note::
Whatever version of libpq `!psycopg2` is compiled with, it will be
possible to connect to PostgreSQL servers of any supported version: just
install the most recent libpq version or the most practical, without
trying to match it to the version of the PostgreSQL server you will have
to connect to.
.. index::
single: setup.py
single: setup.cfg
Non-standard builds
-------------------
If you have less standard requirements such as:
- creating a :ref:`debug build <debug-build>`,
- using :program:`pg_config` not in the :envvar:`PATH`,
then take a look at the ``setup.cfg`` file.
Some of the options available in ``setup.cfg`` are also available as command
line arguments of the ``build_ext`` sub-command. For instance you can specify
an alternate :program:`pg_config` location using:
.. code-block:: console
$ python setup.py build_ext --pg-config /path/to/pg_config build
Use ``python setup.py build_ext --help`` to get a list of the options
supported.
.. index::
single: debug
single: PSYCOPG_DEBUG
.. _debug-build:
Creating a debug build
^^^^^^^^^^^^^^^^^^^^^^
In case of problems, Psycopg can be configured to emit detailed debug
messages, which can be very useful for diagnostics and to report a bug. In
order to create a debug package:
- `Download`__ and unpack the Psycopg *source package* (the ``.tar.gz``
package).
- Edit the ``setup.cfg`` file adding the ``PSYCOPG_DEBUG`` flag to the
``define`` option.
- :ref:`Compile and install <build-prerequisites>` the package.
- Set the :envvar:`PSYCOPG_DEBUG` environment variable:
.. code-block:: console
$ export PSYCOPG_DEBUG=1
- Run your program (making sure that the `!psycopg2` package imported is the
one you just compiled and not e.g. the system one): you will have a copious
stream of informations printed on stderr.
.. __: https://pypi.org/project/psycopg2/#files
Non-standard Python Implementation
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The `psycopg2` package is the current mature implementation of the adapter: it
is a C extension and as such it is only compatible with CPython_. If you want
to use Psycopg on a different Python implementation (PyPy, Jython, IronPython)
there is a couple of alternative:
- a `Ctypes port`__, but it is not as mature as the C implementation yet
and it is not as feature-complete;
- a `CFFI port`__ which is currently more used and reported more efficient on
PyPy, but please be careful of its version numbers because they are not
aligned to the official psycopg2 ones and some features may differ.
.. _PostgreSQL: https://www.postgresql.org/
.. _Python: https://www.python.org/
.. _libpq: https://www.postgresql.org/docs/current/static/libpq.html
.. _CPython: https://en.wikipedia.org/wiki/CPython
.. _Ctypes: https://docs.python.org/library/ctypes.html
.. __: https://github.com/mvantellingen/psycopg2-ctypes
.. __: https://github.com/chtd/psycopg2cffi
.. index::
single: tests
.. _test-suite:
Running the test suite
----------------------
Once `!psycopg2` is installed you can run the test suite to verify it is
working correctly. From the source directory, you can run:
.. code-block:: console
$ python -c "import tests; tests.unittest.main(defaultTest='tests.test_suite')" --verbose
The tests run against a database called ``psycopg2_test`` on UNIX socket and
the standard port. You can configure a different database to run the test by
setting the environment variables:
- :envvar:`PSYCOPG2_TESTDB`
- :envvar:`PSYCOPG2_TESTDB_HOST`
- :envvar:`PSYCOPG2_TESTDB_PORT`
- :envvar:`PSYCOPG2_TESTDB_USER`
The database should already exist before running the tests.
.. _other-problems:
If you still have problems
--------------------------
Try the following. *In order:*
- Read again the :ref:`build-prerequisites`.
- Read the :ref:`FAQ <faq-compile>`.
- Google for `!psycopg2` *your error message*. Especially useful the week
after the release of a new OS X version.
- Write to the `Mailing List`_.
- If you think that you have discovered a bug, test failure or missing feature
please raise a ticket in the `bug tracker`_.
- Complain on your blog or on Twitter that `!psycopg2` is the worst package
ever and about the quality time you have wasted figuring out the correct
:envvar:`ARCHFLAGS`. Especially useful from the Starbucks near you.
.. _mailing list: https://www.postgresql.org/list/psycopg/
.. _bug tracker: https://github.com/psycopg/psycopg2/issues

View File

@ -1,7 +0,0 @@
.. index::
single: License
License
=======
.. include:: ../../LICENSE

View File

@ -1,388 +0,0 @@
The `psycopg2` module content
==================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. module:: psycopg2
The module interface respects the standard defined in the |DBAPI|_.
.. index::
single: Connection string
double: Connection; Parameters
single: Username; Connection
single: Password; Connection
single: Host; Connection
single: Port; Connection
single: DSN (Database Source Name)
.. function::
connect(dsn=None, connection_factory=None, cursor_factory=None, async=False, \*\*kwargs)
Create a new database session and return a new `connection` object.
The connection parameters can be specified as a `libpq connection
string`__ using the *dsn* parameter::
conn = psycopg2.connect("dbname=test user=postgres password=secret")
or using a set of keyword arguments::
conn = psycopg2.connect(dbname="test", user="postgres", password="secret")
or using a mix of both: if the same parameter name is specified in both
sources, the *kwargs* value will have precedence over the *dsn* value.
Note that either the *dsn* or at least one connection-related keyword
argument is required.
The basic connection parameters are:
- `!dbname` -- the database name (`!database` is a deprecated alias)
- `!user` -- user name used to authenticate
- `!password` -- password used to authenticate
- `!host` -- database host address (defaults to UNIX socket if not provided)
- `!port` -- connection port number (defaults to 5432 if not provided)
Any other connection parameter supported by the client library/server can
be passed either in the connection string or as a keyword. The PostgreSQL
documentation contains the complete list of the `supported parameters`__.
Also note that the same parameters can be passed to the client library
using `environment variables`__.
.. __:
.. _connstring: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-CONNSTRING
.. __:
.. _connparams: https://www.postgresql.org/docs/current/static/libpq-connect.html#LIBPQ-PARAMKEYWORDS
.. __:
.. _connenvvars: https://www.postgresql.org/docs/current/static/libpq-envars.html
Using the *connection_factory* parameter a different class or
connections factory can be specified. It should be a callable object
taking a *dsn* string argument. See :ref:`subclassing-connection` for
details. If a *cursor_factory* is specified, the connection's
`~connection.cursor_factory` is set to it. If you only need customized
cursors you can use this parameter instead of subclassing a connection.
Using *async*\=\ `!True` an asynchronous connection will be created: see
:ref:`async-support` to know about advantages and limitations. *async_* is
a valid alias for the Python version where ``async`` is a keyword.
.. versionchanged:: 2.4.3
any keyword argument is passed to the connection. Previously only the
basic parameters (plus `!sslmode`) were supported as keywords.
.. versionchanged:: 2.5
added the *cursor_factory* parameter.
.. versionchanged:: 2.7
both *dsn* and keyword arguments can be specified.
.. versionchanged:: 2.7
added *async_* alias.
.. seealso::
- `~psycopg2.extensions.parse_dsn`
- libpq `connection string syntax`__
- libpq supported `connection parameters`__
- libpq supported `environment variables`__
.. __: connstring_
.. __: connparams_
.. __: connenvvars_
.. extension::
The non-connection-related keyword parameters are Psycopg extensions
to the |DBAPI|_.
.. data:: apilevel
String constant stating the supported DB API level. For `psycopg2` is
``2.0``.
.. data:: threadsafety
Integer constant stating the level of thread safety the interface
supports. For `psycopg2` is ``2``, i.e. threads can share the module
and the connection. See :ref:`thread-safety` for details.
.. data:: paramstyle
String constant stating the type of parameter marker formatting expected
by the interface. For `psycopg2` is ``pyformat``. See also
:ref:`query-parameters`.
.. data:: __libpq_version__
Integer constant reporting the version of the ``libpq`` library this
``psycopg2`` module was compiled with (in the same format of
`~psycopg2.extensions.ConnectionInfo.server_version`). If this value is
greater or equal than ``90100`` then you may query the version of the
actually loaded library using the `~psycopg2.extensions.libpq_version()`
function.
.. index::
single: Exceptions; DB API
.. _dbapi-exceptions:
Exceptions
----------
In compliance with the |DBAPI|_, the module makes informations about errors
available through the following exceptions:
.. exception:: Warning
Exception raised for important warnings like data truncations while
inserting, etc. It is a subclass of the Python `StandardError`
(`Exception` on Python 3).
.. exception:: Error
Exception that is the base class of all other error exceptions. You can
use this to catch all errors with one single `!except` statement. Warnings
are not considered errors and thus not use this class as base. It
is a subclass of the Python `StandardError` (`Exception` on Python 3).
.. attribute:: pgerror
String representing the error message returned by the backend,
`!None` if not available.
.. attribute:: pgcode
String representing the error code returned by the backend, `!None`
if not available. The `~psycopg2.errorcodes` module contains
symbolic constants representing PostgreSQL error codes.
.. doctest::
:options: +NORMALIZE_WHITESPACE
>>> try:
... cur.execute("SELECT * FROM barf")
... except psycopg2.Error as e:
... pass
>>> e.pgcode
'42P01'
>>> print(e.pgerror)
ERROR: relation "barf" does not exist
LINE 1: SELECT * FROM barf
^
.. attribute:: cursor
The cursor the exception was raised from; `None` if not applicable.
.. attribute:: diag
A `~psycopg2.extensions.Diagnostics` object containing further
information about the error. ::
>>> try:
... cur.execute("SELECT * FROM barf")
... except psycopg2.Error as e:
... pass
>>> e.diag.severity
'ERROR'
>>> e.diag.message_primary
'relation "barf" does not exist'
.. versionadded:: 2.5
.. extension::
The `~Error.pgerror`, `~Error.pgcode`, `~Error.cursor`, and
`~Error.diag` attributes are Psycopg extensions.
.. exception:: InterfaceError
Exception raised for errors that are related to the database interface
rather than the database itself. It is a subclass of `Error`.
.. exception:: DatabaseError
Exception raised for errors that are related to the database. It is a
subclass of `Error`.
.. exception:: DataError
Exception raised for errors that are due to problems with the processed
data like division by zero, numeric value out of range, etc. It is a
subclass of `DatabaseError`.
.. exception:: OperationalError
Exception raised for errors that are related to the database's operation
and not necessarily under the control of the programmer, e.g. an
unexpected disconnect occurs, the data source name is not found, a
transaction could not be processed, a memory allocation error occurred
during processing, etc. It is a subclass of `DatabaseError`.
.. exception:: IntegrityError
Exception raised when the relational integrity of the database is
affected, e.g. a foreign key check fails. It is a subclass of
`DatabaseError`.
.. exception:: InternalError
Exception raised when the database encounters an internal error, e.g. the
cursor is not valid anymore, the transaction is out of sync, etc. It is a
subclass of `DatabaseError`.
.. exception:: ProgrammingError
Exception raised for programming errors, e.g. table not found or already
exists, syntax error in the SQL statement, wrong number of parameters
specified, etc. It is a subclass of `DatabaseError`.
.. exception:: NotSupportedError
Exception raised in case a method or database API was used which is not
supported by the database, e.g. requesting a `!rollback()` on a
connection that does not support transaction or has transactions turned
off. It is a subclass of `DatabaseError`.
.. extension::
Psycopg actually raises a different exception for each :sql:`SQLSTATE`
error returned by the database: the classes are available in the
`psycopg2.errors` module. Every exception class is a subclass of one of
the exception classes defined here though, so they don't need to be
trapped specifically: trapping `!Error` or `!DatabaseError` is usually
what needed to write a generic error handler; trapping a specific error
such as `!NotNullViolation` can be useful to write specific exception
handlers.
This is the exception inheritance layout:
.. parsed-literal::
`!StandardError`
\|__ `Warning`
\|__ `Error`
\|__ `InterfaceError`
\|__ `DatabaseError`
\|__ `DataError`
\|__ `OperationalError`
\|__ `IntegrityError`
\|__ `InternalError`
\|__ `ProgrammingError`
\|__ `NotSupportedError`
.. _type-objects-and-constructors:
Type Objects and Constructors
-----------------------------
.. note::
This section is mostly copied verbatim from the |DBAPI|_
specification. While these objects are exposed in compliance to the
DB API, Psycopg offers very accurate tools to convert data between Python
and PostgreSQL formats. See :ref:`adapting-new-types` and
:ref:`type-casting-from-sql-to-python`
Many databases need to have the input in a particular format for
binding to an operation's input parameters. For example, if an
input is destined for a DATE column, then it must be bound to the
database in a particular string format. Similar problems exist
for "Row ID" columns or large binary items (e.g. blobs or RAW
columns). This presents problems for Python since the parameters
to the .execute*() method are untyped. When the database module
sees a Python string object, it doesn't know if it should be bound
as a simple CHAR column, as a raw BINARY item, or as a DATE.
To overcome this problem, a module must provide the constructors
defined below to create objects that can hold special values.
When passed to the cursor methods, the module can then detect the
proper type of the input parameter and bind it accordingly.
A Cursor Object's description attribute returns information about
each of the result columns of a query. The type_code must compare
equal to one of Type Objects defined below. Type Objects may be
equal to more than one type code (e.g. DATETIME could be equal to
the type codes for date, time and timestamp columns; see the
Implementation Hints below for details).
The module exports the following constructors and singletons:
.. function:: Date(year,month,day)
This function constructs an object holding a date value.
.. function:: Time(hour,minute,second)
This function constructs an object holding a time value.
.. function:: Timestamp(year,month,day,hour,minute,second)
This function constructs an object holding a time stamp value.
.. function:: DateFromTicks(ticks)
This function constructs an object holding a date value from the given
ticks value (number of seconds since the epoch; see the documentation of
the standard Python time module for details).
.. function:: TimeFromTicks(ticks)
This function constructs an object holding a time value from the given
ticks value (number of seconds since the epoch; see the documentation of
the standard Python time module for details).
.. function:: TimestampFromTicks(ticks)
This function constructs an object holding a time stamp value from the
given ticks value (number of seconds since the epoch; see the
documentation of the standard Python time module for details).
.. function:: Binary(string)
This function constructs an object capable of holding a binary (long)
string value.
.. note::
All the adapters returned by the module level factories (`!Binary`,
`!Date`, `!Time`, `!Timestamp` and the `!*FromTicks` variants) expose the
wrapped object (a regular Python object such as `!datetime`) in an
`!adapted` attribute.
.. data:: STRING
This type object is used to describe columns in a database that are
string-based (e.g. CHAR).
.. data:: BINARY
This type object is used to describe (long) binary columns in a database
(e.g. LONG, RAW, BLOBs).
.. data:: NUMBER
This type object is used to describe numeric columns in a database.
.. data:: DATETIME
This type object is used to describe date/time columns in a database.
.. data:: ROWID
This type object is used to describe the "Row ID" column in a database.
.. testcode::
:hide:
conn.rollback()

View File

@ -1,10 +0,0 @@
.. index::
single: Release notes
single: News
.. _news:
Release notes
=============
.. include:: ../../NEWS

View File

@ -1,60 +0,0 @@
`psycopg2.pool` -- Connections pooling
======================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. index::
pair: Connection; Pooling
.. module:: psycopg2.pool
Creating new PostgreSQL connections can be an expensive operation. This
module offers a few pure Python classes implementing simple connection pooling
directly in the client application.
.. class:: AbstractConnectionPool(minconn, maxconn, \*args, \*\*kwargs)
Base class implementing generic key-based pooling code.
New *minconn* connections are created automatically. The pool will support
a maximum of about *maxconn* connections. *\*args* and *\*\*kwargs* are
passed to the `~psycopg2.connect()` function.
The following methods are expected to be implemented by subclasses:
.. method:: getconn(key=None)
Get a free connection from the pool.
The *key* parameter is optional: if used, the connection will be
associated to the key and calling `!getconn()` with the same key again
will return the same connection.
.. method:: putconn(conn, key=None, close=False)
Put away a connection.
If *close* is `!True`, discard the connection from the pool.
*key* should be used consistently with `getconn()`.
.. method:: closeall
Close all the connections handled by the pool.
Note that all the connections are closed, including ones
eventually in use by the application.
The following classes are `AbstractConnectionPool` subclasses ready to
be used.
.. autoclass:: SimpleConnectionPool
.. note:: This pool class is useful only for single-threaded applications.
.. index:: Multithread; Connection pooling
.. autoclass:: ThreadedConnectionPool
.. note:: This pool class can be safely used in multi-threaded applications.

View File

@ -1,147 +0,0 @@
`psycopg2.sql` -- SQL string composition
========================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. module:: psycopg2.sql
.. versionadded:: 2.7
The module contains objects and functions useful to generate SQL dynamically,
in a convenient and safe way. SQL identifiers (e.g. names of tables and
fields) cannot be passed to the `~cursor.execute()` method like query
arguments::
# This will not work
table_name = 'my_table'
cur.execute("insert into %s values (%s, %s)", [table_name, 10, 20])
The SQL query should be composed before the arguments are merged, for
instance::
# This works, but it is not optimal
table_name = 'my_table'
cur.execute(
"insert into %s values (%%s, %%s)" % table_name,
[10, 20])
This sort of works, but it is an accident waiting to happen: the table name
may be an invalid SQL literal and need quoting; even more serious is the
security problem in case the table name comes from an untrusted source. The
name should be escaped using `~psycopg2.extensions.quote_ident()`::
# This works, but it is not optimal
table_name = 'my_table'
cur.execute(
"insert into %s values (%%s, %%s)" % ext.quote_ident(table_name, cur),
[10, 20])
This is now safe, but it somewhat ad-hoc. In case, for some reason, it is
necessary to include a value in the query string (as opposite as in a value)
the merging rule is still different (`~psycopg2.extensions.adapt()` should be
used...). It is also still relatively dangerous: if `!quote_ident()` is
forgotten somewhere, the program will usually work, but will eventually crash
in the presence of a table or field name with containing characters to escape,
or will present a potentially exploitable weakness.
The objects exposed by the `!psycopg2.sql` module allow generating SQL
statements on the fly, separating clearly the variable parts of the statement
from the query parameters::
from psycopg2 import sql
cur.execute(
sql.SQL("insert into {} values (%s, %s)")
.format(sql.Identifier('my_table')),
[10, 20])
Module usage
------------
Usually you should express the template of your query as an `SQL` instance
with `{}`\-style placeholders and use `~SQL.format()` to merge the variable
parts into them, all of which must be `Composable` subclasses. You can still
have `%s`\ -style placeholders in your query and pass values to
`~cursor.execute()`: such value placeholders will be untouched by
`!format()`::
query = sql.SQL("select {field} from {table} where {pkey} = %s").format(
field=sql.Identifier('my_name'),
table=sql.Identifier('some_table'),
pkey=sql.Identifier('id'))
The resulting object is meant to be passed directly to cursor methods such as
`~cursor.execute()`, `~cursor.executemany()`, `~cursor.copy_expert()`, but can
also be used to compose a query as a Python string, using the
`~Composable.as_string()` method::
cur.execute(query, (42,))
If part of your query is a variable sequence of arguments, such as a
comma-separated list of field names, you can use the `SQL.join()` method to
pass them to the query::
query = sql.SQL("select {fields} from {table}").format(
fields=sql.SQL(',').join([
sql.Identifier('field1'),
sql.Identifier('field2'),
sql.Identifier('field3'),
]),
table=sql.Identifier('some_table'))
`!sql` objects
--------------
The `!sql` objects are in the following inheritance hierarchy:
| `Composable`: the base class exposing the common interface
| ``|__`` `SQL`: a literal snippet of an SQL query
| ``|__`` `Identifier`: a PostgreSQL identifier or dot-separated sequence of identifiers
| ``|__`` `Literal`: a value hardcoded into a query
| ``|__`` `Placeholder`: a `%s`\ -style placeholder whose value will be added later e.g. by `~cursor.execute()`
| ``|__`` `Composed`: a sequence of `!Composable` instances.
.. autoclass:: Composable
.. automethod:: as_string
.. autoclass:: SQL
.. autoattribute:: string
.. automethod:: format
.. automethod:: join
.. autoclass:: Identifier
.. versionchanged:: 2.8
added support for multiple strings.
.. autoattribute:: strings
.. versionadded:: 2.8
previous verions only had a `!string` attribute. The attribute
still exists but is deprecate and will only work if the
`!Identifier` wraps a single string.
.. autoclass:: Literal
.. autoattribute:: wrapped
.. autoclass:: Placeholder
.. autoattribute:: name
.. autoclass:: Composed
.. autoattribute:: seq
.. automethod:: join

View File

@ -1,50 +0,0 @@
"""
extension
~~~~~~~~~
A directive to create a box warning that a certain bit of Psycopg is an
extension to the DBAPI 2.0.
:copyright: Copyright 2010 by Daniele Varrazzo.
"""
from docutils import nodes
from sphinx.locale import _
from docutils.parsers.rst import Directive
class extension_node(nodes.Admonition, nodes.Element): pass
class Extension(Directive):
"""
An extension entry, displayed as an admonition.
"""
has_content = True
required_arguments = 0
optional_arguments = 0
final_argument_whitespace = False
option_spec = {}
def run(self):
node = extension_node('\n'.join(self.content))
node += nodes.title(_('DB API extension'), _('DB API extension'))
self.state.nested_parse(self.content, self.content_offset, node)
node['classes'].append('dbapi-extension')
return [node]
def visit_extension_node(self, node):
self.visit_admonition(node)
def depart_extension_node(self, node):
self.depart_admonition(node)
def setup(app):
app.add_node(extension_node,
html=(visit_extension_node, depart_extension_node),
latex=(visit_extension_node, depart_extension_node),
text=(visit_extension_node, depart_extension_node))
app.add_directive('extension', Extension)

View File

@ -1,19 +0,0 @@
"""
sql role
~~~~~~~~
An interpreted text role to style SQL syntax in Psycopg documentation.
:copyright: Copyright 2010 by Daniele Varrazzo.
"""
from docutils import nodes, utils
from docutils.parsers.rst import roles
def sql_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
text = utils.unescape(text)
options['classes'] = ['sql']
return [nodes.literal(rawtext, text, **options)], []
def setup(app):
roles.register_local_role('sql', sql_role)

View File

@ -1,57 +0,0 @@
"""
ticket role
~~~~~~~~~~~
An interpreted text role to link docs to tickets issues.
:copyright: Copyright 2013 by Daniele Varrazzo.
"""
import re
from docutils import nodes, utils
from docutils.parsers.rst import roles
def ticket_role(name, rawtext, text, lineno, inliner, options={}, content=[]):
cfg = inliner.document.settings.env.app.config
if cfg.ticket_url is None:
msg = inliner.reporter.warning(
"ticket not configured: please configure ticket_url in conf.py")
prb = inliner.problematic(rawtext, rawtext, msg)
return [prb], [msg]
rv = [nodes.Text(name + ' ')]
tokens = re.findall(r'(#?\d+)|([^\d#]+)', text)
for ticket, noise in tokens:
if ticket:
num = int(ticket.replace('#', ''))
# Push numbers of the oldel tickets ahead.
# We moved the tickets from a different tracker to GitHub and the
# latter already had a few ticket numbers taken (as merge
# requests).
remap_until = cfg.ticket_remap_until
remap_offset = cfg.ticket_remap_offset
if remap_until and remap_offset:
if num <= remap_until:
num += remap_offset
url = cfg.ticket_url % num
roles.set_classes(options)
node = nodes.reference(ticket, utils.unescape(ticket),
refuri=url, **options)
rv.append(node)
else:
assert noise
rv.append(nodes.Text(noise))
return rv, []
def setup(app):
app.add_config_value('ticket_url', None, 'env')
app.add_config_value('ticket_remap_until', None, 'env')
app.add_config_value('ticket_remap_offset', None, 'env')
app.add_role('ticket', ticket_role)
app.add_role('tickets', ticket_role)

View File

@ -1,57 +0,0 @@
#!/usr/bin/env python
"""Create the docs table of the sqlstate errors.
"""
import re
import sys
from collections import namedtuple
from psycopg2._psycopg import sqlstate_errors
def main():
sqlclasses = {}
clsfile = sys.argv[1]
with open(clsfile) as f:
for l in f:
m = re.match(r'/\* Class (..) - (.+) \*/', l)
if m is not None:
sqlclasses[m.group(1)] = m.group(2)
Line = namedtuple('Line', 'colstate colexc colbase sqlstate')
lines = [Line('SQLSTATE', 'Exception', 'Base exception', None)]
for k in sorted(sqlstate_errors):
exc = sqlstate_errors[k]
lines.append(Line(
f"``{k}``", f"`!{exc.__name__}`",
f"`!{get_base_exception(exc).__name__}`", k))
widths = [max(len(l[c]) for l in lines) for c in range(3)]
h = Line(*(['=' * w for w in widths] + [None]))
lines.insert(0, h)
lines.insert(2, h)
lines.append(h)
h1 = '-' * (sum(widths) + len(widths) - 1)
sqlclass = None
for l in lines:
cls = l.sqlstate[:2] if l.sqlstate else None
if cls and cls != sqlclass:
print(f"**Class {cls}**: {sqlclasses[cls]}")
print(h1)
sqlclass = cls
print("%-*s %-*s %-*s" % (
widths[0], l.colstate, widths[1], l.colexc, widths[2], l.colbase))
def get_base_exception(exc):
for cls in exc.__mro__:
if cls.__module__ == 'psycopg2':
return cls
if __name__ == '__main__':
sys.exit(main())

View File

@ -1,19 +0,0 @@
`psycopg2.tz` -- ``tzinfo`` implementations for Psycopg 2
===============================================================
.. sectionauthor:: Daniele Varrazzo <daniele.varrazzo@gmail.com>
.. module:: psycopg2.tz
.. deprecated:: 2.9
The module will be dropped in psycopg 2.10. Use `datetime.timezone`
instead.
This module holds two different tzinfo implementations that can be used as the
`tzinfo` argument to `~datetime.datetime` constructors, directly passed to
Psycopg functions or used to set the `cursor.tzinfo_factory` attribute in
cursors.
.. autoclass:: psycopg2.tz.FixedOffsetTimezone
.. autoclass:: psycopg2.tz.LocalTimezone

File diff suppressed because it is too large Load Diff

89
examples/binary.py Normal file
View File

@ -0,0 +1,89 @@
# binary.py - working with binary data
#
# Copyright (C) 2001-2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_binary (id int4, name text, img bytea)")
except:
conn.rollback()
curs.execute("DROP TABLE test_binary")
curs.execute("CREATE TABLE test_binary (id int4, name text, img bytea)")
conn.commit()
# first we try two inserts, one with an explicit Binary call and the other
# using a buffer on a file object.
data1 = {'id':1, 'name':'somehackers.jpg',
'img':psycopg2.Binary(open('somehackers.jpg').read())}
data2 = {'id':2, 'name':'whereareyou.jpg',
'img':buffer(open('whereareyou.jpg').read())}
curs.execute("""INSERT INTO test_binary
VALUES (%(id)s, %(name)s, %(img)s)""", data1)
curs.execute("""INSERT INTO test_binary
VALUES (%(id)s, %(name)s, %(img)s)""", data2)
# now we try to extract the images as simple text strings
print "Extracting the images as strings..."
curs.execute("SELECT * FROM test_binary")
for row in curs.fetchall():
name, ext = row[1].split('.')
new_name = name + '_S.' + ext
print " writing %s to %s ..." % (name+'.'+ext, new_name),
open(new_name, 'wb').write(row[2])
print "done"
print " python type of image data is", type(row[2])
# extract exactly the same data but using a binary cursor
print "Extracting the images using a binary cursor:"
curs.execute("""DECLARE zot CURSOR FOR
SELECT img, name FROM test_binary FOR READ ONLY""")
curs.execute("""FETCH ALL FROM zot""")
for row in curs.fetchall():
name, ext = row[1].split('.')
new_name = name + '_B.' + ext
print " writing %s to %s ..." % (name+'.'+ext, new_name),
open(new_name, 'wb').write(row[0])
print "done"
print " python type of image data is", type(row[0])
# this rollback is requires because we can't drop a table with a binary cusor
# declared and still open
conn.rollback()
curs.execute("DROP TABLE test_binary")
conn.commit()
print "\nNow try to load the new images, to check it worked!"

178
examples/copy_from.py Normal file
View File

@ -0,0 +1,178 @@
# copy_from.py -- example about copy_from
#
# Copyright (C) 2002 Tom Jenkins <tjenkins@devis.com>
# Copyright (C) 2005 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import os
import StringIO
import psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_copy (fld1 text, fld2 text, fld3 int4)")
except:
conn.rollback()
curs.execute("DROP TABLE test_copy")
curs.execute("CREATE TABLE test_copy (fld1 text, fld2 text, fld3 int4)")
conn.commit()
# copy_from with default arguments, from open file
io = open('copy_from.txt', 'wr')
data = ['Tom\tJenkins\t37\n',
'Madonna\t\N\t45\n',
'Federico\tDi Gregorio\t\N\n']
io.writelines(data)
io.close()
io = open('copy_from.txt', 'r')
curs.copy_from(io, 'test_copy')
print "1) Copy %d records from file object " % len(data) + \
"using defaults (sep: \\t and null = \\N)"
io.close()
curs.execute("SELECT * FROM test_copy")
rows = curs.fetchall()
print " Select returned %d rows" % len(rows)
for r in rows:
print " %s %s\t%s" % (r[0], r[1], r[2])
curs.execute("delete from test_copy")
conn.commit()
# copy_from using custom separator, from open file
io = open('copy_from.txt', 'wr')
data = ['Tom:Jenkins:37\n',
'Madonna:\N:45\n',
'Federico:Di Gregorio:\N\n']
io.writelines(data)
io.close()
io = open('copy_from.txt', 'r')
curs.copy_from(io, 'test_copy', ':')
print "2) Copy %d records from file object using sep = :" % len(data)
io.close()
curs.execute("SELECT * FROM test_copy")
rows = curs.fetchall()
print " Select returned %d rows" % len(rows)
for r in rows:
print " %s %s\t%s" % (r[0], r[1], r[2])
curs.execute("delete from test_copy")
conn.commit()
# copy_from using custom null identifier, from open file
io = open('copy_from.txt', 'wr')
data = ['Tom\tJenkins\t37\n',
'Madonna\tNULL\t45\n',
'Federico\tDi Gregorio\tNULL\n']
io.writelines(data)
io.close()
io = open('copy_from.txt', 'r')
curs.copy_from(io, 'test_copy', null='NULL')
print "3) Copy %d records from file object using null = NULL" % len(data)
io.close()
curs.execute("SELECT * FROM test_copy")
rows = curs.fetchall()
print " Select using cursor returned %d rows" % len(rows)
for r in rows:
print " %s %s\t%s" % (r[0], r[1], r[2])
curs.execute("delete from test_copy")
conn.commit()
# copy_from using custom separator and null identifier
io = open('copy_from.txt', 'wr')
data = ['Tom:Jenkins:37\n', 'Madonna:NULL:45\n', 'Federico:Di Gregorio:NULL\n']
io.writelines(data)
io.close()
io = open('copy_from.txt', 'r')
curs.copy_from(io, 'test_copy', ':', 'NULL')
print "4) Copy %d records from file object " % len(data) + \
"using sep = : and null = NULL"
io.close()
curs.execute("SELECT * FROM test_copy")
rows = curs.fetchall()
print " Select using cursor returned %d rows" % len(rows)
for r in rows:
print " %s %s\t%s" % (r[0], r[1], r[2])
curs.execute("delete from test_copy")
conn.commit()
# anything can be used as a file if it has .read() and .readline() methods
data = StringIO.StringIO()
data.write('\n'.join(['Tom\tJenkins\t37',
'Madonna\t\N\t45',
'Federico\tDi Gregorio\t\N']))
data.seek(0)
curs.copy_from(data, 'test_copy')
print "5) Copy 3 records from StringIO object using defaults"
curs.execute("SELECT * FROM test_copy")
rows = curs.fetchall()
print " Select using cursor returned %d rows" % len(rows)
for r in rows:
print " %s %s\t%s" % (r[0], r[1], r[2])
curs.execute("delete from test_copy")
conn.commit()
# simple error test
print "6) About to raise an error"
data = StringIO.StringIO()
data.write('\n'.join(['Tom\tJenkins\t37',
'Madonna\t\N\t45',
'Federico\tDi Gregorio\taaa']))
data.seek(0)
try:
curs.copy_from(data, 'test_copy')
except StandardError, err:
conn.rollback()
print " Catched error (as expected):\n", err
conn.rollback()
curs.execute("DROP TABLE test_copy")
os.unlink('copy_from.txt')
conn.commit()

104
examples/copy_to.py Normal file
View File

@ -0,0 +1,104 @@
# copy_to.py -- example about copy_to
#
# Copyright (C) 2002 Tom Jenkins <tjenkins@devis.com>
# Copyright (C) 2005 Federico Di Gregorio <fog@initd.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import os
import StringIO
import psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_copy (fld1 text, fld2 text, fld3 int4)")
except:
conn.rollback()
curs.execute("DROP TABLE test_copy")
curs.execute("CREATE TABLE test_copy (fld1 text, fld2 text, fld3 int4)")
conn.commit()
# demostrate copy_to functionality
data = [('Tom', 'Jenkins', '37'),
('Madonna', None, '45'),
('Federico', 'Di Gregorio', None)]
query = "INSERT INTO test_copy VALUES (%s, %s, %s)"
curs.executemany(query, data)
conn.commit()
# copy_to using defaults
io = open('copy_to.txt', 'w')
curs.copy_to(io, 'test_copy')
print "1) Copy %d records into file object using defaults: " % len (data) + \
"sep = \\t and null = \\N"
io.close()
rows = open('copy_to.txt', 'r').readlines()
print " File has %d rows:" % len(rows)
for r in rows:
print " ", r,
# copy_to using custom separator
io = open('copy_to.txt', 'w')
curs.copy_to(io, 'test_copy', ':')
print "2) Copy %d records into file object using sep = :" % len(data)
io.close()
rows = open('copy_to.txt', 'r').readlines()
print " File has %d rows:" % len(rows)
for r in rows:
print " ", r,
# copy_to using custom null identifier
io = open('copy_to.txt', 'w')
curs.copy_to(io, 'test_copy', null='NULL')
print "3) Copy %d records into file object using null = NULL" % len(data)
io.close()
rows = open('copy_to.txt', 'r').readlines()
print " File has %d rows:" % len(rows)
for r in rows:
print " ", r,
# copy_to using custom separator and null identifier
io = open('copy_to.txt', 'w')
curs.copy_to(io, 'test_copy', ':', 'NULL')
print "4) Copy %d records into file object using sep = : and null ) NULL" % \
len(data)
io.close()
rows = open('copy_to.txt', 'r').readlines()
print " File has %d rows:" % len(rows)
for r in rows:
print " ", r,
curs.execute("DROP TABLE test_copy")
os.unlink('copy_to.txt')
conn.commit()

63
examples/cursor.py Normal file
View File

@ -0,0 +1,63 @@
# cursor.py - how to subclass the cursor type
#
# Copyright (C) 2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
import sys
import psycopg2
import psycopg2.extensions
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dsn:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
class NoDataError(psycopg2.ProgrammingError):
"""Exception that will be raised by our cursor."""
pass
class Cursor(psycopg2.extensions.cursor):
"""A custom cursor."""
def fetchone(self):
"""Like fetchone but raise an exception if no data is available.
Note that to have .fetchmany() and .fetchall() to raise the same
exception we'll have to override them too; even if internally psycopg
uses the same function to fetch rows, the code path from Python is
different.
"""
d = psycopg2.extensions.cursor.fetchone(self)
if d is None:
raise NoDataError("no more data")
return d
curs = conn.cursor(cursor_factory=Cursor)
curs.execute("SELECT 1 AS foo")
print "Result of fetchone():", curs.fetchone()
# now let's raise the exception
try:
curs.fetchone()
except NoDataError, err:
print "Exception caugth:", err
conn.rollback()

144
examples/dialtone.py Normal file
View File

@ -0,0 +1,144 @@
"""
This example/recipe has been contributed by Valentino Volonghi (dialtone)
Mapping arbitrary objects to a PostgreSQL database with psycopg2
- Problem
You need to store arbitrary objects in a PostgreSQL database without being
intrusive for your classes (don't want inheritance from an 'Item' or
'Persistent' object).
- Solution
"""
from datetime import datetime
import psycopg2
from psycopg2.extensions import adapt, register_adapter
try:
sorted()
except:
def sorted(seq):
seq.sort()
return seq
# Here is the adapter for every object that we may ever need to
# insert in the database. It receives the original object and does
# its job on that instance
class ObjectMapper(object):
def __init__(self, orig, curs=None):
self.orig = orig
self.tmp = {}
self.items, self.fields = self._gatherState()
def _gatherState(self):
adaptee_name = self.orig.__class__.__name__
fields = sorted([(field, getattr(self.orig, field))
for field in persistent_fields[adaptee_name]])
items = []
for item, value in fields:
items.append(item)
return items, fields
def getTableName(self):
return self.orig.__class__.__name__
def getMappedValues(self):
tmp = []
for i in self.items:
tmp.append("%%(%s)s"%i)
return ", ".join(tmp)
def getValuesDict(self):
return dict(self.fields)
def getFields(self):
return self.items
def generateInsert(self):
qry = "INSERT INTO"
qry += " " + self.getTableName() + " ("
qry += ", ".join(self.getFields()) + ") VALUES ("
qry += self.getMappedValues() + ")"
return qry, self.getValuesDict()
# Here are the objects
class Album(object):
id = 0
def __init__(self):
self.creation_time = datetime.now()
self.album_id = self.id
Album.id = Album.id + 1
self.binary_data = buffer('12312312312121')
class Order(object):
id = 0
def __init__(self):
self.items = ['rice','chocolate']
self.price = 34
self.order_id = self.id
Order.id = Order.id + 1
register_adapter(Album, ObjectMapper)
register_adapter(Order, ObjectMapper)
# Describe what is needed to save on each object
# This is actually just configuration, you can use xml with a parser if you
# like to have plenty of wasted CPU cycles ;P.
persistent_fields = {'Album': ['album_id', 'creation_time', 'binary_data'],
'Order': ['order_id', 'items', 'price']
}
print adapt(Album()).generateInsert()
print adapt(Album()).generateInsert()
print adapt(Album()).generateInsert()
print adapt(Order()).generateInsert()
print adapt(Order()).generateInsert()
print adapt(Order()).generateInsert()
"""
- Discussion
Psycopg 2 has a great new feature: adaptation. The big thing about
adaptation is that it enable the programmer to glue most of the
code out there without many difficulties.
This recipe tries to focus the attention on a way to generate SQL queries to
insert completely new objects inside a database. As you can see objects do
not know anything about the code that is handling them. We specify all the
fields that we need for each object through the persistent_fields dict.
The most important lines of this recipe are:
register_adapter(Album, ObjectMapper)
register_adapter(Order, ObjectMapper)
In these line we notify the system that when we call adapt with an Album instance
as an argument we want it to istantiate ObjectMapper passing the Album instance
as argument (self.orig in the ObjectMapper class).
The output is something like this (for each call to generateInsert):
('INSERT INTO Album (album_id, binary_data, creation_time) VALUES
(%(album_id)s, %(binary_data)s, %(creation_time)s)',
{'binary_data': <read-only buffer for 0x402de070, ...>,
'creation_time': datetime.datetime(2004, 9, 10, 20, 48, 29, 633728),
'album_id': 1}
)
This is a tuple of {SQL_QUERY, FILLING_DICT}, and all the quoting/converting
stuff (from python's datetime to postgres s and from python's buffer to
postgres' blob) is handled with the same adaptation process hunder the hood
by psycopg2.
At last, just notice that ObjectMapper is working for both Album and Order
instances without any glitches at all, and both classes could have easily been
coming from closed source libraries or C coded ones (which are not easily
modified), whereas a common pattern in todays ORMs or OODBs is to provide
a basic 'Persistent' object that already knows how to store itself in the
database.
"""

45
examples/dict.py Normal file
View File

@ -0,0 +1,45 @@
# dict.py - using DictCUrsor/DictRow
#
# Copyright (C) 2005 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
import sys
import psycopg2
import psycopg2.extras
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dsn:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor(cursor_factory=psycopg2.extras.DictCursor)
curs.execute("SELECT 1 AS foo, 'cip' AS bar, date(now()) as zot")
data = curs.fetchone()
print "Some data accessed both as tuple and dict:"
print " ", data['foo'], data['bar'], data['zot']
print " ", data[0], data[1], data[2]
# execute another query and demostrate we can still access the row
curs.execute("SELECT 2 AS foo")
print "Some more data accessed both as tuple and dict:"
print " ", data['foo'], data['bar'], data['zot']
print " ", data[0], data[1], data[2]

99
examples/dt.py Normal file
View File

@ -0,0 +1,99 @@
# datetime.py - example of using date and time types
#
# Copyright (C) 2001-2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import psycopg2
import mx.DateTime
import datetime
from psycopg2.extensions import adapt
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
curs = conn.cursor()
try:
curs.execute("""CREATE TABLE test_dt (
k int4, d date, t time, dt timestamp, z interval)""")
except:
conn.rollback()
curs.execute("DROP TABLE test_dt")
curs.execute("""CREATE TABLE test_dt (
k int4, d date, t time, dt timestamp, z interval)""")
conn.commit()
# build and insert some data using mx.DateTime
mx1 = (
1,
mx.DateTime.Date(2004, 10, 19),
mx.DateTime.Time(0, 11, 17.015),
mx.DateTime.Timestamp(2004, 10, 19, 0, 11, 17.5),
mx.DateTime.DateTimeDelta(13, 15, 17, 59.9))
from psycopg2.extensions import adapt
import psycopg2.extras
print adapt(mx1)
print "Inserting mx.DateTime values..."
curs.execute("INSERT INTO test_dt VALUES (%s, %s, %s, %s, %s)", mx1)
# build and insert some values using the datetime adapters
dt1 = (
2,
datetime.date(2004, 10, 19),
datetime.time(0, 11, 17, 15000),
datetime.datetime(2004, 10, 19, 0, 11, 17, 500000),
datetime.timedelta(13, 15*3600+17*60+59, 900000))
print "Inserting Python datetime values..."
curs.execute("INSERT INTO test_dt VALUES (%s, %s, %s, %s, %s)", dt1)
# now extract the row from database and print them
print "Extracting values inserted with mx.DateTime wrappers:"
curs.execute("SELECT d, t, dt, z FROM test_dt WHERE k = 1")
for n, x in zip(mx1[1:], curs.fetchone()):
try:
# this will work only is psycopg has been compiled with datetime
# as the default typecaster for date/time values
s = repr(n) + "\n -> " + str(adapt(n)) + \
"\n -> " + repr(x) + "\n -> " + x.isoformat()
except:
s = repr(n) + "\n -> " + str(adapt(n)) + \
"\n -> " + repr(x) + "\n -> " + str(x)
print s
print
print "Extracting values inserted with Python datetime wrappers:"
curs.execute("SELECT d, t, dt, z FROM test_dt WHERE k = 2")
for n, x in zip(dt1[1:], curs.fetchone()):
try:
# this will work only is psycopg has been compiled with datetime
# as the default typecaster for date/time values
s = repr(n) + "\n -> " + repr(x) + "\n -> " + x.isoformat()
except:
s = repr(n) + "\n -> " + repr(x) + "\n -> " + str(x)
print s
print
curs.execute("DROP TABLE test_dt")
conn.commit()

105
examples/encoding.py Normal file
View File

@ -0,0 +1,105 @@
# enkoding.py - show to change client enkoding (and test it works)
# -*- encoding: utf8 -*-
#
# Copyright (C) 2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
import sys
import psycopg2
import psycopg2.extensions
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Initial encoding for this connection is", conn.encoding
print "\n** This example is supposed to be run in a UNICODE terminal! **\n"
print "Available encodings:"
encs = psycopg2.extensions.encodings.items()
encs.sort()
for a, b in encs:
print " ", a, "<->", b
print "Using STRING typecaster"
print "Setting backend encoding to LATIN1 and executing queries:"
conn.set_client_encoding('LATIN1')
curs = conn.cursor()
curs.execute("SELECT %s::TEXT AS foo", ('àèìòù',))
x = curs.fetchone()[0]
print " ->", unicode(x, 'latin-1').encode('utf-8'), type(x)
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", unicode(x, 'latin-1').encode('utf-8'), type(x)
print "Setting backend encoding to UTF8 and executing queries:"
conn.set_client_encoding('UNICODE')
curs = conn.cursor()
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù'.encode('utf-8'),))
x = curs.fetchone()[0]
print " ->", x, type(x)
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", x, type(x)
print "Using UNICODE typecaster"
psycopg2.extensions.register_type(psycopg2.extensions.UNICODE)
print "Setting backend encoding to LATIN1 and executing queries:"
conn.set_client_encoding('LATIN1')
curs = conn.cursor()
curs.execute("SELECT %s::TEXT AS foo", ('àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
print "Setting backend encoding to UTF8 and executing queries:"
conn.set_client_encoding('UNICODE')
curs = conn.cursor()
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù'.encode('utf-8'),))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
curs.execute("SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
print "Executing full UNICODE queries"
print "Setting backend encoding to LATIN1 and executing queries:"
conn.set_client_encoding('LATIN1')
curs = conn.cursor()
curs.execute(u"SELECT %s::TEXT AS foo", ('àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
curs.execute(u"SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
print "Setting backend encoding to UTF8 and executing queries:"
conn.set_client_encoding('UNICODE')
curs = conn.cursor()
curs.execute(u"SELECT %s::TEXT AS foo", (u'àèìòù'.encode('utf-8'),))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)
curs.execute(u"SELECT %s::TEXT AS foo", (u'àèìòù',))
x = curs.fetchone()[0]
print " ->", x.encode('utf-8'), ":", type(x)

81
examples/fetch.py Normal file
View File

@ -0,0 +1,81 @@
# fetch.py -- example about declaring cursors
#
# Copyright (C) 2001-2005 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
#
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_fetch (val int4)")
except:
conn.rollback()
curs.execute("DROP TABLE test_fetch")
curs.execute("CREATE TABLE test_fetch (val int4)")
conn.commit()
# we use this function to format the output
def flatten(l):
"""Flattens list of tuples l."""
return map(lambda x: x[0], l)
# insert 20 rows in the table
for i in range(20):
curs.execute("INSERT INTO test_fetch VALUES(%s)", (i,))
conn.commit()
# does some nice tricks with the transaction and postgres cursors
# (remember to always commit or rollback before a DECLARE)
#
# we don't need to DECLARE ourselves, psycopg now support named
# cursors (but we leave the code here, comments, as an example of
# what psycopg is doing under the hood)
#
#curs.execute("DECLARE crs CURSOR FOR SELECT * FROM test_fetch")
#curs.execute("FETCH 10 FROM crs")
#print "First 10 rows:", flatten(curs.fetchall())
#curs.execute("MOVE -5 FROM crs")
#print "Moved back cursor by 5 rows (to row 5.)"
#curs.execute("FETCH 10 FROM crs")
#print "Another 10 rows:", flatten(curs.fetchall())
#curs.execute("FETCH 10 FROM crs")
#print "The remaining rows:", flatten(curs.fetchall())
ncurs = conn.cursor("crs")
ncurs.execute("SELECT * FROM test_fetch")
print "First 10 rows:", flatten(ncurs.fetchmany(10))
ncurs.scroll(-5)
print "Moved back cursor by 5 rows (to row 5.)"
print "Another 10 rows:", flatten(ncurs.fetchmany(10))
print "Another one:", list(ncurs.fetchone())
print "The remaining rows:", flatten(ncurs.fetchall())
conn.rollback()
curs.execute("DROP TABLE test_fetch")
conn.commit()

59
examples/lastrowid.py Normal file
View File

@ -0,0 +1,59 @@
# lastrowid.py - example of using .lastrowid attribute
#
# Copyright (C) 2001-2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys, psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_oid (name text, surname text)")
except:
conn.rollback()
curs.execute("DROP TABLE test_oid")
curs.execute("CREATE TABLE test_oid (name text, surname text)")
conn.commit()
data = ({'name':'Federico', 'surname':'Di Gregorio'},
{'name':'Pierluigi', 'surname':'Di Nunzio'})
curs.execute("""INSERT INTO test_oid
VALUES (%(name)s, %(surname)s)""", data[0])
foid = curs.lastrowid
print "Oid for %(name)s %(surname)s" % data[0], "is", foid
curs.execute("""INSERT INTO test_oid
VALUES (%(name)s, %(surname)s)""", data[1])
moid = curs.lastrowid
print "Oid for %(name)s %(surname)s" % data[1], "is", moid
curs.execute("SELECT * FROM test_oid WHERE oid = %s", (foid,))
print "Oid", foid, "selected %s %s" % curs.fetchone()
curs.execute("SELECT * FROM test_oid WHERE oid = %s", (moid,))
print "Oid", moid, "selected %s %s" % curs.fetchone()
curs.execute("DROP TABLE test_oid")
conn.commit()

47
examples/mogrify.py Normal file
View File

@ -0,0 +1,47 @@
# mogrify.py - test all possible simple type mogrifications
# -*- encoding: latin1 -*-
#
# Copyright (C) 2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
import sys, psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
curs.execute("SELECT %(foo)s AS foo", {'foo':'bar'})
curs.execute("SELECT %(foo)s AS foo", {'foo':None})
curs.execute("SELECT %(foo)s AS foo", {'foo':True})
curs.execute("SELECT %(foo)s AS foo", {'foo':42})
curs.execute("SELECT %(foo)s AS foo", {'foo':u'yatt<EFBFBD>!'})
curs.execute("SELECT %(foo)s AS foo", {'foo':u'bar'})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':'bar'})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':None})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':True})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':42})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':u'yatt<EFBFBD>!'})
print curs.mogrify("SELECT %(foo)s AS foo", {'foo':u'bar'})
conn.rollback()

122
examples/myfirstrecipe.py Normal file
View File

@ -0,0 +1,122 @@
"""
Using a tuple as a bound variable in "SELECT ... IN (...)" clauses
in PostgreSQL using psycopg 2
Some time ago someone asked on the psycopg mailing list how to have a
bound variable expand to the right SQL for an SELECT IN clause:
SELECT * FROM atable WHERE afield IN (value1, value2, value3)
with the values to be used in the IN clause to be passed to the cursor
.execute() method in a tuple as a bound variable, i.e.:
in_values = ("value1", "value2", "value3")
curs.execute("SELECT ... IN %s", (in_values,))
psycopg 1 does support typecasting from Python to PostgreSQL (and back)
only for simple types and this problem has no elegant solution (short or
writing a wrapper class returning the pre-quoted text in an __str__
method.
But psycopg 2 offers a simple and elegant solution by partially
implementing the Object Adaptation from PEP 246. psycopg 2 (still in
beta and currently labeled as 1.99.9) moves the type-casting logic into
external adapters and a somehow broken adapt() function.
While the original adapt() takes 3 arguments, psycopg's one only takes
1: the bound variable to be adapted. The result is an object supporting
a not-yet well defined protocol that we can call IPsycopgSQLQuote:
class IPsycopgSQLQuote:
def getquoted(self):
"Returns a quoted string representing the bound variable."
def getbinary(self):
"Returns a binary quoted string representing the bound variable."
def getbuffer(self):
"Returns the wrapped object itself."
__str__ = getquoted
Then one of the functions (usually .getquoted()) is called by psycopg at
the right time to obtain the right, sql-quoted representation for the
corresponding bound variable.
The nice part is that the default, built-in adapters, derived from
psycopg 1 tyecasting code can be overridden by the programmer, simply
replacing them in the psycopg.extensions.adapters dictionary.
Then the solution to the original problem is now obvious: write an
adapter that adapts tuple objects into the right SQL string, by calling
recursively adapt() on each element.
Note: psycopg 2 adapter code is still very young and will probably move
to a more 'standard' (3 arguments) implementation for the adapt()
function; as long as that does not slow down too much query execution.
Psycopg 2 development can be tracked on the psycopg mailing list:
http://lists.initd.org/mailman/listinfo/psycopg
and on the psycopg 2 wiki:
http://wiki.initd.org/Projects/Psycopg2
"""
import psycopg2
import psycopg2.extensions
from psycopg2.extensions import adapt as psycoadapt
from psycopg2.extensions import register_adapter
class AsIs(object):
"""An adapter that just return the object 'as is'.
psycopg 1.99.9 has some optimizations that make impossible to call
adapt() without adding some basic adapters externally. This limitation
will be lifted in a future release.
"""
def __init__(self, obj):
self.__obj = obj
def getquoted(self):
return self.__obj
class SQL_IN(object):
"""Adapt a tuple to an SQL quotable object."""
def __init__(self, seq):
self._seq = seq
def prepare(self, conn):
pass
def getquoted(self):
# this is the important line: note how every object in the
# list is adapted and then how getquoted() is called on it
qobjs = [str(psycoadapt(o).getquoted()) for o in self._seq]
return '(' + ', '.join(qobjs) + ')'
__str__ = getquoted
# add our new adapter class to psycopg list of adapters
register_adapter(tuple, SQL_IN)
register_adapter(float, AsIs)
register_adapter(int, AsIs)
# usually we would call:
#
# conn = psycopg.connect("...")
# curs = conn.cursor()
# curs.execute("SELECT ...", (("this", "is", "the", "tuple"),))
#
# but we have no connection to a database right now, so we just check
# the SQL_IN class by calling psycopg's adapt() directly:
if __name__ == '__main__':
print "Note how the string will be SQL-quoted, but the number will not:"
print psycoadapt(("this is an 'sql quoted' str\\ing", 1, 2.0))

43
examples/notify.py Normal file
View File

@ -0,0 +1,43 @@
# notify.py - example of getting notifies
#
# Copyright (C) 2001-2005 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import psycopg2
import select
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
conn.set_isolation_level(0)
curs = conn.cursor()
curs.execute("listen test")
print "Waiting for 'NOTIFY test'"
while 1:
if select.select([curs],[],[],5)==([],[],[]):
print "Timeout"
else:
if curs.isready():
print "Got NOTIFY: %s" % str(curs.connection.notifies.pop())

53
examples/simple.py Normal file
View File

@ -0,0 +1,53 @@
# simple.py - very simple example of plain DBAPI-2.0 usage
# currently used as test-me-stress-me script for psycopg 2.0
#
# Copyright (C) 2001-2003 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
class SimpleQuoter(object):
def sqlquote(x=None):
return "'bar'"
import sys
import psycopg2
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Encoding for this connection is", conn.encoding
curs = conn.cursor()
curs.execute("SELECT 1 AS foo")
print curs.fetchone()
curs.execute("SELECT 1 AS foo")
print curs.fetchmany()
curs.execute("SELECT 1 AS foo")
print curs.fetchall()
conn.rollback()
sys.exit(0)
curs.execute("SELECT 1 AS foo", async=1)
curs.execute("SELECT %(foo)s AS foo", {'foo':'bar'})
curs.execute("SELECT %(foo)s AS foo", {'foo':None})
curs.execute("SELECT %(foo)f AS foo", {'foo':42})
curs.execute("SELECT %(foo)s AS foo", {'foo':SimpleQuoter()})

BIN
examples/somehackers.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 22 KiB

160
examples/threads.py Normal file
View File

@ -0,0 +1,160 @@
# threads.py -- example of multiple threads using psycopg
# -*- encoding: latin1 -*-
#
# Copyright (C) 2001-2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## some others parameters
INSERT_THREADS = ('A', 'B', 'C')
SELECT_THREADS = ('1', '2')
ROWS = 1000
COMMIT_STEP = 20
SELECT_SIZE = 10000
SELECT_STEP = 500
SELECT_DIV = 250
# the available modes are:
# 0 - one connection for all insert and one for all select threads
# 1 - connections generated using the connection pool
MODE = 1
## don't modify anything below tis line (except for experimenting)
import sys, psycopg2, threading
from psycopg2.pool import ThreadedConnectionPool
if len(sys.argv) > 1:
DSN = sys.argv[1]
if len(sys.argv) > 2:
MODE = int(sys.argv[2])
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
curs = conn.cursor()
try:
curs.execute("""CREATE TABLE test_threads (
name text, value1 int4, value2 float)""")
except:
conn.rollback()
curs.execute("DROP TABLE test_threads")
curs.execute("""CREATE TABLE test_threads (
name text, value1 int4, value2 float)""")
conn.commit()
## this function inserts a big number of rows and creates and destroys
## a large number of cursors
def insert_func(conn_or_pool, rows):
name = threading.currentThread().getName()
if MODE == 0:
conn = conn_or_pool
else:
conn = conn_or_pool.getconn()
for i in range(rows):
if divmod(i, COMMIT_STEP)[1] == 0:
conn.commit()
if MODE == 1:
conn_or_pool.putconn(conn)
s = name + ": COMMIT STEP " + str(i)
print s
if MODE == 1:
conn = conn_or_pool.getconn()
c = conn.cursor()
try:
c.execute("INSERT INTO test_threads VALUES (%s, %s, %s)",
(str(i), i, float(i)))
except psycopg2.ProgrammingError, err:
print name, ": an error occurred; skipping this insert"
print err
conn.commit()
## a nice select function that prints the current number of rows in the
## database (and transefer them, putting some pressure on the network)
def select_func(conn_or_pool, z):
name = threading.currentThread().getName()
if MODE == 0:
conn = conn_or_pool
conn.set_isolation_level(0)
for i in range(SELECT_SIZE):
if divmod(i, SELECT_STEP)[1] == 0:
try:
if MODE == 1:
conn = conn_or_pool.getconn()
conn.set_isolation_level(0)
c = conn.cursor()
c.execute("SELECT * FROM test_threads WHERE value2 < %s",
(int(i/z),))
l = c.fetchall()
if MODE == 1:
conn_or_pool.putconn(conn)
s = name + ": number of rows fetched: " + str(len(l))
print s
except psycopg2.ProgrammingError, err:
print name, ": an error occurred; skipping this select"
print err
## create the connection pool or the connections
if MODE == 0:
conn_insert = psycopg2.connect(DSN)
conn_select = psycopg2.connect(DSN)
else:
m = len(INSERT_THREADS) + len(SELECT_THREADS)
n = m/2
conn_insert = conn_select = ThreadedConnectionPool(n, m, DSN)
## create the threads
threads = []
print "Creating INSERT threads:"
for name in INSERT_THREADS:
t = threading.Thread(None, insert_func, 'Thread-'+name,
(conn_insert, ROWS))
t.setDaemon(0)
threads.append(t)
print "Creating SELECT threads:"
for name in SELECT_THREADS:
t = threading.Thread(None, select_func, 'Thread-'+name,
(conn_select, SELECT_DIV))
t.setDaemon(0)
threads.append(t)
## really start the threads now
for t in threads:
t.start()
# and wait for them to finish
for t in threads:
t.join()
print t.getName(), "exited OK"
conn.commit()
curs.execute("SELECT count(name) FROM test_threads")
print "Inserted", curs.fetchone()[0], "rows."
curs.execute("DROP TABLE test_threads")
conn.commit()

69
examples/tz.py Normal file
View File

@ -0,0 +1,69 @@
# tz.py - example of datetime objects with time zones
# -*- encoding: utf8 -*-
#
# Copyright (C) 2004 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below this line (except for experimenting)
import sys
import psycopg2
import datetime
from psycopg2.tz import ZERO, LOCAL, FixedOffsetTimezone
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_tz (t timestamp with time zone)")
except:
conn.rollback()
curs.execute("DROP TABLE test_tz")
curs.execute("CREATE TABLE test_tz (t timestamp with time zone)")
conn.commit()
d = datetime.datetime(1971, 10, 19, 22, 30, 0, tzinfo=LOCAL)
curs.execute("INSERT INTO test_tz VALUES (%s)", (d,))
print "Inserted timestamp with timezone:", d
print "Time zone:", d.tzinfo.tzname(d), "offset:", d.tzinfo.utcoffset(d)
tz = FixedOffsetTimezone(-5*60, "EST")
d = datetime.datetime(1971, 10, 19, 22, 30, 0, tzinfo=tz)
curs.execute("INSERT INTO test_tz VALUES (%s)", (d,))
print "Inserted timestamp with timezone:", d
print "Time zone:", d.tzinfo.tzname(d), "offset:", d.tzinfo.utcoffset(d)
curs.execute("SELECT * FROM test_tz")
d = curs.fetchone()[0]
curs.execute("INSERT INTO test_tz VALUES (%s)", (d,))
print "Inserted SELECTed timestamp:", d
print "Time zone:", d.tzinfo.tzname(d), "offset:", d.tzinfo.utcoffset(d)
curs.execute("SELECT * FROM test_tz")
for d in curs:
u = d[0].utcoffset() or ZERO
print "UTC time: ", d[0] - u
print "Local time:", d[0]
print "Time zone:", d[0].tzinfo.tzname(d[0]), d[0].tzinfo.utcoffset(d[0])
curs.execute("DROP TABLE test_tz")
conn.commit()

126
examples/usercast.py Normal file
View File

@ -0,0 +1,126 @@
# usercast.py -- example of user defined typecasters
# -*- encoding: latin-1 -*-
#
# Copyright (C) 2001-2005 Federico Di Gregorio <fog@debian.org>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by the
# Free Software Foundation; either version 2, or (at your option) any later
# version.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTIBILITY
# or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License
# for more details.
## put in DSN your DSN string
DSN = 'dbname=test'
## don't modify anything below tis line (except for experimenting)
import sys
import psycopg2
import psycopg2.extensions
import whrandom
# importing psycopg.extras will give us a nice tuple adapter: this is wrong
# because the adapter is meant to be used in SQL IN clauses while we use
# tuples to represent points but it works and the example is about Rect, not
# "Point"
import psycopg2.extras
if len(sys.argv) > 1:
DSN = sys.argv[1]
print "Opening connection using dns:", DSN
conn = psycopg2.connect(DSN)
print "Initial encoding for this connection is", conn.encoding
curs = conn.cursor()
try:
curs.execute("CREATE TABLE test_cast (p1 point, p2 point, b box)")
except:
conn.rollback()
curs.execute("DROP TABLE test_cast")
curs.execute("CREATE TABLE test_cast (p1 point, p2 point, b box)")
conn.commit()
# this is the callable object we use as a typecast (the typecast is
# usually a function, but we use a class, just to demonstrate the
# flexibility of the psycopg casting system
class Rect(object):
"""Very simple rectangle.
Note that we use this type as a data holder, as an adapter of itself for
the ISQLQuote protocol used by psycopg's adapt() (see __confrom__ below)
and eventually as a type-caster for the data extracted from the database
(that's why __init__ takes the curs argument.)
"""
def __init__(self, s=None, curs=None):
"""Init the rectangle from the optional string s."""
self.x = self.y = self.width = self.height = 0.0
if s: self.from_string(s)
def __conform__(self, proto):
"""This is a terrible hack, just ignore proto and return self."""
if proto == psycopg2.extensions.ISQLQuote:
return self
def from_points(self, x0, y0, x1, y1):
"""Init the rectangle from points."""
if x0 > x1: (x0, x1) = (x1, x0)
if y0 > y1: (y0, y1) = (y1, y0)
self.x = x0
self.y = y0
self.width = x1 - x0
self.height = y1 - y0
def from_string(self, s):
"""Init the rectangle from a string."""
seq = eval(s)
self.from_points(seq[0][0], seq[0][1], seq[1][0], seq[1][1])
def getquoted(self):
"""Format self as a string usable by the db to represent a box."""
s = "'((%d,%d),(%d,%d))'" % (
self.x, self.y, self.x + self.width, self.y + self.height)
return s
def show(self):
"""Format a description of the box."""
s = "X: %d\tY: %d\tWidth: %d\tHeight: %d" % (
self.x, self.y, self.width, self.height)
return s
# here we select from the empty table, just to grab the description
curs.execute("SELECT b FROM test_cast WHERE 0=1")
boxoid = curs.description[0][1]
print "Oid for the box datatype is", boxoid
# and build the user cast object
BOX = psycopg2.extensions.new_type((boxoid,), "BOX", Rect)
psycopg2.extensions.register_type(BOX)
# now insert 100 random data (2 points and a box in each row)
for i in range(100):
p1 = (whrandom.randint(0,100), whrandom.randint(0,100))
p2 = (whrandom.randint(0,100), whrandom.randint(0,100))
b = Rect()
b.from_points(whrandom.randint(0,100), whrandom.randint(0,100),
whrandom.randint(0,100), whrandom.randint(0,100))
curs.execute("INSERT INTO test_cast VALUES ('%(p1)s', '%(p2)s', %(box)s)",
{'box':b, 'p1':p1, 'p2':p2})
print "Added 100 boxed to the database"
# select and print all boxes with at least one point inside
curs.execute("SELECT b FROM test_cast WHERE p1 @ b OR p2 @ b")
boxes = curs.fetchall()
print "Found %d boxes with at least a point inside:" % len(boxes)
for box in boxes:
print " ", box[0].show()
curs.execute("DROP TABLE test_cast")
conn.commit()

BIN
examples/whereareyou.jpg Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 34 KiB

Some files were not shown because too many files have changed in this diff Show More