Docs expansions about thread/processes safety.

This commit is contained in:
Daniele Varrazzo 2010-11-05 23:58:10 +00:00
parent 4125b3fce0
commit 5b65e75122
3 changed files with 44 additions and 18 deletions

View File

@ -404,17 +404,21 @@ calling green thread and giving other green threads the possibility to be
scheduled), allowing non modified code and third party libraries (such as
SQLAlchemy_) to be used in coroutine-based programs.
Notice that, while I/O correctly yields control to other coroutines, each
connection has a lock allowing a single cursor at a time to communicate with the
backend: such lock is not *green*, so blocking against it would block the
entire program waiting for data, not the single coroutine. Therefore,
programmers are advised to either avoid sharing connections between coroutines
or to use a library-friendly lock to synchronize shared connections, e.g. for
pooling.
.. warning::
Psycopg connections are not *green thread safe* and can't be used
concurrently by different green threads. Each connection has a lock
used to serialize requests from different cursors to the backend process.
The lock is held for the duration of the command: if the control switched
to a different thread and the latter tried to access the same connection,
the result would be a deadlock.
Therefore, programmers are advised to either avoid sharing connections
between coroutines or to use a library-friendly lock to synchronize shared
connections, e.g. for pooling.
Coroutine libraries authors should provide a callback implementation (and
probably register it) to make Psycopg as green as they want. An example
callback (using `!select()` to block) is provided as
possibly a method to register it) to make Psycopg as green as they want. An
example callback (using `!select()` to block) is provided as
`psycopg2.extras.wait_select()`: it boils down to something similar to::
def wait_select(conn):

View File

@ -35,10 +35,11 @@ I receive the error *current transaction is aborted, commands ignored until end
.. |SAVEPOINT| replace:: :sql:`SAVEPOINT`
.. _SAVEPOINT: http://www.postgresql.org/docs/9.0/static/sql-savepoint.html
Why do i get the error *current transaction is aborted, commands ignored until end of transaction block* when I use `!multiprocessing` (or any other forking system) and not when use `!threading`?
Why do I get the error *current transaction is aborted, commands ignored until end of transaction block* when I use `!multiprocessing` (or any other forking system) and not when use `!threading`?
Psycopg's connections can't be shared across processes (but are thread
safe). If you are forking the Python process ensure to create a new
connection in each forked child.
connection in each forked child. See :ref:`thread-safety` for further
informations.
Problems with type conversions

View File

@ -477,17 +477,38 @@ method and to read the data using `~cursor.fetchone()` and
.. index:: Thread safety, Multithread
.. index:: Thread safety, Multithread, Multiprocess
.. _thread-safety:
Thread safety
-------------
Thread and process safety
-------------------------
The Psycopg module is *thread-safe*: threads can access the same database
using separate sessions (by creating a `connection` per thread) or using
the same session (accessing to the same connection and creating separate
`cursor`\ s). In |DBAPI|_ parlance, Psycopg is *level 2 thread safe*.
The Psycopg module and the `connection` objects are *thread-safe*: many
threads can access the same database either using separate sessions and
creating a `!connection` per thread or using the same using the same
connection and creating separate `cursor`\ s. In |DBAPI|_ parlance, Psycopg is
*level 2 thread safe*.
The difference between the above two approaches is that, using different
connections, the commands will be executed in different sessions and will be
served by different server processes. On the other hand, using many cursors on
the same connection, all the commands will be executed in the same session
(and in the same transaction if the connection is not in :ref:`autocommit
<transactions-control>` mode), but they will be serialized.
The above observations are only valid for regular threads: they don't apply to
forked processes nor to green threads. `libpq` connections `shouldn't be used by a
forked processes`__, so when using a module such as |multiprocessing|__ or a
forking web deploy method such as FastCGI ensure to create the connections
*after* the fork.
.. __: http://www.postgresql.org/docs/9.0/static/libpq-connect.html#LIBPQ-CONNECT
.. |multiprocessing| replace:: `!multiprocessing`
.. __: http://docs.python.org/library/multiprocessing.html
Connections shouldn't be shared either by different green threads: doing so
may result in a deadlock. See :ref:`green-support` for further details.