Docs expansions about thread/processes safety.

This commit is contained in:
Daniele Varrazzo 2010-11-05 23:58:10 +00:00
parent 4125b3fce0
commit 5b65e75122
3 changed files with 44 additions and 18 deletions

View File

@ -404,17 +404,21 @@ calling green thread and giving other green threads the possibility to be
scheduled), allowing non modified code and third party libraries (such as scheduled), allowing non modified code and third party libraries (such as
SQLAlchemy_) to be used in coroutine-based programs. SQLAlchemy_) to be used in coroutine-based programs.
Notice that, while I/O correctly yields control to other coroutines, each .. warning::
connection has a lock allowing a single cursor at a time to communicate with the Psycopg connections are not *green thread safe* and can't be used
backend: such lock is not *green*, so blocking against it would block the concurrently by different green threads. Each connection has a lock
entire program waiting for data, not the single coroutine. Therefore, used to serialize requests from different cursors to the backend process.
programmers are advised to either avoid sharing connections between coroutines The lock is held for the duration of the command: if the control switched
or to use a library-friendly lock to synchronize shared connections, e.g. for to a different thread and the latter tried to access the same connection,
pooling. the result would be a deadlock.
Therefore, programmers are advised to either avoid sharing connections
between coroutines or to use a library-friendly lock to synchronize shared
connections, e.g. for pooling.
Coroutine libraries authors should provide a callback implementation (and Coroutine libraries authors should provide a callback implementation (and
probably register it) to make Psycopg as green as they want. An example possibly a method to register it) to make Psycopg as green as they want. An
callback (using `!select()` to block) is provided as example callback (using `!select()` to block) is provided as
`psycopg2.extras.wait_select()`: it boils down to something similar to:: `psycopg2.extras.wait_select()`: it boils down to something similar to::
def wait_select(conn): def wait_select(conn):

View File

@ -35,10 +35,11 @@ I receive the error *current transaction is aborted, commands ignored until end
.. |SAVEPOINT| replace:: :sql:`SAVEPOINT` .. |SAVEPOINT| replace:: :sql:`SAVEPOINT`
.. _SAVEPOINT: http://www.postgresql.org/docs/9.0/static/sql-savepoint.html .. _SAVEPOINT: http://www.postgresql.org/docs/9.0/static/sql-savepoint.html
Why do i get the error *current transaction is aborted, commands ignored until end of transaction block* when I use `!multiprocessing` (or any other forking system) and not when use `!threading`? Why do I get the error *current transaction is aborted, commands ignored until end of transaction block* when I use `!multiprocessing` (or any other forking system) and not when use `!threading`?
Psycopg's connections can't be shared across processes (but are thread Psycopg's connections can't be shared across processes (but are thread
safe). If you are forking the Python process ensure to create a new safe). If you are forking the Python process ensure to create a new
connection in each forked child. connection in each forked child. See :ref:`thread-safety` for further
informations.
Problems with type conversions Problems with type conversions

View File

@ -477,17 +477,38 @@ method and to read the data using `~cursor.fetchone()` and
.. index:: Thread safety, Multithread .. index:: Thread safety, Multithread, Multiprocess
.. _thread-safety: .. _thread-safety:
Thread safety Thread and process safety
------------- -------------------------
The Psycopg module is *thread-safe*: threads can access the same database The Psycopg module and the `connection` objects are *thread-safe*: many
using separate sessions (by creating a `connection` per thread) or using threads can access the same database either using separate sessions and
the same session (accessing to the same connection and creating separate creating a `!connection` per thread or using the same using the same
`cursor`\ s). In |DBAPI|_ parlance, Psycopg is *level 2 thread safe*. connection and creating separate `cursor`\ s. In |DBAPI|_ parlance, Psycopg is
*level 2 thread safe*.
The difference between the above two approaches is that, using different
connections, the commands will be executed in different sessions and will be
served by different server processes. On the other hand, using many cursors on
the same connection, all the commands will be executed in the same session
(and in the same transaction if the connection is not in :ref:`autocommit
<transactions-control>` mode), but they will be serialized.
The above observations are only valid for regular threads: they don't apply to
forked processes nor to green threads. `libpq` connections `shouldn't be used by a
forked processes`__, so when using a module such as |multiprocessing|__ or a
forking web deploy method such as FastCGI ensure to create the connections
*after* the fork.
.. __: http://www.postgresql.org/docs/9.0/static/libpq-connect.html#LIBPQ-CONNECT
.. |multiprocessing| replace:: `!multiprocessing`
.. __: http://docs.python.org/library/multiprocessing.html
Connections shouldn't be shared either by different green threads: doing so
may result in a deadlock. See :ref:`green-support` for further details.