Merge branch 'scrollable' into devel

NEWS

@@ -1,6 +1,10 @@
 What's new in psycopg 2.4.6
 ---------------------------
 
+  - Added support for backward scrollable cursors. Thanks to Jon Nelson
+    for the initial patch (ticket #108).
+  - connection.reset() implemented using DISCARD ALL on server versions
+    supporting it.
   - Fixed 'cursor()' arguments propagation in connection subclasses
     and overriding of the 'cursor_factory' argument.  Thanks to
     Corry Haines for the report and the initial patch (ticket #105).
@@ -9,8 +13,6 @@ What's new in psycopg 2.4.6
     Thanks to Manu Cupcic for the report (ticket #110).
   - 'register_hstore()', 'register_composite()', 'tpc_recover()' work with
     RealDictConnection and Cursor (ticket #114).
-  - connection.reset() implemented using DISCARD ALL on server versions
-    supporting it.
 
 
 What's new in psycopg 2.4.5

doc/src/connection.rst

@@ -21,16 +21,17 @@ The ``connection`` class
     Connections are thread safe and can be shared among many threads. See
     :ref:`thread-safety` for details.
 
-    .. method:: cursor([name] [, cursor_factory] [, withhold])
+    .. method:: cursor(name=None, cursor_factory=None, scrollable=None, withhold=False)
           
         Return a new `cursor` object using the connection.
 
         If *name* is specified, the returned cursor will be a :ref:`server
         side cursor <server-side-cursors>` (also known as *named cursor*).
-        Otherwise it will be a regular *client side* cursor. By default a
+        Otherwise it will be a regular *client side* cursor.  By default a
-        :sql:`WITHOUT HOLD` cursor is created; to create a :sql:`WITH HOLD`
+        named cursor is declared without :sql:`SCROLL` option and
-        cursor, pass a `!True` value as the *withhold* parameter.  See
+        :sql:`WITHOUT HOLD`: set the argument or property `~cursor.scrollable`
-        :ref:`server-side-cursors`.
+        to `!True`/`!False` and or `~cursor.withhold` to `!True` to change the
+        declaration.
 
         The name can be a string not valid as a PostgreSQL identifier: for
         example it may start with a digit and contain non-alphanumeric
@@ -46,14 +47,16 @@ The ``connection`` class
             Consider it as part of the query, not as a query parameter.
 
         The *cursor_factory* argument can be used to create non-standard
-        cursors. The class returned should be a subclass of
+        cursors. The class returned must be a subclass of
         `psycopg2.extensions.cursor`. See :ref:`subclassing-cursor` for
         details.
 
+        .. versionchanged:: 2.4.3 added the *withhold* argument.
+        .. versionchanged:: 2.4.6 added the *scrollable* argument.
+
         .. extension::
 
-            The `name` and `cursor_factory` parameters are Psycopg
+            All the function arguments are Psycopg extensions to the |DBAPI|.
-            extensions to the |DBAPI|.
 
 
     .. index::

doc/src/cursor.rst

@@ -114,13 +114,44 @@ The ``cursor`` class
             The `name` attribute is a Psycopg extension to the |DBAPI|.
 
 
+    .. attribute:: scrollable
+
+        Read/write attribute: specifies if a named cursor is declared
+        :sql:`SCROLL`, hence is capable to scroll backwards (using
+        `~cursor.scroll()`). If `!True`, the cursor can be scrolled backwards,
+        if `!False` it is never scrollable. If `!None` (default) the cursor
+        scroll option is not specified, usually but not always meaning no
+        backward scroll (see the |declare-notes|__).
+
+        .. |declare-notes| replace:: :sql:`DECLARE` notes
+        .. __: http://www.postgresql.org/docs/current/static/sql-declare.html#SQL-DECLARE-NOTES
+
+        .. note::
+
+            set the value before calling `~cursor.execute()` or use the
+            `connection.cursor()` *scrollable* parameter, otherwise the value
+            will have no effect.
+
+        .. versionadded:: 2.4.6
+
+        .. extension::
+
+            The `scrollable` attribute is a Psycopg extension to the |DBAPI|.
+
+
     .. attribute:: withhold
 
         Read/write attribute: specifies if a named cursor lifetime should
         extend outside of the current transaction, i.e., it is possible to
-        fetch from the cursor even after a `commection.commit()` (but not after
+        fetch from the cursor even after a `connection.commit()` (but not after
         a `connection.rollback()`).  See :ref:`server-side-cursors`
 
+        .. note::
+
+            set the value before calling `~cursor.execute()` or use the
+            `connection.cursor()` *withhold* parameter, otherwise the value
+            will have no effect.
+
         .. versionadded:: 2.4.3
 
         .. extension::
@@ -297,7 +328,8 @@ The ``cursor`` class
         not changed.
 
         The method can be used both for client-side cursors and
-        :ref:`server-side cursors <server-side-cursors>`.
+        :ref:`server-side cursors <server-side-cursors>`. Server-side cursors
+        can usually scroll backwards only if declared `~cursor.scrollable`.
 
         .. note:: 
 

doc/src/usage.rst

@@ -576,7 +576,9 @@ cursor is created using the `~connection.cursor()` method specifying the
 *name* parameter. Such cursor will behave mostly like a regular cursor,
 allowing the user to move in the dataset using the `~cursor.scroll()`
 method and to read the data using `~cursor.fetchone()` and
-`~cursor.fetchmany()` methods.
+`~cursor.fetchmany()` methods. Normally you can only scroll forward in a
+cursor: if you need to scroll backwards you should declare your cursor
+`~cursor.scrollable`.
 
 Named cursors are also :ref:`iterable <cursor-iterable>` like regular cursors.
 Note however that before Psycopg 2.4 iteration was performed fetching one

psycopg/connection_type.c

@@ -52,63 +52,70 @@
 static PyObject *
 psyco_conn_cursor(connectionObject *self, PyObject *args, PyObject *kwargs)
 {
-    PyObject *obj;
+    PyObject *obj = NULL;
+    PyObject *rv = NULL;
     PyObject *name = Py_None;
     PyObject *factory = (PyObject *)&cursorType;
     PyObject *withhold = Py_False;
+    PyObject *scrollable = Py_None;
 
-    static char *kwlist[] = {"name", "cursor_factory", "withhold", NULL};
+    static char *kwlist[] = {
-
+        "name", "cursor_factory", "withhold", "scrollable", NULL};
-    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "|OOO", kwlist,
-                                     &name, &factory, &withhold)) {
-        return NULL;
-    }
-
-    if (PyObject_IsTrue(withhold) && (name == Py_None)) {
-        PyErr_SetString(ProgrammingError,
-            "'withhold=True can be specified only for named cursors");
-        return NULL;
-    }
 
     EXC_IF_CONN_CLOSED(self);
 
+    if (!PyArg_ParseTupleAndKeywords(
+            args, kwargs, "|OOOO", kwlist,
+            &name, &factory, &withhold, &scrollable)) {
+        goto exit;
+    }
+
     if (self->status != CONN_STATUS_READY &&
         self->status != CONN_STATUS_BEGIN &&
         self->status != CONN_STATUS_PREPARED) {
         PyErr_SetString(OperationalError,
                         "asynchronous connection attempt underway");
-        return NULL;
+        goto exit;
     }
 
     if (name != Py_None && self->async == 1) {
         PyErr_SetString(ProgrammingError,
                         "asynchronous connections "
                         "cannot produce named cursors");
-        return NULL;
+        goto exit;
     }
 
     Dprintf("psyco_conn_cursor: new %s cursor for connection at %p",
         (name == Py_None ? "unnamed" : "named"), self);
 
     if (!(obj = PyObject_CallFunctionObjArgs(factory, self, name, NULL))) {
-        return NULL;
+        goto exit;
     }
 
     if (PyObject_IsInstance(obj, (PyObject *)&cursorType) == 0) {
         PyErr_SetString(PyExc_TypeError,
             "cursor factory must be subclass of psycopg2._psycopg.cursor");
-        Py_DECREF(obj);
+        goto exit;
-        return NULL;
     }
 
-    if (PyObject_IsTrue(withhold))
+    if (0 != psyco_curs_withhold_set((cursorObject *)obj, withhold)) {
-        ((cursorObject*)obj)->withhold = 1;
+        goto exit;
+    }
+    if (0 != psyco_curs_scrollable_set((cursorObject *)obj, scrollable)) {
+        goto exit;
+    }
 
     Dprintf("psyco_conn_cursor: new cursor at %p: refcnt = "
         FORMAT_CODE_PY_SSIZE_T,
         obj, Py_REFCNT(obj)
-      );
+    );
-    return obj;
+
+    rv = obj;
+    obj = NULL;
+
+exit:
+    Py_XDECREF(obj);
+    return rv;
 }
 
 

psycopg/cursor.h

@@ -44,6 +44,11 @@ struct cursorObject {
     int notuples:1;          /* 1 if the command was not a SELECT query */
     int withhold:1;          /* 1 if the cursor is named and uses WITH HOLD */
 
+    int scrollable;          /* 1 if the cursor is named and SCROLLABLE,
+                                0 if not scrollable
+                               -1 if undefined (PG may decide scrollable or not)
+                              */
+
     long int rowcount;       /* number of rows affected by last execute */
     long int columns;        /* number of columns fetched from the db */
     long int arraysize;      /* how many rows should fetchmany() return */
@@ -84,9 +89,11 @@ struct cursorObject {
 };
 
 
-/* C-callable functions in cursor_int.c and cursor_ext.c */
+/* C-callable functions in cursor_int.c and cursor_type.c */
 BORROWED HIDDEN PyObject *curs_get_cast(cursorObject *self, PyObject *oid);
 HIDDEN void curs_reset(cursorObject *self);
+HIDDEN int psyco_curs_withhold_set(cursorObject *self, PyObject *pyvalue);
+HIDDEN int psyco_curs_scrollable_set(cursorObject *self, PyObject *pyvalue);
 
 /* exception-raising macros */
 #define EXC_IF_CURS_CLOSED(self) \

psycopg/cursor_type.c

@@ -370,6 +370,7 @@ _psyco_curs_execute(cursorObject *self,
     int res = -1;
     int tmp;
     PyObject *fquery, *cvt = NULL;
+    const char *scroll;
 
     operation = _psyco_curs_validate_sql_basic(self, operation);
 
@@ -396,6 +397,21 @@ _psyco_curs_execute(cursorObject *self,
         if (0 > _mogrify(vars, operation, self, &cvt)) { goto exit; }
     }
 
+    switch (self->scrollable) {
+        case -1:
+            scroll = "";
+            break;
+        case 0:
+            scroll = "NO SCROLL ";
+            break;
+        case 1:
+            scroll = "SCROLL ";
+            break;
+        default:
+            PyErr_SetString(InternalError, "unexpected scrollable value");
+            goto exit;
+    }
+
     if (vars && cvt) {
         if (!(fquery = _psyco_curs_merge_query_args(self, operation, cvt))) {
             goto exit;
@@ -403,8 +419,9 @@ _psyco_curs_execute(cursorObject *self,
 
         if (self->name != NULL) {
             self->query = Bytes_FromFormat(
-                "DECLARE \"%s\" CURSOR %s HOLD FOR %s",
+                "DECLARE \"%s\" %sCURSOR %s HOLD FOR %s",
                 self->name,
+                scroll,
                 self->withhold ? "WITH" : "WITHOUT",
                 Bytes_AS_STRING(fquery));
             Py_DECREF(fquery);
@@ -416,8 +433,9 @@ _psyco_curs_execute(cursorObject *self,
     else {
         if (self->name != NULL) {
             self->query = Bytes_FromFormat(
-                "DECLARE \"%s\" CURSOR %s HOLD FOR %s",
+                "DECLARE \"%s\" %sCURSOR %s HOLD FOR %s",
                 self->name,
+                scroll,
                 self->withhold ? "WITH" : "WITHOUT",
                 Bytes_AS_STRING(operation));
         }
@@ -1565,12 +1583,12 @@ psyco_curs_withhold_get(cursorObject *self)
     return ret;
 }
 
-static int
+int
 psyco_curs_withhold_set(cursorObject *self, PyObject *pyvalue)
 {
     int value;
 
-    if (self->name == NULL) {
+    if (pyvalue != Py_False && self->name == NULL) {
         PyErr_SetString(ProgrammingError,
             "trying to set .withhold on unnamed cursor");
         return -1;
@@ -1584,6 +1602,54 @@ psyco_curs_withhold_set(cursorObject *self, PyObject *pyvalue)
     return 0;
 }
 
+#define psyco_curs_scrollable_doc \
+"Set or return cursor use of SCROLL"
+
+static PyObject *
+psyco_curs_scrollable_get(cursorObject *self)
+{
+    PyObject *ret = NULL;
+
+    switch (self->scrollable) {
+        case -1:
+            ret = Py_None;
+            break;
+        case 0:
+            ret = Py_False;
+            break;
+        case 1:
+            ret = Py_True;
+            break;
+        default:
+            PyErr_SetString(InternalError, "unexpected scrollable value");
+    }
+
+    Py_XINCREF(ret);
+    return ret;
+}
+
+int
+psyco_curs_scrollable_set(cursorObject *self, PyObject *pyvalue)
+{
+    int value;
+
+    if (pyvalue != Py_None && self->name == NULL) {
+        PyErr_SetString(ProgrammingError,
+            "trying to set .scrollable on unnamed cursor");
+        return -1;
+    }
+
+    if (pyvalue == Py_None) {
+        value = -1;
+    } else if ((value = PyObject_IsTrue(pyvalue)) == -1) {
+        return -1;
+    }
+
+    self->scrollable = value;
+
+    return 0;
+}
+
 #endif
 
 
@@ -1710,6 +1776,10 @@ static struct PyGetSetDef cursorObject_getsets[] = {
       (getter)psyco_curs_withhold_get,
       (setter)psyco_curs_withhold_set,
       psyco_curs_withhold_doc, NULL },
+    { "scrollable",
+      (getter)psyco_curs_scrollable_get,
+      (setter)psyco_curs_scrollable_set,
+      psyco_curs_scrollable_doc, NULL },
 #endif
     {NULL}
 };
@@ -1740,6 +1810,7 @@ cursor_setup(cursorObject *self, connectionObject *conn, const char *name)
 
     self->closed = 0;
     self->withhold = 0;
+    self->scrollable = 0;
     self->mark = conn->mark;
     self->pgres = NULL;
     self->notuples = 1;

tests/test_cursor.py

@@ -213,6 +213,71 @@ class CursorTests(unittest.TestCase):
         curs.execute("drop table withhold")
         self.conn.commit()
 
+    def test_scrollable(self):
+        self.assertRaises(psycopg2.ProgrammingError, self.conn.cursor,
+                          scrollable=True)
+
+        curs = self.conn.cursor()
+        curs.execute("create table scrollable (data int)")
+        curs.executemany("insert into scrollable values (%s)",
+            [ (i,) for i in range(100) ])
+        curs.close()
+
+        for t in range(2):
+            if not t:
+                curs = self.conn.cursor("S")
+                self.assertEqual(curs.scrollable, None);
+                curs.scrollable = True
+            else:
+                curs = self.conn.cursor("S", scrollable=True)
+
+            self.assertEqual(curs.scrollable, True);
+            curs.itersize = 10
+
+            # complex enough to make postgres cursors declare without
+            # scroll/no scroll to fail
+            curs.execute("""
+                select x.data
+                from scrollable x
+                join scrollable y on x.data = y.data
+                order by y.data""")
+            for i, (n,) in enumerate(curs):
+                self.assertEqual(i, n)
+
+            curs.scroll(-1)
+            for i in range(99, -1, -1):
+                curs.scroll(-1)
+                self.assertEqual(i, curs.fetchone()[0])
+                curs.scroll(-1)
+
+            curs.close()
+
+    def test_not_scrollable(self):
+        self.assertRaises(psycopg2.ProgrammingError, self.conn.cursor,
+                          scrollable=False)
+
+        curs = self.conn.cursor()
+        curs.execute("create table scrollable (data int)")
+        curs.executemany("insert into scrollable values (%s)",
+            [ (i,) for i in range(100) ])
+        curs.close()
+
+        curs = self.conn.cursor("S")    # default scrollability
+        curs.execute("select * from scrollable")
+        self.assertEqual(curs.scrollable, None)
+        curs.scroll(2)
+        try:
+            curs.scroll(-1)
+        except psycopg2.OperationalError:
+            return self.skipTest("can't evaluate non-scrollable cursor")
+        curs.close()
+
+        curs = self.conn.cursor("S", scrollable=False)
+        self.assertEqual(curs.scrollable, False)
+        curs.execute("select * from scrollable")
+        curs.scroll(2)
+        self.assertRaises(psycopg2.OperationalError, curs.scroll, -1)
+
     @skip_before_postgres(8, 2)
     def test_iter_named_cursor_efficient(self):
         curs = self.conn.cursor('tmp')