Commit Graph

769 Commits

Author SHA1 Message Date
Daniele Varrazzo
e154cbe5aa Skip connection tests which cannot pass on CockroachDB
Features not supported seem:

- isolation level (always serializable)
- client encodings
- notices (maybe there is a way to generate them)
- 2 phase commit
- reset (because of the lack of transaction deferrable)
- backend pid
2020-07-21 22:22:23 +01:00
Daniele Varrazzo
bca72937d8 Expose libpq PG* vars as testconfig content 2020-07-21 22:21:44 +01:00
Daniele Varrazzo
f8c1cff6a3 Skip cancel tests on CockroachDB
One test moved to the async tests module, as it really belongs there.
2020-07-21 22:21:32 +01:00
Daniele Varrazzo
ee34198bf6 All the sync tests pass on CockroachDB
Added decorator to skip tests on crdb
2020-07-21 22:21:24 +01:00
Daniele Varrazzo
659910ee81 Allow most of the async tests to pass on CockroachDB
Added function to get crdb version from a connection
2020-07-21 01:43:57 +01:00
Rafi Shamim
a61f30b2d2 Handle failure in setup of IsolationLevelsTestCase
If the CREATE TABLE statement fails, the setup would fail
without committing or rolling back the active transaction, so the
transaction would hold onto its resources indefinitely.

Normally, the transaction would be closed when the connection is closed
in the `tearDown` function. However, `tearDown` is not called if there
was an error during `setUp` ([as specified by the `unittest` docs](https://docs.python.org/3/library/unittest.html#unittest.TestCase.tearDown)), so
we need to handle this case specially.
2020-07-07 18:48:44 -04:00
Rafi Shamim
f935476b3f Handle failure in setup of ConnectionTwoPhaseTest
Previously, this test had a bug, because if the CREATE TABLE statement
failed, the setup would fail without committing or rolling back the
active transaction.
2020-05-12 12:48:35 -04:00
Daniele Varrazzo
62743c3be1 Fixed copy() on DictRow
Close #1073.
2020-04-06 14:10:02 +12:00
Daniele Varrazzo
054123254e Column objects can be sliced
Close #1034.
2020-03-11 10:50:56 +13:00
Daniele Varrazzo
5e0b02afb2 Copyright bumped to 2020 2020-01-17 21:21:11 +00:00
Daniele Varrazzo
2f094841b2 Slow test is slow 2019-10-19 16:29:16 +01:00
Daniele Varrazzo
8f11821c17 Use travis test config variable 0/1 instead of blank strings
Travis web interface doesn't allow anymore to set an empty string as
variable value.
2019-10-19 15:18:38 +02:00
Daniele Varrazzo
f08019e356 Added decorator to clean up the adaptation mappings after tests
Many tests were doing it manually, some weren't doing it and resulted in
failure if run in different order.

Close #948
2019-09-04 18:17:51 +01:00
Changaco
842e383c0c fix NamedTupleCursor._cached_make_nt 2019-06-04 14:30:30 +02:00
Changaco
527592a0a5 improve the NamedTupleCursor cache test 2019-06-04 13:45:37 +02:00
Alexander Kukushkin
6cff5a3e08 Smart replication feedback
This commit makes psycopg2 responsible for sending the status update
(feedback) messages to the server regardless of whether a synchronous or
asynchronous connection is used.

Feedback is sent every *status_update* (default value is 10) seconds,
which could be configured by passing a corresponding parameter to the
`start_replication()` or `start_replication_expert()` methods.
The actual feedback message is sent by the
`pq_read_replication_message()` when the *status_update* timeout is
reached.

The default behavior of the `send_feedback()` method is changed.
It doesn't send a feedback message on every call anymore but just
updates internal structures. There is still a way to *force* sending
a message if *force* or *reply* parameters are set.

The new approach has certain advantages:
1. The client can simply call the `send_feedback()` for every
   processed message and the library will take care of not overwhelming
   the server. Actually, in the synchronous mode it is even mandatory
   to confirm every processed message.
2. The library tracks internally the pointer of the last received
   message which is not keepalive. If the client confirmed the last
   message and after that server sends only keepalives with increasing
   *wal_end*, the library can safely move forward *flush* position to
   the *wal_end* and later automatically report it to the server.

Reporting of the *wal_end* received from keepalive messages is very
important. Not doing so casing:
1. Excessive disk usage, because the replication slot prevents from
   WAL being cleaned up.
2. The smart and fast shutdown of the server could last indefinitely
   because walsender waits until the client report *flush* position
   equal to the *wal_end*.

This implementation is only extending the existing API and therefore
should not break any of the existing code.
2019-05-06 10:27:44 +02:00
Daniele Varrazzo
cc815e8e8d RealDictRow inherits from OrderedDict
Now its state is unmodified, so apart from special-casing creation
and initial population can work unmodified, and all the desired
properties just work (modifiability, picklability...)

Close #886.
2019-04-06 19:43:31 +01:00
Daniele Varrazzo
21d16b6f67 Fixed RealDictCursor.pop()
Addresses #886, but there might be something else broken there.
2019-04-06 19:43:09 +01:00
Daniele Varrazzo
46106e1b78 Test notifies are received ok polling an async cur 2019-04-05 19:04:48 +01:00
Daniele Varrazzo
755a128ffb Handle ok poll() without a cursor having executed queries
Close #887
2019-04-05 18:04:55 +01:00
Martins Grunskis
f946042a79 Store WAL end pointer in the replication cursor 2019-03-30 21:23:11 +00:00
Daniele Varrazzo
953bc66ca6 Allow tests to be ignored if dynamic binding failed 2019-03-18 19:49:16 +00:00
Daniele Varrazzo
e569e49b8b Merge branch 'fix-829' 2019-03-18 01:53:59 +00:00
Daniele Varrazzo
0c581380c7 Allow incorrect result for pgconn_ptr test on OSX
I don't know why it returns 0 instead of the right value. At least it
doesn't segfault, so don't skip the test altogether.

The test is unrelated to this branch: will cherry-pick elsewhere (if I
remember it...)
2019-03-18 01:41:56 +00:00
Daniele Varrazzo
62a078fe0c subprocess test function moved into a module
It won't work on windows if it's in the script: failing with errors
such as:

    AttributeError: 'module' object has no attribute 'process'

or:

    Can't get attribute 'process' on <module '__main__' (built-in)>
2019-03-17 23:55:04 +00:00
Daniele Varrazzo
7c5afd6977 Added test to reproduce ticket #829
Unrelated processes close the FD of the connection. This happens in
Python 3.6 but not 2.7. Let's see if travis shows where else it fails...
2019-03-17 23:06:55 +00:00
Daniele Varrazzo
734845b79a Added pq_get_result_async() replaced pg_get_last_result()
The new function keeps together PQconsumeInput() with PQisBusy(), in
order to handle the condition in which not all the results of a sequence
of statements arrive in the same roundtrip.

Added pointer to a PGresult to the connection to keep the state across
async communication: it can probably be used to simplify other code
paths where a result is brought forward manually.

Close #802
Close #855
Close #856
2019-03-17 01:20:06 +00:00
Daniele Varrazzo
d61c902230 Shorter timeout in the async tests select 2019-03-16 20:05:40 +00:00
Daniele Varrazzo
dfb301b42b Use PY2, PY3 for conditional code instead of sys.version_info 2019-03-16 19:54:40 +00:00
Daniele Varrazzo
e8135ee2cf Use errors module to catch a specific postgres error 2019-03-16 19:13:58 +00:00
Daniele Varrazzo
b0119fef81 Other import moved to top level in tests 2019-03-16 19:13:58 +00:00
Daniele Varrazzo
8cfe176a85 Dropped repeated conditional import of reload in test 2019-03-16 19:04:40 +00:00
Daniele Varrazzo
aaba4dcf87 TextIOBase moved to tests compat imports 2019-03-16 18:41:59 +00:00
Jon Dufresne
d90ad8627d Move imports to the top of the module across tests
Allows removing many duplicate imports and better follows PEP8
guidelines:

https://www.python.org/dev/peps/pep-0008/#imports

> Imports are always put at the top of the file, just after any module
> comments and docstrings, and before module globals and constants.
2019-03-16 18:03:41 +00:00
Jon Dufresne
194efc4375 Fix invalid exception handler: except e:
Fixes flake8 warning:

    ./tests/test_connection.py:390:16: F821 undefined name 'e'
    ./tests/test_connection.py:391:61: F821 undefined name 'e'
    ./tests/test_connection.py:408:16: F821 undefined name 'e'
    ./tests/test_connection.py:409:61: F821 undefined name 'e'

In the event of an unexpected error, let the exception bubble up the
stack for a more informative test failure message.
2019-03-16 17:59:18 +00:00
Jon Dufresne
ad4c6a4673 Use unittest 'skip' feature to skip mxDateTimeTests
Rather than deleting, the class, use the skip feature. Provides a more
informative message during test output.

Never skip DatetimeTests as all supported Python environments have the
datetime module builtin.
2019-03-13 15:01:34 +00:00
Jon Dufresne
afbbdd18b6 Remove unused variable
The variable i is immediately overwritten by the next line.
2019-03-13 11:13:24 +00:00
Jon Dufresne
03bb44dd2c Convert while 1: statements to while True:
A slightly more readable and modern syntax.
2019-03-13 11:13:05 +00:00
Jon Dufresne
18f5d5ad05 Remove unused imports from tests/test_ipaddress.py 2019-03-13 11:12:48 +00:00
Daniele Varrazzo
a68df50c7b Added ConnectionInfo.dsn_parameters attribute 2019-02-17 02:37:09 +00:00
Daniele Varrazzo
599432552a Copyright year updated
ag -l Copyright | xargs sed -i \
        "s/\(.*copyright (C) [0-9]\+\)\(-[0-9]\+\)\?\(.*\)/\1-$(date +%Y)\3/I"
2019-02-17 01:36:36 +00:00
Daniele Varrazzo
d08be18671 Merge branch 'libpq-ptrs' 2019-02-17 00:17:43 +01:00
Daniele Varrazzo
6bbfce7b89 Skip password encryption test if libpq < 10 2019-02-16 22:57:16 +01:00
Daniele Varrazzo
3b7c083c3d Skip tests involving ctypes on Windows
No idea about how to import libpq.
2019-02-16 18:12:52 +01:00
Daniele Varrazzo
7c7bbb9742 Added connection.pgconn_ptr and cursor.pgresult_ptr
Allow interacting with libpq in Python via ctypes.

See #782.
2019-02-16 18:12:52 +01:00
Daniele Varrazzo
3de4d17519 Fixed use of StandardError in test
In Py3 it's gone.
2019-02-11 00:26:01 +00:00
Daniele Varrazzo
30c1befa64 SQLSTATE error classes implemented in C
The module is only used to export them to Python.
2019-02-10 03:45:14 +00:00
Daniele Varrazzo
35ec7ad9c1 Use a proper LRU cache for namedtuples
Previous one didn't refresh by last use. Use the stdlib version for py3
and one of our own for py2.

Max size set to 512, which should be fine for everyone (tweaking is
still possible by monkeypatching, as the tests do, but I don't want to
make an interface of it).
2019-02-02 19:29:20 +00:00
Daniele Varrazzo
805527fcd6 Added caching of types generated by NamedTupleCursor
see #838
2019-02-02 14:22:25 +00:00
Daniele Varrazzo
ddbe495d70 Added BYTESARRAY typecaster 2019-01-18 15:47:01 +00:00
Daniele Varrazzo
4ab4247189 Added test for BYTES typecaster 2019-01-18 15:10:17 +00:00
Daniele Varrazzo
f3695e36c7 Merge remote-tracking branch 'eternalflow/execute-values-returning-clause-support' 2018-12-27 14:53:12 +01:00
Jon Dufresne
eb2d1766c6 Remove unnecessary test decorator 'skip_if_cant_cast'
ctypes is available and works on all supported Pythons. It has been
available since Python 2.5. The tests were written when Python 2.4 was
still supported.
2018-12-04 18:20:17 -08:00
Daniele Varrazzo
68bacbb194
Merge pull request #822 from jdufresne/decorate-all
Simplify 'decorate_all_tests' usage by decorating the test class
2018-12-04 14:22:02 +00:00
Jon Dufresne
a739c09f67 Simplify 'decorate_all_tests' usage by decorating the test class
Skip tests as early as possible by decorating the whole class with
unittest.skipIf instead of every test method individually.
2018-12-01 19:40:27 -08:00
Jon Dufresne
17dc5a85a3 Remove workarounds for unsupported libpq < 9.1
Per http://initd.org/psycopg/docs/install.html#prerequisites:

> PostgreSQL client library version from 9.1
2018-12-01 08:58:39 -08:00
Ivan Kotelnikov
b40ac15efc rename param name to fetch 2018-11-24 20:51:18 +02:00
Ivan Kotelnikov
e2b99d3a8e write test for fetch_result flag 2018-11-22 12:30:37 +02:00
Daniele Varrazzo
73a680f45d Convert int subclasses to long before adapting
Fixes adaptation of int/long subclasses whose str() is not the number,
such IntEnum

Close #591

Note that I thought it would have needed a new adapter, so I considered
it a new feature. But it is more a shortcoming of the int adapter
failing to do something reasonable (poor Liskov, always mistreated) so I
may actually backport it if there is a new 2.7 release.
2018-11-16 18:16:03 +00:00
Federico Di Gregorio
01f8475f53
Merge branch 'master' into feature-expose-pgconn 2018-11-07 14:15:29 +01:00
Daniele Varrazzo
a83696fc50 Fixed adaptation of lists of empty lists
...somehow. Postgres doesn't support them and converts them into a
simple empty array. However this is not really our concern: the syntax
we return is valid.

Close #788
2018-10-30 01:48:51 +00:00
Daniele Varrazzo
2f24a2e22d Use class decorators to decorate all tests
Test decorators changed so that they can be applied either to a method
or to a class. Of course their double nature is implemented by a
decorator.
2018-10-30 00:23:56 +00:00
Daniele Varrazzo
8fb0f694f7 Don't barf on Composite passed to execute_values()
Close #794
2018-10-23 01:57:48 +01:00
Daniele Varrazzo
05f9e231a0 Full flake8 3.5 cleanup 2018-10-23 00:39:14 +01:00
Daniele Varrazzo
0236c68da9 Merge branch 'connection-info' 2018-10-15 01:24:00 +01:00
Daniele Varrazzo
b205764fdd Merge branch 'master' into errors-module 2018-10-15 00:58:32 +01:00
Daniele Varrazzo
e7227ce87b Added errors.lookup() function 2018-10-15 00:56:51 +01:00
Daniele Varrazzo
44bd2927c5 Use the connection.info properties instead of the legacy methods 2018-10-13 03:28:42 +01:00
Daniele Varrazzo
704e6797e7 Guard from some info functions not available in some libpq versions 2018-10-13 03:09:39 +01:00
Daniele Varrazzo
d138e42ee5 Added ConnectionInfo.parameter_status() 2018-10-13 02:40:17 +01:00
Daniele Varrazzo
9f6a3a5e96 Added ConnectionInfo.ssl_attribute() 2018-10-13 02:21:38 +01:00
Daniele Varrazzo
cb3d5f9d92 Added all the missing ConnectionInfo attributes 2018-10-13 01:36:07 +01:00
Daniele Varrazzo
439dff974d Added ConnectionInfo.error_message 2018-10-13 00:47:04 +01:00
Daniele Varrazzo
d29aa1c437 Fixed refcount in connection's readonly and deferrable getters
Close #790
2018-10-12 23:32:13 +01:00
Daniele Varrazzo
1ac6359fef Added other members to the ConnectionInfo class
Starting deprecating softly some of the methods bloating the connection
class.
2018-10-12 04:18:59 +01:00
Daniele Varrazzo
0a04c8892d Added several ConnectionInfo attributes 2018-10-12 04:18:59 +01:00
Daniele Varrazzo
0e2b516a3c Moving host attribute to a connection.info object 2018-10-11 22:42:52 +01:00
Daniele Varrazzo
f99a8de6d0 Added table_oid, table_column on cursor.description items
Close #661
2018-10-11 03:37:09 +01:00
Daniele Varrazzo
c314512115 Fixed infinite loop in pq_get_last_result after COPY
There will be an error downstream but we have to get out of this
function first.

Close #781
2018-10-10 23:57:34 +01:00
Daniele Varrazzo
9148157697 Merge branch 'master' into https 2018-10-10 22:07:33 +01:00
Daniele Varrazzo
cd9d74c462 Merge branch 'conn-get-host' 2018-10-10 22:03:36 +01:00
Daniele Varrazzo
f5f6b420b2 Merge remote-tracking branch 'origin/diag-schema-name-nonloc' 2018-10-10 21:46:51 +01:00
Federico Di Gregorio
296c80d1f2 Commented out test to avoid beaking master 2018-10-10 11:05:02 +02:00
Federico Di Gregorio
1fe9f1ac5b Fixed test for issue #788 2018-10-10 09:35:18 +02:00
Federico Di Gregorio
ebcfbe03f9 Added test for issue #788 2018-10-10 09:14:56 +02:00
Federico Di Gregorio
81addddaee Added connection.get_native_connection() 2018-10-07 13:54:24 +02:00
Marco De Paoli
1c553bb703 Added connection.host
Return the server host name of the current connect.
2018-10-06 15:19:01 +02:00
Daniele Varrazzo
4e0b2ec9c9 Added Diagnostics.severity_nonlocalized attribute
Close #783.
2018-10-04 16:13:46 +01:00
Daniele Varrazzo
4aa02b7855 sql.Identifier can wrap a sequence of strings to represent qualified names
Close #732.
2018-10-04 12:46:10 +01:00
Michel Albert
d317977205 Implement __str__ for range types 2018-09-30 10:15:55 +02:00
Jon Dufresne
b07e34e0b8 Prefer https:// URLs when available 2018-09-22 19:02:33 -07:00
Daniele Varrazzo
344ce15261 Better testing of encryption function with libpq < 10 2018-08-17 02:09:22 +01:00
Daniele Varrazzo
ce1ac3aea9 Read exceptions to raise from a Python module 2018-08-17 02:00:01 +01:00
Daniele Varrazzo
466efe4461 Bump tests for selective closure of named cursor to pg 8.2
Previous versions don't support the features as they don't have the
pg_cursors view. But they are too old to care.
2018-07-24 21:16:36 +01:00
Daniele Varrazzo
6d8f4f9f0d Close named cursor if exist, even if we didn't run execute
Close #746
2018-07-24 21:16:36 +01:00
Daniele Varrazzo
0bb7d0db48 DictCursor and RealDictCursor rows maintain columns order
Close #177.
2018-05-21 03:14:08 +01:00
Daniele Varrazzo
e8a831dda2 Merge branch 'master' into drop-2to3 2018-05-20 23:56:29 +01:00
Daniele Varrazzo
a0f7027ad5 Added tests to verify iter methods on dict cursors
ISTM the refactoring in #648 broke something
2018-05-20 23:55:18 +01:00
Daniele Varrazzo
82ae44ac3a DictCursor/RealDictCursor tests split 2018-05-20 23:35:26 +01:00
Daniele Varrazzo
9cf658ec6e Fixed refcount handling in encrypt_password
Added tests to check bad types, which discovered the above problem: on
type error we would have decref'd on exit something that was only
borrowed (because we wouldn't have performed matching increfs).
2018-05-20 21:18:36 +01:00