Merge branch 'master' of https://github.com/encode/django-rest-framework

.travis.yml

@@ -21,11 +21,13 @@ matrix:
       - { python: "3.7", env: DJANGO=2.2 }
       - { python: "3.7", env: DJANGO=master }
 
-      - { python: "3.7", env: TOXENV=base }
+      - { python: "3.8", env: DJANGO=master }
-      - { python: "3.7", env: TOXENV=lint }
-      - { python: "3.7", env: TOXENV=docs }
 
-      - python: "3.7"
+      - { python: "3.8", env: TOXENV=base }
+      - { python: "3.8", env: TOXENV=lint }
+      - { python: "3.8", env: TOXENV=docs }
+
+      - python: "3.8"
         env: TOXENV=dist
         script:
           - python setup.py bdist_wheel

README.md

@@ -26,8 +26,9 @@ The initial aim is to provide a single full-time position on REST framework.
 [![][kloudless-img]][kloudless-url]
 [![][esg-img]][esg-url]
 [![][lightson-img]][lightson-url]
+[![][retool-img]][retool-url]
 
-Many thanks to all our [wonderful sponsors][sponsors], and in particular to our premium backers, [Sentry][sentry-url], [Stream][stream-url], [Rollbar][rollbar-url], [Cadre][cadre-url], [Kloudless][kloudless-url], [ESG][esg-url], and [Lights On Software][lightson-url].
+Many thanks to all our [wonderful sponsors][sponsors], and in particular to our premium backers, [Sentry][sentry-url], [Stream][stream-url], [Rollbar][rollbar-url], [Cadre][cadre-url], [Kloudless][kloudless-url], [ESG][esg-url], [Lights On Software][lightson-url], and [Retool][retool-url].
 
 ---
 
@@ -199,6 +200,7 @@ Please see the [security policy][security-policy].
 [kloudless-img]: https://raw.githubusercontent.com/encode/django-rest-framework/master/docs/img/premium/kloudless-readme.png
 [esg-img]: https://raw.githubusercontent.com/encode/django-rest-framework/master/docs/img/premium/esg-readme.png
 [lightson-img]: https://raw.githubusercontent.com/encode/django-rest-framework/master/docs/img/premium/lightson-readme.png
+[retool-img]: https://raw.githubusercontent.com/encode/django-rest-framework/master/docs/img/premium/retool-readme.png
 
 [sentry-url]: https://getsentry.com/welcome/
 [stream-url]: https://getstream.io/try-the-api/?utm_source=drf&utm_medium=banner&utm_campaign=drf
@@ -207,6 +209,7 @@ Please see the [security policy][security-policy].
 [kloudless-url]: https://hubs.ly/H0f30Lf0
 [esg-url]: https://software.esg-usa.com/
 [lightson-url]: https://lightsonsoftware.com
+[retool-url]: https://retool.com/?utm_source=djangorest&utm_medium=sponsorship
 
 [oauth1-section]: https://www.django-rest-framework.org/api-guide/authentication/#django-rest-framework-oauth
 [oauth2-section]: https://www.django-rest-framework.org/api-guide/authentication/#django-oauth-toolkit

docs/api-guide/fields.md

@@ -52,7 +52,7 @@ The `default` is not applied during partial update operations. In the partial up
 
 May be set to a function or other callable, in which case the value will be evaluated each time it is used. When called, it will receive no arguments. If the callable has a `set_context` method, that will be called each time before getting the value with the field instance as only argument. This works the same way as for [validators](validators.md#using-set_context).
 
-When serializing the instance, default will be used if the the object attribute or dictionary key is not present in the instance.
+When serializing the instance, default will be used if the object attribute or dictionary key is not present in the instance.
 
 Note that setting a `default` value implies that the field is not required. Including both the `default` and `required` keyword arguments is invalid and will raise an error.
 

docs/api-guide/schemas.md

@@ -1,6 +1,6 @@
 ---
 source:
-    - schemas.py
+    - schemas
 ---
 
 # Schema
@@ -90,6 +90,7 @@ The `get_schema_view()` helper takes the following keyword arguments:
             url='https://www.example.org/api/',
             urlconf='myproject.urls'
         )
+
 * `patterns`: List of url patterns to limit the schema introspection to. If you
   only want the `myproject.api` urls to be exposed in the schema:
 

docs/api-guide/serializers.md

@@ -887,10 +887,10 @@ To implement a read-only serializer using the `BaseSerializer` class, we just ne
 It's simple to create a read-only serializer for converting `HighScore` instances into primitive data types.
 
     class HighScoreSerializer(serializers.BaseSerializer):
-        def to_representation(self, obj):
+        def to_representation(self, instance):
             return {
-                'score': obj.score,
+                'score': instance.score,
-                'player_name': obj.player_name
+                'player_name': instance.player_name
             }
 
 We can now use this class to serialize single `HighScore` instances:
@@ -945,10 +945,10 @@ Here's a complete example of our previous `HighScoreSerializer`, that's been upd
                 'player_name': player_name
             }
 
-        def to_representation(self, obj):
+        def to_representation(self, instance):
             return {
-                'score': obj.score,
+                'score': instance.score,
-                'player_name': obj.player_name
+                'player_name': instance.player_name
             }
 
         def create(self, validated_data):
@@ -965,10 +965,10 @@ The following class is an example of a generic serializer that can handle coerci
         A read-only serializer that coerces arbitrary complex objects
         into primitive representations.
         """
-        def to_representation(self, obj):
+        def to_representation(self, instance):
             output = {}
-            for attribute_name in dir(obj):
+            for attribute_name in dir(instance):
-                attribute = getattr(obj, attribute_name)
+                attribute = getattr(instance, attribute_name)
                 if attribute_name.startswith('_'):
                     # Ignore private attributes.
                     pass
@@ -1010,11 +1010,11 @@ Some reasons this might be useful include...
 
 The signatures for these methods are as follows:
 
-#### `.to_representation(self, obj)`
+#### `.to_representation(self, instance)`
 
 Takes the object instance that requires serialization, and should return a primitive representation. Typically this means returning a structure of built-in Python datatypes. The exact types that can be handled will depend on the render classes you have configured for your API.
 
-May be overridden in order modify the representation style. For example:
+May be overridden in order to modify the representation style. For example:
 
     def to_representation(self, instance):
         """Convert `username` to lowercase."""

docs/api-guide/validators.md

@@ -218,7 +218,7 @@ in the `.validate()` method, or else in the view.
 For example:
 
     class BillingRecordSerializer(serializers.ModelSerializer):
-        def validate(self, data):
+        def validate(self, attrs):
             # Apply custom validation either here, or in the view.
 
         class Meta:

docs/community/3.10-announcement.md

@@ -84,7 +84,7 @@ urlpatterns = [
 
 ### Customization
 
-For customizations that you want to apply across the the entire API, you can subclass `rest_framework.schemas.openapi.SchemaGenerator` and provide it as an argument
+For customizations that you want to apply across the entire API, you can subclass `rest_framework.schemas.openapi.SchemaGenerator` and provide it as an argument
 to the `generateschema` command or `get_schema_view()` helper function.
 
 For specific per-view customizations, you can subclass `AutoSchema`,

docs/community/project-management.md

@@ -195,7 +195,6 @@ If `@tomchristie` ceases to participate in the project then `@j4mie` has respons
 
 The following issues still need to be addressed:
 
-* [Consider moving the repo into a proper GitHub organization][github-org].
 * Ensure `@jamie` has back-up access to the `django-rest-framework.org` domain setup and admin.
 * Document ownership of the [live example][sandbox] API.
 * Document ownership of the [mailing list][mailing-list] and IRC channel.
@@ -206,6 +205,5 @@ The following issues still need to be addressed:
 [transifex-project]: https://www.transifex.com/projects/p/django-rest-framework/
 [transifex-client]: https://pypi.org/project/transifex-client/
 [translation-memory]: http://docs.transifex.com/guides/tm#let-tm-automatically-populate-translations
-[github-org]: https://github.com/encode/django-rest-framework/issues/2162
 [sandbox]: https://restframework.herokuapp.com/
 [mailing-list]: https://groups.google.com/forum/#!forum/django-rest-framework

docs/community/third-party-packages.md

@@ -211,6 +211,8 @@ To submit new content, [open an issue][drf-create-issue] or [create a pull reque
 * [djangorestframework-queryfields][djangorestframework-queryfields] - Serializer mixin allowing clients to control which fields will be sent in the API response.
 * [drf-flex-fields][drf-flex-fields] - Serializer providing dynamic field expansion and sparse field sets via URL parameters.
 * [drf-action-serializer][drf-action-serializer] - Serializer providing per-action fields config for use with ViewSets to prevent having to write multiple serializers.
+* [djangorestframework-dataclasses][djangorestframework-dataclasses] - Serializer providing automatic field generation for Python dataclasses, like the built-in ModelSerializer does for models.
+* [django-restql][django-restql] - Turn your REST API into a GraphQL like API(It allows clients to control which fields will be sent in a response, uses GraphQL like syntax, supports read and write on both flat and nested fields).
 
 ### Serializer fields
 
@@ -348,5 +350,7 @@ To submit new content, [open an issue][drf-create-issue] or [create a pull reque
 [drf-access-policy]: https://github.com/rsinger86/drf-access-policy
 [drf-flex-fields]: https://github.com/rsinger86/drf-flex-fields
 [drf-action-serializer]: https://github.com/gregschmit/drf-action-serializer
+[djangorestframework-dataclasses]: https://github.com/oxan/djangorestframework-dataclasses
+[django-restql]: https://github.com/yezyilomo/django-restql
 [djangorestframework-mvt]: https://github.com/corteva/djangorestframework-mvt
 [django-rest-framework-guardian]: https://github.com/rpkilby/django-rest-framework-guardian

docs/index.md

@@ -73,10 +73,11 @@ continued development by **[signing up for a paid plan][funding]**.
     <li><a href="https://cadre.com" style="background-image: url(https://fund-rest-framework.s3.amazonaws.com/cadre.png)">Cadre</a></li>
     <li><a href="https://hubs.ly/H0f30Lf0" style="background-image: url(https://fund-rest-framework.s3.amazonaws.com/kloudless-plus-text.png)">Kloudless</a></li>
     <li><a href="https://lightsonsoftware.com" style="background-image: url(https://fund-rest-framework.s3.amazonaws.com/lightson-dark.png)">Lights On Software</a></li>
+    <li><a href="https://retool.com/?utm_source=djangorest&utm_medium=sponsorship" style="background-image: url(https://fund-rest-framework.s3.amazonaws.com/retool-sidebar.png)">Retool</a></li>
 </ul>
 <div style="clear: both; padding-bottom: 20px;"></div>
 
-*Many thanks to all our [wonderful sponsors][sponsors], and in particular to our premium backers, [Sentry](https://getsentry.com/welcome/), [Stream](https://getstream.io/?utm_source=drf&utm_medium=banner&utm_campaign=drf), [ESG](https://software.esg-usa.com/), [Rollbar](https://rollbar.com/?utm_source=django&utm_medium=sponsorship&utm_campaign=freetrial), [Cadre](https://cadre.com), [Kloudless](https://hubs.ly/H0f30Lf0), and [Lights On Software](https://lightsonsoftware.com).*
+*Many thanks to all our [wonderful sponsors][sponsors], and in particular to our premium backers, [Sentry](https://getsentry.com/welcome/), [Stream](https://getstream.io/?utm_source=drf&utm_medium=banner&utm_campaign=drf), [ESG](https://software.esg-usa.com/), [Rollbar](https://rollbar.com/?utm_source=django&utm_medium=sponsorship&utm_campaign=freetrial), [Cadre](https://cadre.com), [Kloudless](https://hubs.ly/H0f30Lf0), [Lights On Software](https://lightsonsoftware.com), and [Retool](https://retool.com/?utm_source=djangorest&utm_medium=sponsorship).*
 
 ---
 

docs/topics/documenting-your-api.md

@@ -74,7 +74,7 @@ See the [Swagger UI documentation][swagger-ui] for advanced usage.
 ### A minimal example with ReDoc.
 
 Assuming you've followed the example from the schemas documentation for routing
-a dynamic `SchemaView`, a minimal Django template for using Swagger UI might be
+a dynamic `SchemaView`, a minimal Django template for using ReDoc might be
 this:
 
 ```html
@@ -221,7 +221,7 @@ If the python `Markdown` library is installed, then [markdown syntax][markdown]
         [ref]: http://example.com/activating-accounts
         """
 
-Note that when using viewsets the basic docstring is used for all generated views.  To provide descriptions for each view, such as for the the list and retrieve views, use docstring sections as described in [Schemas as documentation: Examples][schemas-examples].
+Note that when using viewsets the basic docstring is used for all generated views.  To provide descriptions for each view, such as for the list and retrieve views, use docstring sections as described in [Schemas as documentation: Examples][schemas-examples].
 
 #### The `OPTIONS` method
 

requirements/requirements-codestyle.txt

@@ -1,7 +1,7 @@
 # PEP8 code linting, which we run on all commits.
-flake8==3.5.0
+flake8==3.7.8
-flake8-tidy-imports==1.1.0
+flake8-tidy-imports==3.0.0
-pycodestyle==2.3.1
+pycodestyle==2.5.0
 
 # Sort and lint imports
-isort==4.3.3
+isort==4.3.21

rest_framework/fields.py

@@ -1062,9 +1062,7 @@ class DecimalField(Field):
         except decimal.DecimalException:
             self.fail('invalid')
 
-        # Check for NaN. It is the only value that isn't equal to itself,
+        if value.is_nan():
-        # so we can use this to identify NaN values.
-        if value != value:
             self.fail('invalid')
 
         # Check for infinity and negative infinity.
@@ -1764,8 +1762,8 @@ class JSONField(Field):
             # When HTML form input is used, mark up the input
             # as being a JSON string, rather than a JSON primitive.
             class JSONString(str):
-                def __new__(self, value):
+                def __new__(cls, value):
-                    ret = str.__new__(self, value)
+                    ret = str.__new__(cls, value)
                     ret.is_json_string = True
                     return ret
             return JSONString(dictionary[self.field_name])

rest_framework/relations.py

@@ -46,8 +46,8 @@ class Hyperlink(str):
     We use this for hyperlinked URLs that may render as a named link
     in some contexts, or render as a plain URL in others.
     """
-    def __new__(self, url, obj):
+    def __new__(cls, url, obj):
-        ret = str.__new__(self, url)
+        ret = str.__new__(cls, url)
         ret.obj = obj
         return ret
 

rest_framework/routers.py

@@ -38,9 +38,7 @@ def escape_curly_brackets(url_path):
     """
     Double brackets in regex of url_path for escape string formatting
     """
-    if ('{' and '}') in url_path:
+    return url_path.replace('{', '{{').replace('}', '}}')
-        url_path = url_path.replace('{', '{{').replace('}', '}}')
-    return url_path
 
 
 def flatten(list_of_lists):

rest_framework/schemas/__init__.py

@@ -23,8 +23,8 @@ Other access should target the submodules directly
 from rest_framework.settings import api_settings
 
 from . import coreapi, openapi
-from .inspectors import DefaultSchema  # noqa
 from .coreapi import AutoSchema, ManualSchema, SchemaGenerator  # noqa
+from .inspectors import DefaultSchema  # noqa
 
 
 def get_schema_view(

rest_framework/schemas/openapi.py

@@ -387,7 +387,7 @@ class AutoSchema(ViewInspector):
         result = {
             'properties': properties
         }
-        if len(required) > 0:
+        if required:
             result['required'] = required
 
         return result
@@ -463,7 +463,7 @@ class AutoSchema(ViewInspector):
         content = self._map_serializer(serializer)
         # No required fields for PATCH
         if method == 'PATCH':
-            del content['required']
+            content.pop('required', None)
         # No read_only fields for request.
         for name, schema in content['properties'].copy().items():
             if 'readOnly' in schema:

rest_framework/serializers.py

@@ -16,12 +16,11 @@ import traceback
 from collections import OrderedDict
 from collections.abc import Mapping
 
-from django.core.exceptions import ImproperlyConfigured
+from django.core.exceptions import FieldDoesNotExist, ImproperlyConfigured
 from django.core.exceptions import ValidationError as DjangoValidationError
 from django.db import models
 from django.db.models import DurationField as ModelDurationField
 from django.db.models.fields import Field as DjangoModelField
-from django.db.models.fields import FieldDoesNotExist
 from django.utils import timezone
 from django.utils.functional import cached_property
 from django.utils.translation import gettext_lazy as _
@@ -791,6 +790,8 @@ def raise_errors_on_nested_writes(method_name, serializer, validated_data):
     * Silently ignore the nested part of the update.
     * Automatically create a profile instance.
     """
+    ModelClass = serializer.Meta.model
+    model_field_info = model_meta.get_field_info(ModelClass)
 
     # Ensure we don't have a writable nested field. For example:
     #
@@ -800,6 +801,7 @@ def raise_errors_on_nested_writes(method_name, serializer, validated_data):
     assert not any(
         isinstance(field, BaseSerializer) and
         (field.source in validated_data) and
+        (field.source in model_field_info.relations) and
         isinstance(validated_data[field.source], (list, dict))
         for field in serializer._writable_fields
     ), (
@@ -818,9 +820,19 @@ def raise_errors_on_nested_writes(method_name, serializer, validated_data):
     # class UserSerializer(ModelSerializer):
     #     ...
     #     address = serializer.CharField('profile.address')
+    #
+    # Though, non-relational fields (e.g., JSONField) are acceptable. For example:
+    #
+    # class NonRelationalPersonModel(models.Model):
+    #     profile = JSONField()
+    #
+    # class UserSerializer(ModelSerializer):
+    #     ...
+    #     address = serializer.CharField('profile.address')
     assert not any(
         len(field.source_attrs) > 1 and
         (field.source_attrs[0] in validated_data) and
+        (field.source_attrs[0] in model_field_info.relations) and
         isinstance(validated_data[field.source_attrs[0]], (list, dict))
         for field in serializer._writable_fields
     ), (

setup.cfg

@@ -6,7 +6,7 @@ addopts=--tb=short --strict -ra
 testspath = tests
 
 [flake8]
-ignore = E501
+ignore = E501,W504
 banned-modules = json = use from rest_framework.utils import json!
 
 [isort]

setup.py

@@ -101,6 +101,7 @@ setup(
         'Programming Language :: Python :: 3.5',
         'Programming Language :: Python :: 3.6',
         'Programming Language :: Python :: 3.7',
+        'Programming Language :: Python :: 3.8',
         'Programming Language :: Python :: 3 :: Only',
         'Topic :: Internet :: WWW/HTTP',
     ],

tests/schemas/test_coreapi.py

@@ -24,8 +24,8 @@ from rest_framework.utils import formatting
 from rest_framework.views import APIView
 from rest_framework.viewsets import GenericViewSet, ModelViewSet
 
-from . import views
 from ..models import BasicModel, ForeignKeySource, ManyToManySource
+from . import views
 
 factory = APIRequestFactory()
 

tests/schemas/test_openapi.py

@@ -169,6 +169,31 @@ class TestOperationIntrospection(TestCase):
         for response in inspector._get_responses(path, method).values():
             assert 'required' not in response['content']['application/json']['schema']
 
+    def test_empty_required_with_patch_method(self):
+        path = '/'
+        method = 'PATCH'
+
+        class Serializer(serializers.Serializer):
+            read_only = serializers.CharField(read_only=True)
+            write_only = serializers.CharField(write_only=True, required=False)
+
+        class View(generics.GenericAPIView):
+            serializer_class = Serializer
+
+        view = create_view(
+            View,
+            method,
+            create_request(path)
+        )
+        inspector = AutoSchema()
+        inspector.view = view
+
+        request_body = inspector._get_request_body(path, method)
+        # there should be no empty 'required' property, see #6834
+        assert 'required' not in request_body['content']['application/json']['schema']
+        for response in inspector._get_responses(path, method).values():
+            assert 'required' not in response['content']['application/json']['schema']
+
     def test_response_body_generation(self):
         path = '/'
         method = 'POST'

tests/test_fields.py

@@ -1080,6 +1080,7 @@ class TestDecimalField(FieldValues):
     invalid_inputs = (
         ('abc', ["A valid number is required."]),
         (Decimal('Nan'), ["A valid number is required."]),
+        (Decimal('Snan'), ["A valid number is required."]),
         (Decimal('Inf'), ["A valid number is required."]),
         ('12.345', ["Ensure that there are no more than 3 digits in total."]),
         (200000000000.0, ["Ensure that there are no more than 3 digits in total."]),

tests/test_relations.py

@@ -251,7 +251,7 @@ class TestHyperlinkedIdentityField(APISimpleTestCase):
     def test_improperly_configured(self):
         """
         If a matching view cannot be reversed with the given instance,
-        the the user has misconfigured something, as the URL conf and the
+        the user has misconfigured something, as the URL conf and the
         hyperlinked field do not match.
         """
         self.field.reverse = fail_reverse

tests/test_serializer_nested.py

@@ -4,6 +4,8 @@ from django.http import QueryDict
 from django.test import TestCase
 
 from rest_framework import serializers
+from rest_framework.compat import postgres_fields
+from rest_framework.serializers import raise_errors_on_nested_writes
 
 
 class TestNestedSerializer:
@@ -302,3 +304,50 @@ class TestNestedWriteErrors(TestCase):
             'serializer `tests.test_serializer_nested.DottedAddressSerializer`, '
             'or set `read_only=True` on dotted-source serializer fields.'
         )
+
+
+if postgres_fields:
+    class NonRelationalPersonModel(models.Model):
+        """Model declaring a postgres JSONField"""
+        data = postgres_fields.JSONField()
+
+
+@pytest.mark.skipif(not postgres_fields, reason='psycopg2 is not installed')
+class TestNestedNonRelationalFieldWrite:
+    """
+    Test that raise_errors_on_nested_writes does not raise `AssertionError` when the
+    model field is not a relation.
+    """
+
+    def test_nested_serializer_create_and_update(self):
+
+        class NonRelationalPersonDataSerializer(serializers.Serializer):
+            occupation = serializers.CharField()
+
+        class NonRelationalPersonSerializer(serializers.ModelSerializer):
+            data = NonRelationalPersonDataSerializer()
+
+            class Meta:
+                model = NonRelationalPersonModel
+                fields = ['data']
+
+        serializer = NonRelationalPersonSerializer(data={'data': {'occupation': 'developer'}})
+        assert serializer.is_valid()
+        assert serializer.validated_data == {'data': {'occupation': 'developer'}}
+        raise_errors_on_nested_writes('create', serializer, serializer.validated_data)
+        raise_errors_on_nested_writes('update', serializer, serializer.validated_data)
+
+    def test_dotted_source_field_create_and_update(self):
+
+        class DottedNonRelationalPersonSerializer(serializers.ModelSerializer):
+            occupation = serializers.CharField(source='data.occupation')
+
+            class Meta:
+                model = NonRelationalPersonModel
+                fields = ['occupation']
+
+        serializer = DottedNonRelationalPersonSerializer(data={'occupation': 'developer'})
+        assert serializer.is_valid()
+        assert serializer.validated_data == {'data': {'occupation': 'developer'}}
+        raise_errors_on_nested_writes('create', serializer, serializer.validated_data)
+        raise_errors_on_nested_writes('update', serializer, serializer.validated_data)

tox.ini

@@ -4,7 +4,7 @@ envlist =
        {py35,py36,py37}-django20,
        {py35,py36,py37}-django21
        {py35,py36,py37}-django22
-       {py36,py37}-djangomaster,
+       {py36,py37,py38}-djangomaster,
        base,dist,lint,docs,
 
 [travis:env]
@@ -44,14 +44,12 @@ deps =
         -rrequirements/requirements-optionals.txt
 
 [testenv:lint]
-basepython = python3.7
 commands = ./runtests.py --lintonly
 deps =
         -rrequirements/requirements-codestyle.txt
         -rrequirements/requirements-testing.txt
 
 [testenv:docs]
-basepython = python3.7
 skip_install = true
 commands = mkdocs build
 deps =