Merge remote-tracking branch 'origin/master' into release/3.2.4

docs/api-guide/authentication.md

@@ -360,6 +360,10 @@ HTTP Signature (currently a [IETF draft][http-signature-ietf-draft]) provides a
 
 [Django-rest-auth][django-rest-auth] library provides a set of REST API endpoints for registration, authentication (including social media authentication), password reset, retrieve and update user details, etc. By having these API endpoints, your client apps such as AngularJS, iOS, Android, and others can communicate to your Django backend site independently via REST APIs for user management.
 
+## django-rest-knox
+
+[Django-rest-knox][django-rest-knox] library provides models and views to handle token based authentication in a more secure and extensible way than the built-in TokenAuthentication scheme - with Single Page Applications and Mobile clients in mind. It provides per-client tokens, and views to generate them when provided some other authentication (usually basic authentication), to delete the token (providing a server enforced logout) and to delete all tokens (logs out all clients that a user is logged into).
+
 [cite]: http://jacobian.org/writing/rest-worst-practices/
 [http401]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2
 [http403]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4
@@ -400,3 +404,4 @@ HTTP Signature (currently a [IETF draft][http-signature-ietf-draft]) provides a
 [mac]: http://tools.ietf.org/html/draft-hammer-oauth-v2-mac-token-05
 [djoser]: https://github.com/sunscrapers/djoser
 [django-rest-auth]: https://github.com/Tivix/django-rest-auth
+[django-rest-knox]: https://github.com/James1345/django-rest-knox

docs/api-guide/fields.md

@@ -51,7 +51,7 @@ Defaults to `False`
 
 If set, this gives the default value that will be used for the field if no input value is supplied.  If not set the default behavior is to not populate the attribute at all.
 
-May be set to a function or other callable, in which case the value will be evaluated each time it is used.
+May be set to a function or other callable, in which case the value will be evaluated each time it is used. When called, it will receive no arguments. If the callable has a `set_context` method, that will be called each time before getting the value with the field instance as only argument. This works the same way as for [validators](validators.md#using-set_context).
 
 Note that setting a `default` value implies that the field is not required. Including both the `default` and `required` keyword arguments is invalid and will raise an error.
 

docs/api-guide/relations.md

@@ -255,7 +255,7 @@ For example, the following serializer:
     class TrackSerializer(serializers.ModelSerializer):
         class Meta:
             model = Track
-            fields = ('order', 'title')
+            fields = ('order', 'title', 'duration')
 
     class AlbumSerializer(serializers.ModelSerializer):
         tracks = TrackSerializer(many=True, read_only=True)
@@ -293,7 +293,7 @@ Be default nested serializers are read-only. If you want to to support write-ope
     class TrackSerializer(serializers.ModelSerializer):
         class Meta:
             model = Track
-            fields = ('order', 'title')
+            fields = ('order', 'title', 'duration')
 
     class AlbumSerializer(serializers.ModelSerializer):
         tracks = TrackSerializer(many=True)
@@ -405,13 +405,15 @@ In this case we'd need to override `HyperlinkedRelatedField` to get the behavior
         def get_url(self, obj, view_name, request, format):
             url_kwargs = {
                 'organization_slug': obj.organization.slug,
-                'customer_pk': obj.pk
            }
+                'customer_pk': obj.pk
+            }
             return reverse(view_name, url_kwargs, request=request, format=format)
 
         def get_object(self, view_name, view_args, view_kwargs):
             lookup_kwargs = {
                'organization__slug': view_kwargs['organization_slug'],
-               'pk': view_kwargs['customer_pk']
            }
+               'pk': view_kwargs['customer_pk']
+            }
             return self.get_queryset().get(**lookup_kwargs)
 
 Note that if you wanted to use this style together with the generic views then you'd also need to override `.get_object` on the view in order to get the correct lookup behavior.

docs/api-guide/serializers.md

@@ -189,6 +189,12 @@ Your `validate_<field_name>` methods should return the validated value or raise
                 raise serializers.ValidationError("Blog post is not about Django")
             return value
 
+---
+
+**Note:** If your `<field_name>` is declared on your serializer with the parameter `required=False` then this validation step will not take place if the field is not included.
+
+---
+
 #### Object-level validation
 
 To do any other validation that requires access to multiple fields, add a method called `.validate()` to your `Serializer` subclass.  This method takes a single argument, which is a dictionary of field values.  It should raise a `ValidationError` if necessary, or just return the validated values.  For example:
@@ -1022,6 +1028,13 @@ A new interface for controlling this behavior is currently planned for REST fram
 
 The following third party packages are also available.
 
+## Django REST marshmallow
+
+The [django-rest-marshmallow][django-rest-marshmallow] package provides an alternative implementation for serializers, using the python [marshmallow][marshmallow] library. It exposes the same API as the REST framework serializers, and can be used as a drop-in replacement in some use-cases.
+
+## Serpy
+The [serpy][serpy] package is an alternative implementation for serializers that is built for speed. [Serpy][serpy] serializes complex datatypes to simple native types. The native types can be easily converted to JSON or any other format needed.
+
 ## MongoengineModelSerializer
 
 The [django-rest-framework-mongoengine][mongoengine] package provides a `MongoEngineModelSerializer` serializer class that supports using MongoDB as the storage layer for Django REST framework.
@@ -1038,6 +1051,9 @@ The [django-rest-framework-hstore][django-rest-framework-hstore] package provide
 [relations]: relations.md
 [model-managers]: https://docs.djangoproject.com/en/dev/topics/db/managers/
 [encapsulation-blogpost]: http://www.dabapps.com/blog/django-models-and-encapsulation/
+[django-rest-marshmallow]: http://tomchristie.github.io/django-rest-marshmallow/
+[marshmallow]: https://marshmallow.readthedocs.org/en/latest/
+[serpy]: https://github.com/clarkduvall/serpy
 [mongoengine]: https://github.com/umutbozkurt/django-rest-framework-mongoengine
 [django-rest-framework-gis]: https://github.com/djangonauts/django-rest-framework-gis
 [django-rest-framework-hstore]: https://github.com/djangonauts/django-rest-framework-hstore

docs/api-guide/throttling.md

@@ -148,7 +148,7 @@ For example, given the following views...
         throttle_scope = 'contacts'
         ...
 
-    class ContactDetailView(ApiView):
+    class ContactDetailView(APIView):
         throttle_scope = 'contacts'
         ...
 

docs/tutorial/1-serialization.md

@@ -181,7 +181,7 @@ We can also serialize querysets instead of model instances.  To do so we simply
 
     serializer = SnippetSerializer(Snippet.objects.all(), many=True)
     serializer.data
-    # [{'pk': 1, 'title': u'', 'code': u'foo = "bar"\n', 'linenos': False, 'language': u'python', 'style': u'friendly'}, {'pk': 2, 'title': u'', 'code': u'print "hello, world"\n', 'linenos': False, 'language': u'python', 'style': u'friendly'}]
+    # [OrderedDict([('pk', 1), ('title', u''), ('code', u'foo = "bar"\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('pk', 2), ('title', u''), ('code', u'print "hello, world"\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('pk', 3), ('title', u''), ('code', u'print "hello, world"'), ('linenos', False), ('language', 'python'), ('style', 'friendly')])]
 
 ## Using ModelSerializers
 

requirements/requirements-packaging.txt

@@ -5,4 +5,4 @@ wheel==0.24.0
 twine==1.4.0
 
 # Transifex client for managing translation resources.
-transifex-client==0.10
+transifex-client==0.11b3

rest_framework/fields.py

@@ -608,10 +608,13 @@ class BooleanField(Field):
         super(BooleanField, self).__init__(**kwargs)
 
     def to_internal_value(self, data):
-        if data in self.TRUE_VALUES:
+        try:
-            return True
+            if data in self.TRUE_VALUES:
-        elif data in self.FALSE_VALUES:
+                return True
-            return False
+            elif data in self.FALSE_VALUES:
+                return False
+        except TypeError:  # Input is an unhashable type
+            pass
         self.fail('invalid', input=data)
 
     def to_representation(self, value):
@@ -1262,14 +1265,13 @@ class MultipleChoiceField(ChoiceField):
         super(MultipleChoiceField, self).__init__(*args, **kwargs)
 
     def get_value(self, dictionary):
+        if self.field_name not in dictionary:
+            if getattr(self.root, 'partial', False):
+                return empty
         # We override the default field access in order to support
         # lists in HTML forms.
         if html.is_html_input(dictionary):
-            ret = dictionary.getlist(self.field_name)
+            return dictionary.getlist(self.field_name)
-            if getattr(self.root, 'partial', False) and not ret:
-                ret = empty
-            return ret
-
         return dictionary.get(self.field_name, empty)
 
     def to_internal_value(self, data):
@@ -1416,6 +1418,9 @@ class ListField(Field):
         self.child.bind(field_name='', parent=self)
 
     def get_value(self, dictionary):
+        if self.field_name not in dictionary:
+            if getattr(self.root, 'partial', False):
+                return empty
         # We override the default field access in order to support
         # lists in HTML forms.
         if html.is_html_input(dictionary):

rest_framework/serializers.py

@@ -113,8 +113,13 @@ class BaseSerializer(Field):
             kwargs['child'] = cls()
             return CustomListSerializer(*args, **kwargs)
         """
+        allow_empty = kwargs.pop('allow_empty', None)
         child_serializer = cls(*args, **kwargs)
-        list_kwargs = {'child': child_serializer}
+        list_kwargs = {
+            'child': child_serializer,
+        }
+        if allow_empty is not None:
+            list_kwargs['allow_empty'] = allow_empty
         list_kwargs.update(dict([
             (key, value) for key, value in kwargs.items()
             if key in LIST_SERIALIZER_KWARGS

tests/test_fields.py

@@ -466,6 +466,18 @@ class TestBooleanField(FieldValues):
     }
     field = serializers.BooleanField()
 
+    def test_disallow_unhashable_collection_types(self):
+        inputs = (
+            [],
+            {},
+        )
+        field = serializers.BooleanField()
+        for input_value in inputs:
+            with pytest.raises(serializers.ValidationError) as exc_info:
+                field.run_validation(input_value)
+            expected = ['"{0}" is not a valid boolean.'.format(input_value)]
+            assert exc_info.value.detail == expected
+
 
 class TestNullBooleanField(FieldValues):
     """

tests/test_serializer_lists.py

@@ -289,3 +289,32 @@ class TestListSerializerClass:
         serializer = TestSerializer(data=[], many=True)
         assert not serializer.is_valid()
         assert serializer.errors == {'non_field_errors': ['Non field error']}
+
+
+class TestSerializerPartialUsage:
+    """
+    When not submitting key for list fields or multiple choice, partial
+    serialization should result in an empty state (key not there), not
+    an empty list.
+
+    Regression test for Github issue #2761.
+    """
+    def test_partial_listfield(self):
+        class ListSerializer(serializers.Serializer):
+            listdata = serializers.ListField()
+        serializer = ListSerializer(data=MultiValueDict(), partial=True)
+        result = serializer.to_internal_value(data={})
+        assert "listdata" not in result
+        assert serializer.is_valid()
+        assert serializer.validated_data == {}
+        assert serializer.errors == {}
+
+    def test_partial_multiplechoice(self):
+        class MultipleChoiceSerializer(serializers.Serializer):
+            multiplechoice = serializers.MultipleChoiceField(choices=[1, 2, 3])
+        serializer = MultipleChoiceSerializer(data=MultiValueDict(), partial=True)
+        result = serializer.to_internal_value(data={})
+        assert "multiplechoice" not in result
+        assert serializer.is_valid()
+        assert serializer.validated_data == {}
+        assert serializer.errors == {}

tests/test_serializer_nested.py

@@ -79,17 +79,23 @@ class TestNestedSerializerWithMany:
         class TestSerializer(serializers.Serializer):
             allow_null = NestedSerializer(many=True, allow_null=True)
             not_allow_null = NestedSerializer(many=True)
+            allow_empty = NestedSerializer(many=True, allow_empty=True)
+            not_allow_empty = NestedSerializer(many=True, allow_empty=False)
 
         self.Serializer = TestSerializer
 
     def test_null_allowed_if_allow_null_is_set(self):
         input_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': '2'}, {'example': '3'}]
+            'not_allow_null': [{'example': '2'}, {'example': '3'}],
+            'allow_empty': [{'example': '2'}],
+            'not_allow_empty': [{'example': '2'}],
         }
         expected_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': 2}, {'example': 3}]
+            'not_allow_null': [{'example': 2}, {'example': 3}],
+            'allow_empty': [{'example': 2}],
+            'not_allow_empty': [{'example': 2}],
         }
         serializer = self.Serializer(data=input_data)
 
@@ -99,7 +105,9 @@ class TestNestedSerializerWithMany:
     def test_null_is_not_allowed_if_allow_null_is_not_set(self):
         input_data = {
             'allow_null': None,
-            'not_allow_null': None
+            'not_allow_null': None,
+            'allow_empty': [{'example': '2'}],
+            'not_allow_empty': [{'example': '2'}],
         }
         serializer = self.Serializer(data=input_data)
 
@@ -118,10 +126,44 @@ class TestNestedSerializerWithMany:
 
         input_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': 2}]
+            'not_allow_null': [{'example': 2}],
+            'allow_empty': [{'example': 2}],
+            'not_allow_empty': [{'example': 2}],
         }
         serializer = TestSerializer(data=input_data)
 
         assert serializer.is_valid()
         assert serializer.validated_data == input_data
         assert TestSerializer.validation_was_run
+
+    def test_empty_allowed_if_allow_empty_is_set(self):
+        input_data = {
+            'allow_null': [{'example': '2'}],
+            'not_allow_null': [{'example': '2'}],
+            'allow_empty': [],
+            'not_allow_empty': [{'example': '2'}],
+        }
+        expected_data = {
+            'allow_null': [{'example': 2}],
+            'not_allow_null': [{'example': 2}],
+            'allow_empty': [],
+            'not_allow_empty': [{'example': 2}],
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert serializer.is_valid(), serializer.errors
+        assert serializer.validated_data == expected_data
+
+    def test_empty_not_allowed_if_allow_empty_is_set_to_false(self):
+        input_data = {
+            'allow_null': [{'example': '2'}],
+            'not_allow_null': [{'example': '2'}],
+            'allow_empty': [],
+            'not_allow_empty': [],
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert not serializer.is_valid()
+
+        expected_errors = {'not_allow_empty': {'non_field_errors': [serializers.ListSerializer.default_error_messages['empty']]}}
+        assert serializer.errors == expected_errors