Merge branch 'master' into version-3.3

README.md

@@ -157,7 +157,6 @@ If you believe you’ve found something in Django REST framework which has secur
 
 Send a description of the issue via email to [rest-framework-security@googlegroups.com][security-mail].  The project maintainers will then work with you to resolve any issues where required, prior to any public disclosure.
 
-
 [build-status-image]: https://secure.travis-ci.org/tomchristie/django-rest-framework.svg?branch=master
 [travis]: http://travis-ci.org/tomchristie/django-rest-framework?branch=master
 [coverage-status-image]: https://img.shields.io/codecov/c/github/tomchristie/django-rest-framework/master.svg

docs/api-guide/authentication.md

@@ -360,6 +360,10 @@ HTTP Signature (currently a [IETF draft][http-signature-ietf-draft]) provides a
 
 [Django-rest-auth][django-rest-auth] library provides a set of REST API endpoints for registration, authentication (including social media authentication), password reset, retrieve and update user details, etc. By having these API endpoints, your client apps such as AngularJS, iOS, Android, and others can communicate to your Django backend site independently via REST APIs for user management.
 
+## django-rest-knox
+
+[Django-rest-knox][django-rest-knox] library provides models and views to handle token based authentication in a more secure and extensible way than the built-in TokenAuthentication scheme - with Single Page Applications and Mobile clients in mind. It provides per-client tokens, and views to generate them when provided some other authentication (usually basic authentication), to delete the token (providing a server enforced logout) and to delete all tokens (logs out all clients that a user is logged into).
+
 [cite]: http://jacobian.org/writing/rest-worst-practices/
 [http401]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.2
 [http403]: http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.4
@@ -400,3 +404,4 @@ HTTP Signature (currently a [IETF draft][http-signature-ietf-draft]) provides a
 [mac]: http://tools.ietf.org/html/draft-hammer-oauth-v2-mac-token-05
 [djoser]: https://github.com/sunscrapers/djoser
 [django-rest-auth]: https://github.com/Tivix/django-rest-auth
+[django-rest-knox]: https://github.com/James1345/django-rest-knox

docs/api-guide/fields.md

@@ -51,7 +51,7 @@ Defaults to `False`
 
 If set, this gives the default value that will be used for the field if no input value is supplied.  If not set the default behavior is to not populate the attribute at all.
 
-May be set to a function or other callable, in which case the value will be evaluated each time it is used.
+May be set to a function or other callable, in which case the value will be evaluated each time it is used. When called, it will receive no arguments. If the callable has a `set_context` method, that will be called each time before getting the value with the field instance as only argument. This works the same way as for [validators](validators.md#using-set_context).
 
 Note that setting a `default` value implies that the field is not required. Including both the `default` and `required` keyword arguments is invalid and will raise an error.
 

docs/api-guide/filtering.md

@@ -331,8 +331,6 @@ The `ordering` attribute may be either a string or a list/tuple of strings.
 
 The `DjangoObjectPermissionsFilter` is intended to be used together with the [`django-guardian`][guardian] package, with custom `'view'` permissions added.  The filter will ensure that querysets only returns objects for which the user has the appropriate view permission.
 
-This filter class must be used with views that provide either a `queryset` or a `model` attribute.
-
 If you're using `DjangoObjectPermissionsFilter`, you'll probably also want to add an appropriate object permissions class, to ensure that users can only operate on instances if they have the appropriate object permissions.  The easiest way to do this is to subclass `DjangoObjectPermissions` and add `'view'` permissions to the `perms_map` attribute.
 
 A complete example using both `DjangoObjectPermissionsFilter` and `DjangoObjectPermissions` might look something like this.
@@ -403,6 +401,10 @@ The [django-rest-framework-filters package][django-rest-framework-filters] works
 
 The [djangorestframework-word-filter][django-rest-framework-word-search-filter] developed as alternative to `filters.SearchFilter` which will search full word in text, or exact match.
 
+## Django URL Filter
+
+[django-url-filter][django-url-filter] provides a safe way to filter data via human-friendly URLs. It works very similar to DRF serializers and fields in a sense that they can be nested except they are called filtersets and filters. That provides easy way to filter related data. Also this library is generic-purpose so it can be used to filter other sources of data and not only Django `QuerySet`s.
+
 [cite]: https://docs.djangoproject.com/en/dev/topics/db/queries/#retrieving-specific-objects-with-filters
 [django-filter]: https://github.com/alex/django-filter
 [django-filter-docs]: https://django-filter.readthedocs.org/en/latest/index.html
@@ -413,3 +415,4 @@ The [djangorestframework-word-filter][django-rest-framework-word-search-filter]
 [search-django-admin]: https://docs.djangoproject.com/en/dev/ref/contrib/admin/#django.contrib.admin.ModelAdmin.search_fields
 [django-rest-framework-filters]: https://github.com/philipn/django-rest-framework-filters
 [django-rest-framework-word-search-filter]: https://github.com/trollknurr/django-rest-framework-word-search-filter
+[django-url-filter]: https://github.com/miki725/django-url-filter

docs/api-guide/relations.md

@@ -501,7 +501,7 @@ For example, given the following model for a tag, which has a generic relationsh
         tagged_object = GenericForeignKey('content_type', 'object_id')
 
         def __unicode__(self):
-            return self.tag
+            return self.tag_name
 
 And the following two models, which may be have associated tags:
 

docs/api-guide/serializers.md

@@ -189,6 +189,12 @@ Your `validate_<field_name>` methods should return the validated value or raise
                 raise serializers.ValidationError("Blog post is not about Django")
             return value
 
+---
+
+**Note:** If your `<field_name>` is declared on your serializer with the parameter `required=False` then this validation step will not take place if the field is not included.
+
+---
+
 #### Object-level validation
 
 To do any other validation that requires access to multiple fields, add a method called `.validate()` to your `Serializer` subclass.  This method takes a single argument, which is a dictionary of field values.  It should raise a `ValidationError` if necessary, or just return the validated values.  For example:
@@ -1043,6 +1049,13 @@ A new interface for controlling this behavior is currently planned for REST fram
 
 The following third party packages are also available.
 
+## Django REST marshmallow
+
+The [django-rest-marshmallow][django-rest-marshmallow] package provides an alternative implementation for serializers, using the python [marshmallow][marshmallow] library. It exposes the same API as the REST framework serializers, and can be used as a drop-in replacement in some use-cases.
+
+## Serpy
+The [serpy][serpy] package is an alternative implementation for serializers that is built for speed. [Serpy][serpy] serializes complex datatypes to simple native types. The native types can be easily converted to JSON or any other format needed.
+
 ## MongoengineModelSerializer
 
 The [django-rest-framework-mongoengine][mongoengine] package provides a `MongoEngineModelSerializer` serializer class that supports using MongoDB as the storage layer for Django REST framework.
@@ -1059,6 +1072,9 @@ The [django-rest-framework-hstore][django-rest-framework-hstore] package provide
 [relations]: relations.md
 [model-managers]: https://docs.djangoproject.com/en/dev/topics/db/managers/
 [encapsulation-blogpost]: http://www.dabapps.com/blog/django-models-and-encapsulation/
+[django-rest-marshmallow]: http://tomchristie.github.io/django-rest-marshmallow/
+[marshmallow]: https://marshmallow.readthedocs.org/en/latest/
+[serpy]: https://github.com/clarkduvall/serpy
 [mongoengine]: https://github.com/umutbozkurt/django-rest-framework-mongoengine
 [django-rest-framework-gis]: https://github.com/djangonauts/django-rest-framework-gis
 [django-rest-framework-hstore]: https://github.com/djangonauts/django-rest-framework-hstore

docs/api-guide/testing.md

@@ -200,6 +200,7 @@ You can use any of REST framework's test case classes as you would for the regul
     from django.core.urlresolvers import reverse
     from rest_framework import status
     from rest_framework.test import APITestCase
+    from myproject.apps.core.models import Account
 
     class AccountTests(APITestCase):
         def test_create_account(self):
@@ -210,7 +211,8 @@ You can use any of REST framework's test case classes as you would for the regul
             data = {'name': 'DabApps'}
             response = self.client.post(url, data, format='json')
             self.assertEqual(response.status_code, status.HTTP_201_CREATED)
-            self.assertEqual(response.data, data)
+            self.assertEqual(Account.objects.count(), 1)
+            self.assertEqual(Account.objects.get().name, 'DabApps')
 
 ---
 

docs/api-guide/throttling.md

@@ -148,7 +148,7 @@ For example, given the following views...
         throttle_scope = 'contacts'
         ...
 
-    class ContactDetailView(ApiView):
+    class ContactDetailView(APIView):
         throttle_scope = 'contacts'
         ...
 

docs/topics/third-party-resources.md

@@ -233,6 +233,7 @@ To submit new content, [open an issue][drf-create-issue] or [create a pull reque
 ### Filtering
 
 * [djangorestframework-chain][djangorestframework-chain] - Allows arbitrary chaining of both relations and lookup filters.
+* [django-url-filter][django-url-filter] - Allows a safe way to filter data via human-friendly URLs. It is a generic library which is not tied to DRF but it provides easy integration with DRF.
 
 ### Misc
 
@@ -343,3 +344,4 @@ To submit new content, [open an issue][drf-create-issue] or [create a pull reque
 [drf-tracking]: https://github.com/aschn/drf-tracking
 [django-rest-framework-braces]: https://github.com/dealertrack/django-rest-framework-braces
 [dry-rest-permissions]: https://github.com/Helioscene/dry-rest-permissions
+[django-url-filter]: https://github.com/miki725/django-url-filter

docs/tutorial/1-serialization.md

@@ -181,7 +181,7 @@ We can also serialize querysets instead of model instances.  To do so we simply
 
     serializer = SnippetSerializer(Snippet.objects.all(), many=True)
     serializer.data
-    # [{'pk': 1, 'title': u'', 'code': u'foo = "bar"\n', 'linenos': False, 'language': u'python', 'style': u'friendly'}, {'pk': 2, 'title': u'', 'code': u'print "hello, world"\n', 'linenos': False, 'language': u'python', 'style': u'friendly'}]
+    # [OrderedDict([('pk', 1), ('title', u''), ('code', u'foo = "bar"\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('pk', 2), ('title', u''), ('code', u'print "hello, world"\n'), ('linenos', False), ('language', 'python'), ('style', 'friendly')]), OrderedDict([('pk', 3), ('title', u''), ('code', u'print "hello, world"'), ('linenos', False), ('language', 'python'), ('style', 'friendly')])]
 
 ## Using ModelSerializers
 

docs/tutorial/6-viewsets-and-routers.md

@@ -45,7 +45,7 @@ Next we're going to replace the `SnippetList`, `SnippetDetail` and `SnippetHighl
             return Response(snippet.highlighted)
 
         def perform_create(self, serializer):
-                serializer.save(owner=self.request.user)
+            serializer.save(owner=self.request.user)
 
 This time we've used the `ModelViewSet` class in order to get the complete set of default read and write operations.
 

requirements/requirements-packaging.txt

@@ -5,4 +5,4 @@ wheel==0.24.0
 twine==1.4.0
 
 # Transifex client for managing translation resources.
-transifex-client==0.10
+transifex-client==0.11b3

rest_framework/compat.py

@@ -120,12 +120,11 @@ else:
 # Django-guardian is optional. Import only if guardian is in INSTALLED_APPS
 # Fixes (#1712). We keep the try/except for the test suite.
 guardian = None
-if 'guardian' in settings.INSTALLED_APPS:
-    try:
-        import guardian
-        import guardian.shortcuts  # Fixes #1624
-    except ImportError:
-        pass
+try:
+    import guardian
+    import guardian.shortcuts  # Fixes #1624
+except ImportError:
+    pass
 
 
 def get_model_name(model_cls):

rest_framework/fields.py

@@ -608,10 +608,13 @@ class BooleanField(Field):
         super(BooleanField, self).__init__(**kwargs)
 
     def to_internal_value(self, data):
-        if data in self.TRUE_VALUES:
-            return True
-        elif data in self.FALSE_VALUES:
-            return False
+        try:
+            if data in self.TRUE_VALUES:
+                return True
+            elif data in self.FALSE_VALUES:
+                return False
+        except TypeError:  # Input is an unhashable type
+            pass
         self.fail('invalid', input=data)
 
     def to_representation(self, value):
@@ -1189,7 +1192,7 @@ class DurationField(Field):
     def to_internal_value(self, value):
         if isinstance(value, datetime.timedelta):
             return value
-        parsed = parse_duration(value)
+        parsed = parse_duration(six.text_type(value))
         if parsed is not None:
             return parsed
         self.fail('invalid', format='[DD] [HH:[MM:]]ss[.uuuuuu]')

rest_framework/serializers.py

@@ -117,8 +117,13 @@ class BaseSerializer(Field):
             kwargs['child'] = cls()
             return CustomListSerializer(*args, **kwargs)
         """
+        allow_empty = kwargs.pop('allow_empty', None)
         child_serializer = cls(*args, **kwargs)
-        list_kwargs = {'child': child_serializer}
+        list_kwargs = {
+            'child': child_serializer,
+        }
+        if allow_empty is not None:
+            list_kwargs['allow_empty'] = allow_empty
         list_kwargs.update(dict([
             (key, value) for key, value in kwargs.items()
             if key in LIST_SERIALIZER_KWARGS

rest_framework/utils/html.py

@@ -78,7 +78,7 @@ def parse_html_dict(dictionary, prefix=''):
         }
     }
     """
-    ret = {}
+    ret = MultiValueDict()
     regex = re.compile(r'^%s\.(.+)$' % re.escape(prefix))
     for field, value in dictionary.items():
         match = regex.match(field)

tests/test_fields.py

@@ -466,6 +466,18 @@ class TestBooleanField(FieldValues):
     }
     field = serializers.BooleanField()
 
+    def test_disallow_unhashable_collection_types(self):
+        inputs = (
+            [],
+            {},
+        )
+        field = serializers.BooleanField()
+        for input_value in inputs:
+            with pytest.raises(serializers.ValidationError) as exc_info:
+                field.run_validation(input_value)
+            expected = ['"{0}" is not a valid boolean.'.format(input_value)]
+            assert exc_info.value.detail == expected
+
 
 class TestNullBooleanField(FieldValues):
     """
@@ -1069,6 +1081,7 @@ class TestDurationField(FieldValues):
         '3 08:32:01.000123': datetime.timedelta(days=3, hours=8, minutes=32, seconds=1, microseconds=123),
         '08:01': datetime.timedelta(minutes=8, seconds=1),
         datetime.timedelta(days=3, hours=8, minutes=32, seconds=1, microseconds=123): datetime.timedelta(days=3, hours=8, minutes=32, seconds=1, microseconds=123),
+        3600: datetime.timedelta(hours=1),
     }
     invalid_inputs = {
         'abc': ['Duration has wrong format. Use one of these formats instead: [DD] [HH:[MM:]]ss[.uuuuuu].'],

tests/test_serializer_nested.py

@@ -79,17 +79,23 @@ class TestNestedSerializerWithMany:
         class TestSerializer(serializers.Serializer):
             allow_null = NestedSerializer(many=True, allow_null=True)
             not_allow_null = NestedSerializer(many=True)
+            allow_empty = NestedSerializer(many=True, allow_empty=True)
+            not_allow_empty = NestedSerializer(many=True, allow_empty=False)
 
         self.Serializer = TestSerializer
 
     def test_null_allowed_if_allow_null_is_set(self):
         input_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': '2'}, {'example': '3'}]
+            'not_allow_null': [{'example': '2'}, {'example': '3'}],
+            'allow_empty': [{'example': '2'}],
+            'not_allow_empty': [{'example': '2'}],
         }
         expected_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': 2}, {'example': 3}]
+            'not_allow_null': [{'example': 2}, {'example': 3}],
+            'allow_empty': [{'example': 2}],
+            'not_allow_empty': [{'example': 2}],
         }
         serializer = self.Serializer(data=input_data)
 
@@ -99,7 +105,9 @@ class TestNestedSerializerWithMany:
     def test_null_is_not_allowed_if_allow_null_is_not_set(self):
         input_data = {
             'allow_null': None,
-            'not_allow_null': None
+            'not_allow_null': None,
+            'allow_empty': [{'example': '2'}],
+            'not_allow_empty': [{'example': '2'}],
         }
         serializer = self.Serializer(data=input_data)
 
@@ -118,10 +126,44 @@ class TestNestedSerializerWithMany:
 
         input_data = {
             'allow_null': None,
-            'not_allow_null': [{'example': 2}]
+            'not_allow_null': [{'example': 2}],
+            'allow_empty': [{'example': 2}],
+            'not_allow_empty': [{'example': 2}],
         }
         serializer = TestSerializer(data=input_data)
 
         assert serializer.is_valid()
         assert serializer.validated_data == input_data
         assert TestSerializer.validation_was_run
+
+    def test_empty_allowed_if_allow_empty_is_set(self):
+        input_data = {
+            'allow_null': [{'example': '2'}],
+            'not_allow_null': [{'example': '2'}],
+            'allow_empty': [],
+            'not_allow_empty': [{'example': '2'}],
+        }
+        expected_data = {
+            'allow_null': [{'example': 2}],
+            'not_allow_null': [{'example': 2}],
+            'allow_empty': [],
+            'not_allow_empty': [{'example': 2}],
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert serializer.is_valid(), serializer.errors
+        assert serializer.validated_data == expected_data
+
+    def test_empty_not_allowed_if_allow_empty_is_set_to_false(self):
+        input_data = {
+            'allow_null': [{'example': '2'}],
+            'not_allow_null': [{'example': '2'}],
+            'allow_empty': [],
+            'not_allow_empty': [],
+        }
+        serializer = self.Serializer(data=input_data)
+
+        assert not serializer.is_valid()
+
+        expected_errors = {'not_allow_empty': {'non_field_errors': [serializers.ListSerializer.default_error_messages['empty']]}}
+        assert serializer.errors == expected_errors