Merge branch 'bugfix/nested_through_many_to_many' into pk_related

* bugfix/nested_through_many_to_many: (29 commits)
  Updated failing test from #1575
  Add colon to time zone offset in readable_datetime_formats
  Replaced singular "is" by plural "are"
  Removed unnecessary "that"
  Added missing "the" word
  Added test case to check if the proper attributes are set on html widgets.
  Failing test case: an item that wasn't in the relation before is created although it already exist.
  Added an test on updates through many to many field.
  Removed superfluous "./"s
  Added missing "the" word
  Added missing "to" word
  Automatically set the field name as value for the HTML `id` attribute on the rendered widget.
  Fix missing message in ValidationError
  Mark strings in AuthTokenSerializer as translatable
  Deal with reversed GenericFK.
  Removed the former virtual field list.
  Added help_text to expected response in test
  Sync test result w/ new label
  typo
  Fixed the reversed relation for virtual fields (generic foreign keys).
  ...

docs/api-guide/generic-views.md

@@ -70,7 +70,7 @@ The following attributes control the basic view behavior.
 
 **Shortcuts**:
 
-* `model` - This shortcut may be used instead of setting either (or both) of the `queryset`/`serializer_class` attributes, although using the explicit style is generally preferred.  If used instead of `serializer_class`, then then `DEFAULT_MODEL_SERIALIZER_CLASS` setting will determine the base serializer class.  Note that `model` is only ever used for generating a default queryset or serializer class - the `queryset` and `serializer_class` attributes are always preferred if provided.
+* `model` - This shortcut may be used instead of setting either (or both) of the `queryset`/`serializer_class` attributes, although using the explicit style is generally preferred.  If used instead of `serializer_class`, then `DEFAULT_MODEL_SERIALIZER_CLASS` setting will determine the base serializer class.  Note that `model` is only ever used for generating a default queryset or serializer class - the `queryset` and `serializer_class` attributes are always preferred if provided.
 
 **Pagination**:
 

docs/api-guide/pagination.md

@@ -103,6 +103,7 @@ You can also set the pagination style on a per-view basis, using the `ListAPIVie
         max_paginate_by = 100
 
 Note that using a `paginate_by` value of `None` will turn off pagination for the view.
+Note if you use the `PAGINATE_BY_PARAM` settings, you also have to set the `paginate_by_param` attribute in your view to `None` in order to turn off pagination for those requests that contain the `paginate_by_param` parameter.
 
 For more complex requirements such as serialization that differs depending on the requested media type you can override the `.get_paginate_by()` and `.get_pagination_serializer_class()` methods.
 
@@ -157,4 +158,4 @@ The [`DRF-extensions` package][drf-extensions] includes a [`PaginateByMaxMixin`
 
 [cite]: https://docs.djangoproject.com/en/dev/topics/pagination/
 [drf-extensions]: http://chibisov.github.io/drf-extensions/docs/
-[paginate-by-max-mixin]: http://chibisov.github.io/drf-extensions/docs/#paginatebymaxmixin
+[paginate-by-max-mixin]: http://chibisov.github.io/drf-extensions/docs/#paginatebymaxmixin

docs/tutorial/1-serialization.md

@@ -104,7 +104,7 @@ Don't forget to sync the database for the first time.
 
 ## Creating a Serializer class
 
-The first thing we need to get started on our Web API is provide a way of serializing and deserializing the snippet instances into representations such as `json`.  We can do this by declaring serializers that work very similar to Django's forms.  Create a file in the `snippets` directory named `serializers.py` and add the following.
+The first thing we need to get started on our Web API is to provide a way of serializing and deserializing the snippet instances into representations such as `json`.  We can do this by declaring serializers that work very similar to Django's forms.  Create a file in the `snippets` directory named `serializers.py` and add the following.
 
     from django.forms import widgets
     from rest_framework import serializers
@@ -143,7 +143,7 @@ The first thing we need to get started on our Web API is provide a way of serial
             # Create new instance
             return Snippet(**attrs)
 
-The first part of serializer class defines the fields that get serialized/deserialized.  The `restore_object` method defines how fully fledged instances get created when deserializing data.
+The first part of the serializer class defines the fields that get serialized/deserialized.  The `restore_object` method defines how fully fledged instances get created when deserializing data.
 
 Notice that we can also use various attributes that would typically be used on form fields, such as `widget=widgets.Textarea`.  These can be used to control how the serializer should render when displayed as an HTML form.  This is particularly useful for controlling how the browsable API should be displayed, as we'll see later in the tutorial.
 

docs/tutorial/4-authentication-and-permissions.md

@@ -44,11 +44,11 @@ When that's all done we'll need to update our database tables.
 Normally we'd create a database migration in order to do that, but for the purposes of this tutorial, let's just delete the database and start again.
 
     rm tmp.db
-    python ./manage.py syncdb
+    python manage.py syncdb
 
 You might also want to create a few different users, to use for testing the API.  The quickest way to do this will be with the `createsuperuser` command.
 
-    python ./manage.py createsuperuser
+    python manage.py createsuperuser
 
 ## Adding endpoints for our User models
 

docs/tutorial/6-viewsets-and-routers.md

@@ -21,7 +21,7 @@ First of all let's refactor our `UserList` and `UserDetail` views into a single
         queryset = User.objects.all()
         serializer_class = UserSerializer
 
-Here we've used `ReadOnlyModelViewSet` class to automatically provide the default 'read-only' operations.  We're still setting the `queryset` and `serializer_class` attributes exactly as we did when we were using regular views, but we no longer need to provide the same information to two separate classes.
+Here we've used the `ReadOnlyModelViewSet` class to automatically provide the default 'read-only' operations.  We're still setting the `queryset` and `serializer_class` attributes exactly as we did when we were using regular views, but we no longer need to provide the same information to two separate classes.
 
 Next we're going to replace the `SnippetList`, `SnippetDetail` and `SnippetHighlight` view classes.  We can remove the three views, and again replace them with a single class.
 
@@ -85,7 +85,7 @@ In the `urls.py` file we bind our `ViewSet` classes into a set of concrete views
 
 Notice how we're creating multiple views from each `ViewSet` class, by binding the http methods to the required action for each view.
 
-Now that we've bound our resources into concrete views, that we can register the views with the URL conf as usual.
+Now that we've bound our resources into concrete views, we can register the views with the URL conf as usual.
 
     urlpatterns = format_suffix_patterns(patterns('snippets.views',
         url(r'^$', 'api_root'),
@@ -138,7 +138,7 @@ You can review the final [tutorial code][repo] on GitHub, or try out a live exam
 
 ## Onwards and upwards
 
-We've reached the end of our tutorial.  If you want to get more involved in the REST framework project, here's a few places you can start:
+We've reached the end of our tutorial.  If you want to get more involved in the REST framework project, here are a few places you can start:
 
 * Contribute on [GitHub][github] by reviewing and submitting issues, and making pull requests.
 * Join the [REST framework discussion group][group], and help build the community.

rest_framework/authtoken/serializers.py

@@ -1,4 +1,6 @@
 from django.contrib.auth import authenticate
+from django.utils.translation import ugettext_lazy as _
+
 from rest_framework import serializers
 
 
@@ -15,10 +17,13 @@ class AuthTokenSerializer(serializers.Serializer):
 
             if user:
                 if not user.is_active:
-                    raise serializers.ValidationError('User account is disabled.')
+                    msg = _('User account is disabled.')
+                    raise serializers.ValidationError(msg)
                 attrs['user'] = user
                 return attrs
             else:
-                raise serializers.ValidationError('Unable to login with provided credentials.')
+                msg = _('Unable to login with provided credentials.')
+                raise serializers.ValidationError(msg)
         else:
-            raise serializers.ValidationError('Must include "username" and "password"')
+            msg = _('Must include "username" and "password"')
+            raise serializers.ValidationError(msg)

rest_framework/fields.py

@@ -62,7 +62,7 @@ def get_component(obj, attr_name):
 
 def readable_datetime_formats(formats):
     format = ', '.join(formats).replace(ISO_8601,
-             'YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HHMM|-HHMM|Z]')
+             'YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HH:MM|-HH:MM|Z]')
     return humanize_strptime(format)
 
 
@@ -154,7 +154,12 @@ class Field(object):
     def widget_html(self):
         if not self.widget:
             return ''
-        return self.widget.render(self._name, self._value)
+
+        attrs = {}
+        if 'id' not in self.widget.attrs:
+            attrs['id'] = self._name
+
+        return self.widget.render(self._name, self._value, attrs=attrs)
 
     def label_tag(self):
         return '<label for="%s">%s:</label>' % (self._name, self.label)

rest_framework/serializers.py

@@ -16,7 +16,7 @@ import datetime
 import inspect
 import types
 from decimal import Decimal
-from django.contrib.contenttypes.generic import GenericForeignKey
+from django.contrib.contenttypes.generic import GenericForeignKey, GenericRelation
 from django.core.paginator import Page
 from django.db import models
 from django.forms import widgets
@@ -833,6 +833,15 @@ class ModelSerializer(Serializer):
             if model_field.verbose_name is not None:
                 kwargs['label'] = model_field.verbose_name
 
+            if not model_field.editable:
+                kwargs['read_only'] = True
+
+            if model_field.verbose_name is not None:
+                kwargs['label'] = model_field.verbose_name
+
+            if model_field.help_text is not None:
+                kwargs['help_text'] = model_field.help_text
+
         return PrimaryKeyRelatedField(**kwargs)
 
     def get_field(self, model_field):
@@ -932,6 +941,7 @@ class ModelSerializer(Serializer):
         m2m_data = {}
         related_data = {}
         nested_forward_relations = {}
+        generic_fk = []
         meta = self.opts.model._meta
 
         # Reverse fk or one-to-one relations
@@ -950,6 +960,8 @@ class ModelSerializer(Serializer):
         for field in meta.many_to_many + meta.virtual_fields:
             if isinstance(field, GenericForeignKey):
                 continue
+            if isinstance(field, GenericRelation):
+                generic_fk.append(field.name)
             if field.name in attrs:
                 m2m_data[field.name] = attrs.pop(field.name)
 
@@ -973,6 +985,7 @@ class ModelSerializer(Serializer):
         # saved the model get hidden away on these
         # private attributes, so we can deal with them
         # at the point of save.
+        instance._generic_fk = generic_fk
         instance._related_data = related_data
         instance._m2m_data = m2m_data
         instance._nested_forward_relations = nested_forward_relations
@@ -1003,7 +1016,13 @@ class ModelSerializer(Serializer):
 
         if getattr(obj, '_m2m_data', None):
             for accessor_name, object_list in obj._m2m_data.items():
-                setattr(obj, accessor_name, object_list)
+                if accessor_name in getattr(obj, '_generic_fk', []):
+                    # We are dealing with a reversed generic FK
+                    setattr(obj, accessor_name, object_list)
+                [self.save_object(o) for o in object_list if not isinstance(o, GenericRelation)]
+                if accessor_name not in getattr(obj, '_generic_fk', []):
+                    # We need to save m2m data before linking the objects together
+                    setattr(obj, accessor_name, object_list)
             del(obj._m2m_data)
 
         if getattr(obj, '_related_data', None):

rest_framework/tests/models.py

@@ -151,7 +151,8 @@ class ForeignKeySource(RESTFrameworkModel):
 class NullableForeignKeySource(RESTFrameworkModel):
     name = models.CharField(max_length=100)
     target = models.ForeignKey(ForeignKeyTarget, null=True, blank=True,
-                               related_name='nullable_sources')
+                               related_name='nullable_sources',
+                               verbose_name='Optional target object')
 
 
 # OneToOne

rest_framework/tests/test_fields.py

@@ -4,6 +4,7 @@ General serializer field tests.
 from __future__ import unicode_literals
 
 import datetime
+import re
 from decimal import Decimal
 from uuid import uuid4
 from django.core import validators
@@ -103,6 +104,16 @@ class BasicFieldTests(TestCase):
         keys = list(field.to_native(ret).keys())
         self.assertEqual(keys, ['c', 'b', 'a', 'z'])
 
+    def test_widget_html_attributes(self):
+        """
+        Make sure widget_html() renders the correct attributes
+        """
+        r = re.compile('(\S+)=["\']?((?:.(?!["\']?\s+(?:\S+)=|[>"\']))+.)["\']?')
+        form = TimeFieldModelSerializer().data
+        attributes = r.findall(form.fields['clock'].widget_html())
+        self.assertIn(('name', 'clock'), attributes)
+        self.assertIn(('id', 'clock'), attributes)
+
 
 class DateFieldTest(TestCase):
     """
@@ -312,7 +323,7 @@ class DateTimeFieldTest(TestCase):
             f.from_native('04:61:59')
         except validators.ValidationError as e:
             self.assertEqual(e.messages, ["Datetime has wrong format. Use one of these formats instead: "
-                                          "YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HHMM|-HHMM|Z]"])
+                                          "YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HH:MM|-HH:MM|Z]"])
         else:
             self.fail("ValidationError was not properly raised")
 
@@ -326,7 +337,7 @@ class DateTimeFieldTest(TestCase):
             f.from_native('04 -- 31')
         except validators.ValidationError as e:
             self.assertEqual(e.messages, ["Datetime has wrong format. Use one of these formats instead: "
-                                          "YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HHMM|-HHMM|Z]"])
+                                          "YYYY-MM-DDThh:mm[:ss[.uuuuuu]][+HH:MM|-HH:MM|Z]"])
         else:
             self.fail("ValidationError was not properly raised")
 

rest_framework/tests/test_generics.py

@@ -5,6 +5,7 @@ from django.test import TestCase
 from rest_framework import generics, renderers, serializers, status
 from rest_framework.test import APIRequestFactory
 from rest_framework.tests.models import BasicModel, Comment, SlugBasedModel
+from rest_framework.tests.models import ForeignKeySource, ForeignKeyTarget
 from rest_framework.compat import six
 
 factory = APIRequestFactory()
@@ -28,6 +29,13 @@ class InstanceView(generics.RetrieveUpdateDestroyAPIView):
         return queryset.exclude(text='filtered out')
 
 
+class FKInstanceView(generics.RetrieveUpdateDestroyAPIView):
+    """
+    FK: example description for OPTIONS.
+    """
+    model = ForeignKeySource
+
+
 class SlugSerializer(serializers.ModelSerializer):
     slug = serializers.Field()  # read only
 
@@ -407,6 +415,72 @@ class TestInstanceView(TestCase):
         self.assertFalse(self.objects.filter(id=999).exists())
 
 
+class TestFKInstanceView(TestCase):
+    def setUp(self):
+        """
+        Create 3 BasicModel instances.
+        """
+        items = ['foo', 'bar', 'baz']
+        for item in items:
+            t = ForeignKeyTarget(name=item)
+            t.save()
+            ForeignKeySource(name='source_' + item, target=t).save()
+
+        self.objects = ForeignKeySource.objects
+        self.data = [
+            {'id': obj.id, 'name': obj.name}
+            for obj in self.objects.all()
+        ]
+        self.view = FKInstanceView.as_view()
+
+    def test_options_root_view(self):
+        """
+        OPTIONS requests to ListCreateAPIView should return metadata
+        """
+        request = factory.options('/999')
+        with self.assertNumQueries(1):
+            response = self.view(request, pk=999).render()
+        expected = {
+            'name': 'Fk Instance',
+            'description': 'FK: example description for OPTIONS.',
+            'renders': [
+                'application/json',
+                'text/html'
+            ],
+            'parses': [
+                'application/json',
+                'application/x-www-form-urlencoded',
+                'multipart/form-data'
+            ],
+            'actions': {
+                'PUT': {
+                    'id': {
+                        'type': 'integer',
+                        'required': False,
+                        'read_only': True,
+                        'label': 'ID'
+                    },
+                    'name': {
+                        'type': 'string',
+                        'required': True,
+                        'read_only': False,
+                        'label': 'name',
+                        'max_length': 100
+                    },
+                    'target': {
+                        'type': 'field',
+                        'required': True,
+                        'read_only': False,
+                        'label': 'Target',
+                        'help_text': 'Target'
+                    }
+                }
+            }
+        }
+        self.assertEqual(response.status_code, status.HTTP_200_OK)
+        self.assertEqual(response.data, expected)
+
+
 class TestOverriddenGetObject(TestCase):
     """
     Test cases for a RetrieveUpdateDestroyAPIView that does NOT use the

rest_framework/tests/test_serializer_nested.py

@@ -345,3 +345,74 @@ class NestedModelSerializerUpdateTests(TestCase):
         result = deserialize.object
         result.save()
         self.assertEqual(result.id, john.id)
+
+    def test_creation_with_nested_many_to_many_relation(self):
+        class ManyToManyTargetSerializer(serializers.ModelSerializer):
+            class Meta:
+                model = models.ManyToManyTarget
+
+        class ManyToManySourceSerializer(serializers.ModelSerializer):
+            targets = ManyToManyTargetSerializer(many=True, allow_add_remove=True)
+            class Meta:
+                model = models.ManyToManySource
+
+        data = {
+            'name': 'source',
+            'targets': [{
+                'name': 'target1'
+            }, {
+                'name': 'another target'
+            }]
+        }
+
+        source_count = models.ManyToManySource.objects.count()
+        target_count = models.ManyToManyTarget.objects.count()
+
+        deserialize = ManyToManySourceSerializer(data=data)
+        self.assertTrue(deserialize.is_valid(), deserialize.errors)
+        deserialize.save()
+        self.assertEqual(models.ManyToManySource.objects.count(), source_count + 1)
+        self.assertEqual(models.ManyToManyTarget.objects.count(), target_count + 2)
+
+    def test_update_with_nested_many_to_many_relation(self):
+        class ManyToManyTargetSerializer(serializers.ModelSerializer):
+            class Meta:
+                model = models.ManyToManyTarget
+
+        class ManyToManySourceSerializer(serializers.ModelSerializer):
+            targets = ManyToManyTargetSerializer(many=True, allow_add_remove=True)
+            class Meta:
+                model = models.ManyToManySource
+
+        source = models.ManyToManySource.objects.create(name='source')
+        target1 = models.ManyToManyTarget.objects.create(name='target1')
+        target2 = models.ManyToManyTarget.objects.create(name='target2')
+        source.targets = [target1]
+
+        data = {
+            'id': source.id,
+            'name': source.name + '0',
+            'targets': [{
+                'id': target1.id,
+                'name': target1.name + '1',
+            }, {
+                'id': target2.id,
+                'name': target2.name + '2',
+            }]
+        }
+
+        source_count = models.ManyToManySource.objects.count()
+        target_count = models.ManyToManyTarget.objects.count()
+
+        deserialize = ManyToManySourceSerializer(data=data, instance=source)
+        self.assertTrue(deserialize.is_valid(), deserialize.errors)
+        deserialize.save()
+        self.assertEqual(models.ManyToManySource.objects.count(), source_count)
+        self.assertEqual(models.ManyToManyTarget.objects.count(), target_count)
+
+        # Were the models updated ?
+        self.assertEqual(source.name, 'source0')
+        alt_target1 = models.ManyToManyTarget.objects.get(id=target1.id)
+        self.assertEqual(alt_target1.name, target1.name + '1')
+        alt_target2 = models.ManyToManyTarget.objects.get(id=target2.id)
+        self.assertEqual(alt_target2.name, target2.name + '2')

rest_framework/throttling.py

@@ -157,6 +157,8 @@ class AnonRateThrottle(SimpleRateThrottle):
         ident = request.META.get('HTTP_X_FORWARDED_FOR')
         if ident is None:
             ident = request.META.get('REMOTE_ADDR')
+        else:
+            ident = ''.join(ident.split())
 
         return self.cache_format % {
             'scope': self.scope,