3.0 serializer docs

docs/api-guide/serializers.md

@@ -10,7 +10,7 @@ will take some serious design work.
 
 Serializers allow complex data such as querysets and model instances to be converted to native Python datatypes that can then be easily rendered into `JSON`, `XML` or other content types.  Serializers also provide deserialization, allowing parsed data to be converted back into complex types, after first validating the incoming data.
 
-REST framework's serializers work very similarly to Django's `Form` and `ModelForm` classes.  It provides a `Serializer` class which gives you a powerful, generic way to control the output of your responses, as well as a `ModelSerializer` class which provides a useful shortcut for creating serializers that deal with model instances and querysets.
+The serializers in REST framework work very similarly to Django's `Form` and `ModelForm` classes. We provide a `Serializer` class which gives you a powerful, generic way to control the output of your responses, as well as a `ModelSerializer` class which provides a useful shortcut for creating serializers that deal with model instances and querysets.
 
 ## Declaring Serializers
 
@@ -24,7 +24,7 @@ Let's start by creating a simple object we can use for example purposes:
 
     comment = Comment(email='leila@example.com', content='foo bar')
 
-We'll declare a serializer that we can use to serialize and deserialize `Comment` objects.
+We'll declare a serializer that we can use to serialize and deserialize data that corresponds to `Comment` objects.
 
 Declaring a serializer looks very similar to declaring a form:
 
@@ -35,22 +35,6 @@ Declaring a serializer looks very similar to declaring a form:
         content = serializers.CharField(max_length=200)
         created = serializers.DateTimeField()
 
-        def restore_object(self, attrs, instance=None):
-            """
-            Given a dictionary of deserialized field values, either update
-            an existing model instance, or create a new model instance.
-            """
-            if instance is not None:
-                instance.email = attrs.get('email', instance.email)
-                instance.content = attrs.get('content', instance.content)
-                instance.created = attrs.get('created', instance.created)
-                return instance
-            return Comment(**attrs)
-
-The first part of serializer class defines the fields that get serialized/deserialized.  The `restore_object` method defines how fully fledged instances get created when deserializing data.
-
-The `restore_object` method is optional, and is only required if we want our serializer to support deserialization into fully fledged object instances.  If we don't define this method, then deserializing data will simply return a dictionary of items.
-
 ## Serializing objects
 
 We can now use `CommentSerializer` to serialize a comment, or list of comments. Again, using the `Serializer` class looks a lot like using a `Form` class.
@@ -67,21 +51,6 @@ At this point we've translated the model instance into Python native datatypes.
     json
     # '{"email": "leila@example.com", "content": "foo bar", "created": "2012-08-22T16:20:09.822"}'
 
-### Customizing field representation
-
-Sometimes when serializing objects, you may not want to represent everything exactly the way it is in your model.
-
-If you need to customize the serialized value of a particular field, you can do this by creating a `transform_<fieldname>` method. For example if you needed to render some markdown from a text field:
-
-    description = serializers.CharField()
-    description_html = serializers.CharField(source='description', read_only=True)
-
-    def transform_description_html(self, obj, value):
-        from django.contrib.markup.templatetags.markup import markdown
-        return markdown(value)
-
-These methods are essentially the reverse of `validate_<fieldname>` (see *Validation* below.)
-
 ## Deserializing objects
 
 Deserialization is similar. First we parse a stream into Python native datatypes...
@@ -92,26 +61,90 @@ Deserialization is similar.  First we parse a stream into Python native datatype
     stream = StringIO(json)
     data = JSONParser().parse(stream)
 
-...then we restore those native datatypes into a fully populated object instance.
+...then we restore those native datatypes into a dictionary of validated data.
 
     serializer = CommentSerializer(data=data)
     serializer.is_valid()
     # True
-    serializer.object
+    serializer.validated_data
-    # <Comment object at 0x10633b2d0>
+    # {'content': 'foo bar', 'email': 'leila@example.com', 'created': datetime.datetime(2012, 08, 22, 16, 20, 09, 822243)}
 
-When deserializing data, we can either create a new instance, or update an existing instance.
+## Saving instances
 
-    serializer = CommentSerializer(data=data)           # Create new instance
+If we want to be able to return complete object instances based on the validated data we need to implement one or both of the `.create()` and `update()` methods. For example:
-    serializer = CommentSerializer(comment, data=data)  # Update `comment`
 
-By default, serializers must be passed values for all required fields or they will throw validation errors.  You can use the `partial` argument in order to allow partial updates.
+    class CommentSerializer(serializers.Serializer):
+        email = serializers.EmailField()
+        content = serializers.CharField(max_length=200)
+        created = serializers.DateTimeField()
 
-    serializer = CommentSerializer(comment, data={'content': u'foo bar'}, partial=True)  # Update `comment` with partial data
+        def create(self, validated_data):
+            return Comment(**validated_data)
+
+        def update(self, instance, validated_data):
+            instance.email = validated_data.get('email', instance.email)
+            instance.content = validated_data.get('content', instance.content)
+            instance.created = validated_data.get('created', instance.created)
+            return instance
+
+If your object instances correspond to Django models you'll also want to ensure that these methods save the object to the database. For example, if `Comment` was a Django model, the methods might look like this:
+
+        def create(self, validated_data):
+            return Comment.objcts.create(**validated_data)
+
+        def update(self, instance, validated_data):
+            instance.email = validated_data.get('email', instance.email)
+            instance.content = validated_data.get('content', instance.content)
+            instance.created = validated_data.get('created', instance.created)
+            instance.save()
+            return instance
+ 
+Now when deserializing data, we can call `.save()` to return an object instance, based on the validated data.
+
+    comment = serializer.save()
+
+Calling `.save()` will either create a new instance, or update an existing instance, depending on if an existing instance was passed when instantiating the serializer class:
+
+    # .save() will create a new instance.
+    serializer = CommentSerializer(data=data)
+    
+    # .save() will update the existing `comment` instance.
+    serializer = CommentSerializer(comment, data=data)
+
+Both the `.create()` and `.update()` methods are optional. You can implement either neither, one, or both of them, depending on the use-case for your serializer class.
+
+#### Passing additional attributes to `.save()`
+
+Sometimes you'll want your view code to be able to inject additional data at the point of saving the instance. This additional data might include information like the current user, the current time, or anything else that is not part of the request data.
+
+You can do so by including additional keyword arguments when calling `.save()`. For example:
+
+    serializer.save(owner=request.user)
+
+Any additional keyword arguments will be included in the `validated_data` argument when `.create()` or `.update()` are called.
+
+#### Overriding `.save()` directly.
+
+In some cases the `.create()` and `.update()` method names may not be meaningful. For example, in a contact form we may not be creating new instances, but instead sending an email or other message.
+
+In these cases you might instead choose to override `.save()` directly, as being more readable and meaningful.
+
+For example:
+
+    class ContactForm(serializers.Serializer):
+        email = serializers.EmailField()
+        message = serializers.CharField()
+        
+        def save(self):
+            email = self.validated_data['email']
+            message = self.validated_data['message']
+            send_email(from=email, message=message)
+
+Note that in the case above we're now having to access the serializer `.validated_data` property directly.
 
 ## Validation
 
-When deserializing data, you always need to call `is_valid()` before attempting to access the deserialized object.  If any validation errors occur, the `.errors` property will contain a dictionary representing the resulting error messages.  For example:
+When deserializing data, you always need to call `is_valid()` before attempting to access the validated data, or save an object instance. If any validation errors occur, the `.errors` property will contain a dictionary representing the resulting error messages.  For example:
 
     serializer = CommentSerializer(data={'email': 'foobar', 'content': 'baz'})
     serializer.is_valid()
@@ -119,17 +152,26 @@ When deserializing data, you always need to call `is_valid()` before attempting
     serializer.errors
     # {'email': [u'Enter a valid e-mail address.'], 'created': [u'This field is required.']}
 
-Each key in the dictionary will be the field name, and the values will be lists of strings of any error messages corresponding to that field.  The `non_field_errors` key may also be present, and will list any general validation errors.
+Each key in the dictionary will be the field name, and the values will be lists of strings of any error messages corresponding to that field.  The `non_field_errors` key may also be present, and will list any general validation errors. The name of the `non_field_errors` key may be customized using the `NON_FIELD_ERRORS_KEY` REST framework setting.
 
 When deserializing a list of items, errors will be returned as a list of dictionaries representing each of the deserialized items.
 
+#### Raising an exception on invalid data
+
+The `.is_valid()` method takes an optional `raise_exception` flag that will cause it to raise a `serializers.ValidationError` exception if there are validation errors.
+
+These exceptions are automatically dealt with by the default exception handler that REST framework provides, and will return `HTTP 400 Bad Request` responses by default.
+
+    # Return a 400 response if the data was invalid.
+    serializer.is_valid(raise_exception=True)
+
 #### Field-level validation
 
-You can specify custom field-level validation by adding `.validate_<fieldname>` methods to your `Serializer` subclass.  These are analogous to `.clean_<fieldname>` methods on Django forms, but accept slightly different arguments.
+You can specify custom field-level validation by adding `.validate_<fieldname>` methods to your `Serializer` subclass.  These are similar to the `.clean_<fieldname>` methods on Django forms.
 
-They take a dictionary of deserialized attributes as a first argument, and the field name in that dictionary as a second argument (which will be either the name of the field or the value of the `source` argument to the field, if one was provided).
+These methods take a single argument, which is the field value that requires validation.
 
-Your `validate_<fieldname>` methods should either just return the `attrs` dictionary or raise a `ValidationError`.  For example:
+Your `validate_<fieldname>` methods should return the validated value or raise a `ValidationError`.  For example:
 
     from rest_framework import serializers
 
@@ -137,18 +179,17 @@ Your `validate_<fieldname>` methods should either just return the `attrs` dictio
         title = serializers.CharField(max_length=100)
         content = serializers.CharField()
 
-        def validate_title(self, attrs, source):
+        def validate_title(self, value):
             """
             Check that the blog post is about Django.
             """
-            value = attrs[source]
+            if 'django' not in value.lower():
-            if "django" not in value.lower():
                 raise serializers.ValidationError("Blog post is not about Django")
-            return attrs
+            return value
 
 #### Object-level validation
 
-To do any other validation that requires access to multiple fields, add a method called `.validate()` to your `Serializer` subclass.  This method takes a single argument, which is the `attrs` dictionary.  It should raise a `ValidationError` if necessary, or just return `attrs`.  For example:
+To do any other validation that requires access to multiple fields, add a method called `.validate()` to your `Serializer` subclass.  This method takes a single argument, which is a dictionary of field values.  It should raise a `ValidationError` if necessary, or just return the validated values.  For example:
 
     from rest_framework import serializers
 
@@ -157,24 +198,48 @@ To do any other validation that requires access to multiple fields, add a method
         start = serializers.DateTimeField()
         finish = serializers.DateTimeField()
 
-        def validate(self, attrs):
+        def validate(self, data):
             """
             Check that the start is before the stop.
             """
-            if attrs['start'] > attrs['finish']:
+            if data['start'] > data['finish']:
                 raise serializers.ValidationError("finish must occur after start")
-            return attrs
+            return data
 
-## Saving object state
+#### Validators
 
-To save the deserialized objects created by a serializer, call the `.save()` method:
+Individual fields on a serializer can include validators, by declaring them on the field instance, for example:
 
-    if serializer.is_valid():
+    def multiple_of_ten(value):
-        serializer.save()
+        if value % 10 != 0:
+            raise serializers.ValidationError('Not a multiple of ten')
 
-The default behavior of the method is to simply call `.save()` on the deserialized object instance.  You can override the default save behaviour by overriding the `.save_object(obj)` method on the serializer class.
+    class GameRecord(serializers.Serializer):
+        score = IntegerField(validators=[multiple_of_ten])
+        ...
 
-The generic views provided by REST framework call the `.save()` method when updating or creating entities.
+Serializer classes can also include reusable validators that are applied to the complete set of field data. These validators are included by declaring them on an inner `Meta` class, like so:
+
+    class EventSerializer(serializers.Serializer):
+        name = serializers.CharField()
+        room_number = serializers.IntegerField(choices=[101, 102, 103, 201])
+        date = serializers.DateField()
+        
+        class Meta:
+            # Each room only has one event per day.
+            validators = UniqueTogetherValidator(
+                queryset=Event.objects.all(),
+                fields=['room_number', 'date']
+            )
+
+For more information see the [validators documentation](validators.md).
+
+## Partial updates
+
+By default, serializers must be passed values for all required fields or they will raise validation errors. You can use the `partial` argument in order to allow partial updates.
+
+    # Update `comment` with partial data
+    serializer = CommentSerializer(comment, data={'content': u'foo bar'}, partial=True)
 
 ## Dealing with nested objects
 
@@ -214,6 +279,8 @@ Validation of nested objects will work the same as before.  Errors with nested o
     serializer.errors
     # {'user': {'email': [u'Enter a valid e-mail address.']}, 'created': [u'This field is required.']}
 
+**TODO** Document create and update for nested serializers
+
 ## Dealing with multiple objects
 
 The `Serializer` class can also handle serializing or deserializing lists of objects.
@@ -233,6 +300,8 @@ To serialize a queryset or list of objects instead of a single object instance,
 
 #### Deserializing multiple objects for creation
 
+**TODO**
+
 To deserialize a list of object data, and create multiple object instances in a single pass, you should also set the `many=True` flag, and pass a list of data to be deserialized.
 
 This allows you to write views that create multiple items when a `POST` request is made.
@@ -250,6 +319,8 @@ For example:
 
 #### Deserializing multiple objects for update
 
+**TODO**
+
 You can also deserialize a list of objects as part of a bulk update of multiple existing items.
 In this case you need to supply both an existing list or queryset of items, as well as a list of data to update those items with.
 
@@ -278,32 +349,6 @@ When performing a bulk update you may want to allow new items to be created, and
 
 Passing `allow_add_remove=True` ensures that any update operations will completely overwrite the existing queryset, rather than simply updating existing objects.
 
-#### How identity is determined when performing bulk updates
-
-Performing a bulk update is slightly more complicated than performing a bulk creation, because the serializer needs a way to determine how the items in the incoming data should be matched against the existing object instances.
-
-By default the serializer class will use the `id` key on the incoming data to determine the canonical identity of an object.  If you need to change this behavior you should override the `get_identity` method on the `Serializer` class.  For example:
-
-    class AccountSerializer(serializers.Serializer):
-        slug = serializers.CharField(max_length=100)
-        created = serializers.DateTimeField()
-        ...  # Various other fields
-
-        def get_identity(self, data):
-            """
-            This hook is required for bulk update.
-            We need to override the default, to use the slug as the identity.
-
-            Note that the data has not yet been validated at this point,
-            so we need to deal gracefully with incorrect datatypes.
-            """
-            try:
-                return data.get('slug', None)
-            except AttributeError:
-                return None
-
-To map the incoming data items to their corresponding object instances, the `.get_identity()` method will be called both against the incoming data, and against the serialized representation of the existing objects.
-
 ## Including extra context
 
 There are some cases where you need to provide extra context to the serializer in addition to the object being serialized.  One common case is if you're using a serializer that includes hyperlinked relations, which requires the serializer to have access to the current request so that it can properly generate fully qualified URLs.
@@ -314,27 +359,45 @@ You can provide arbitrary additional context by passing a `context` argument whe
     serializer.data
     # {'id': 6, 'owner': u'denvercoder9', 'created': datetime.datetime(2013, 2, 12, 09, 44, 56, 678870), 'details': 'http://example.com/accounts/6/details'}
 
-The context dictionary can be used within any serializer field logic, such as a custom `.to_native()` method, by accessing the `self.context` attribute.
+The context dictionary can be used within any serializer field logic, such as a custom `.to_representation()` method, by accessing the `self.context` attribute.
+
+---
 
--
 # ModelSerializer
 
-Often you'll want serializer classes that map closely to model definitions.
+Often you'll want serializer classes that map closely to Django model definitions.
-The `ModelSerializer` class lets you automatically create a Serializer class with fields that correspond to the Model fields.
+
+The `ModelSerializer` class provides a shortcut that lets you automatically create a `Serializer` class with fields that correspond to the Model fields.
+
+**The `ModelSerializer` class is the same as a regular `Serializer` class, except that**:
+
+* It will automatically generate a set of fields for you, based on the model.
+* It will automatically generate validators for the serializer, such as unique_together validators.
+* It includes simple default implementations of `.create()` and `.update()`.
+
+Declaring a `ModelSerializer` looks like this:
 
     class AccountSerializer(serializers.ModelSerializer):
         class Meta:
             model = Account
 
-By default, all the model fields on the class will be mapped to corresponding serializer fields.
+By default, all the model fields on the class will be mapped to a corresponding serializer fields.
 
-Any relationships such as foreign keys on the model will be mapped to `PrimaryKeyRelatedField`.  Other models fields will be mapped to a corresponding serializer field.
+Any relationships such as foreign keys on the model will be mapped to `PrimaryKeyRelatedField`. Reverse relationships are not included by default unless explicitly included as described below.
 
----
+#### Inspecting the generated `ModelSerializer` class.
 
-**Note**: When validation is applied to a `ModelSerializer`, both the serializer fields, and their corresponding model fields must correctly validate.  If you have optional fields on your model, make sure to correctly set `blank=True` on the model field, as well as setting `required=False` on the serializer field.
+Serializer classes generate helpful verbose representation strings, that allow you to fully inspect the state of their fields. This is particularly useful when working with `ModelSerializers` where you want to determine what set of fields and validators are being automatically created for you.
 
----
+To do so, open the Django shell, using `python manage.py shell`, then import the serializer class, instantiate it, and print the object representation…
+
+    >>> from myapp.serializers import AccountSerializer
+    >>> serializer = AccountSerializer()
+    >>> print repr(serializer)  # Or `print(repr(serializer))` in Python 3.x.
+    AccountSerializer():
+        id = IntegerField(label='ID', read_only=True)
+        name = CharField(allow_blank=True, max_length=100, required=False)
+        owner = PrimaryKeyRelatedField(queryset=User.objects.all())
  
 ## Specifying which fields should be included
 
@@ -347,6 +410,10 @@ For example:
             model = Account
             fields = ('id', 'account_name', 'users', 'created')
 
+The names in the `fields` option will normally map to model fields on the model class.
+
+Alternatively names in the `fields` options can map to properties or methods which take no arguments that exist on the model class.
+
 ## Specifying nested serialization
 
 The default `ModelSerializer` uses primary keys for relationships, but you can also easily generate nested representations using the `depth` option:
@@ -361,37 +428,6 @@ The `depth` option should be set to an integer value that indicates the depth of
 
 If you want to customize the way the serialization is done (e.g. using `allow_add_remove`) you'll need to define the field yourself.
 
-## Specifying which fields should be read-only
-
-You may wish to specify multiple fields as read-only.  Instead of adding each field explicitly with the `read_only=True` attribute, you may use the `read_only_fields` Meta option, like so:
-
-    class AccountSerializer(serializers.ModelSerializer):
-        class Meta:
-            model = Account
-            fields = ('id', 'account_name', 'users', 'created')
-            read_only_fields = ('account_name',)
-
-Model fields which have `editable=False` set, and `AutoField` fields will be set to read-only by default, and do not need to be added to the `read_only_fields` option.
-
-## Specifying which fields should be write-only
-
-You may wish to specify multiple fields as write-only.  Instead of adding each field explicitly with the `write_only=True` attribute, you may use the `write_only_fields` Meta option, like so:
-
-    class CreateUserSerializer(serializers.ModelSerializer):
-        class Meta:
-            model = User
-            fields = ('email', 'username', 'password')
-            write_only_fields = ('password',)  # Note: Password field is write-only
-
-        def restore_object(self, attrs, instance=None):
-            """
-            Instantiate a new User instance.
-            """
-            assert instance is None, 'Cannot update users with CreateUserSerializer'
-            user = User(email=attrs['email'], username=attrs['username'])
-            user.set_password(attrs['password'])
-            return user
-
 ## Specifying fields explicitly
 
 You can add extra fields to a `ModelSerializer` or override the default fields by declaring fields on the class, just as you would for a `Serializer` class.
@@ -405,6 +441,41 @@ You can add extra fields to a `ModelSerializer` or override the default fields b
 
 Extra fields can correspond to any property or callable on the model.
 
+## Specifying which fields should be read-only
+
+You may wish to specify multiple fields as read-only. Instead of adding each field explicitly with the `read_only=True` attribute, you may use the shortcut Meta option, `read_only_fields`.
+
+This option should be a list or tuple of field names, and is declared as follows:
+
+    class AccountSerializer(serializers.ModelSerializer):
+        class Meta:
+            model = Account
+            fields = ('id', 'account_name', 'users', 'created')
+            read_only_fields = ('account_name',)
+
+Model fields which have `editable=False` set, and `AutoField` fields will be set to read-only by default, and do not need to be added to the `read_only_fields` option.
+
+## Specifying additional keyword arguments for fields.
+
+There is also a shortcut allowing you to specify arbitrary additional keyword arguments on fields, using the `extra_kwargs` option. Similarly to `read_only_fields` this means you do not need to explicitly declare the field on the serializer.
+
+This option is a dictionary, mapping field names to a dictionary of keyword arguments. For example:
+
+    class CreateUserSerializer(serializers.ModelSerializer):
+        class Meta:
+            model = User
+            fields = ('email', 'username', 'password')
+            extra_kwargs = {'password': {'write_only': True}}
+            
+        def create(self, validated_data):
+            user = User(
+                email=validated_data['email'],
+                username=validated_data['username']
+            )
+            user.set_password(validated_data['password'])
+            user.save()
+            return user
+
 ## Relational fields
 
 When serializing model instances, there are a number of different ways you might choose to represent relationships.  The default representation for `ModelSerializer` is to use the primary keys of the related instances.
@@ -415,7 +486,7 @@ For full details see the [serializer relations][relations] documentation.
 
 ## Inheritance of the 'Meta' class
 
-The inner `Meta` class on serializers is not inherited from parent classes by default. This is the same behaviour as with Django's `Model` and `ModelForm` classes. If you want the `Meta` class to inherit from a parent class you must do so explicitly. For example:
+The inner `Meta` class on serializers is not inherited from parent classes by default. This is the same behavior as with Django's `Model` and `ModelForm` classes. If you want the `Meta` class to inherit from a parent class you must do so explicitly. For example:
 
     class AccountSerializer(MyBaseSerializer):
         class Meta(MyBaseSerializer.Meta):
@@ -460,8 +531,8 @@ For more specific requirements such as specifying a different lookup for each fi
 
     class AccountSerializer(serializers.HyperlinkedModelSerializer):
         url = serializers.HyperlinkedIdentityField(
-            view_name='account_detail',
+            view_name='account-detail',
-            lookup_field='account_name'
+            lookup_field='slug'
         )
         users = serializers.HyperlinkedRelatedField(
             view_name='user-detail',
@@ -499,8 +570,12 @@ You can also override the URL field's view name and lookup field without overrid
 
 ---
 
+**TODO**: ListSerializer, BaseSerializer, overriding `to_representation` on serializers.
+
 # Advanced serializer usage
 
+**TODO**: Tweak section below
+
 You can create customized subclasses of `ModelSerializer` or `HyperlinkedModelSerializer` that use a different set of default fields.
 
 Doing so should be considered advanced usage, and will only be needed if you have some particular serializer requirements that you often need to repeat.
@@ -528,7 +603,7 @@ For example, if you wanted to be able to set which fields should be used by a se
             # Instantiate the superclass normally
             super(DynamicFieldsModelSerializer, self).__init__(*args, **kwargs)
 
-            if fields:
+            if fields is not None:
                 # Drop any fields that are not specified in the `fields` argument.
                 allowed = set(fields)
                 existing = set(self.fields.keys())
@@ -548,7 +623,9 @@ This would then allow you to do the following:
     >>> print UserSerializer(user, fields=('id', 'email'))
     {'id': 2, 'email': 'jon@example.com'}
 
-## Customising the default fields
+## Customizing the default fields
+
+**TODO**: Remove and note incoming API.
 
 The `field_mapping` attribute is a dictionary that maps model classes to serializer classes.  Overriding the attribute will let you set a different set of default serializer classes.