Schema docs, pagination controls, filter controls

docs/api-guide/schemas.md

@@ -0,0 +1,360 @@
+source: schemas.py
+
+# Schemas
+
+> A machine-readable [schema] describes what resources are available via the API, what their URLs are, how they are represented and what operations they support.
+>
+> — Heroku, [JSON Schema for the Heroku Platform API][cite]
+
+API schemas are a useful tool that allow for a range of use cases, including
+generating reference documentation, or driving dynamic client libraries that
+can interact with your API.
+
+## Representing schemas internally
+
+REST framework uses [Core API][coreapi] in order to model schema information in
+a format-independent representation. This information can then be rendered
+into various different schema formats, or used to generate API documentation.
+
+When using Core API, a schema is represented as a `Document` which is the
+top-level container object for information about the API. Available API
+interactions are represented using `Link` objects. Each link includes a URL,
+HTTP method, and may include a list of `Field` instances, which describe any
+parameters that may be accepted by the API endpoint. The `Link` and `Field`
+instances may also include descriptions, that allow an API schema to be
+rendered into user documentation.
+
+Here's an example of an API description that includes a single `search`
+endpoint:
+
+    coreapi.Document(
+        title='Flight Search API',
+        url='https://api.example.org/',
+        content={
+            'search': coreapi.Link(
+                url='/search/',
+                action='get',
+                fields=[
+                    coreapi.Field(
+                        name='from',
+                        required=True,
+                        location='query',
+                        description='City name or airport code.'
+                    ),
+                    coreapi.Field(
+                        name='to',
+                        required=True,
+                        location='query',
+                        description='City name or airport code.'
+                    ),
+                    coreapi.Field(
+                        name='date',
+                        required=True,
+                        location='query',
+                        description='Flight date in "YYYY-MM-DD" format.'
+                    )
+                ],
+                description='Return flight availability and prices.'
+            )
+        }
+    )
+
+## Schema output formats
+
+In order to be presented in an HTTP response, the internal representation
+has to be rendered into the actual bytes that are used in the response.
+
+[Core JSON][corejson] is designed as a canonical format for use with Core API.
+REST framework includes a renderer class for handling this media type, which
+is available as `renderers.CoreJSONRenderer`.
+
+Other schema formats such as [Open API][open-api] (Formerly "Swagger"),
+[JSON HyperSchema][json-hyperschema], or [API Blueprint][api-blueprint] can
+also be supported by implementing a custom renderer class.
+
+## Schemas vs Hypermedia
+
+It's worth pointing out here that Core API can also be used to model hypermedia
+responses, which present an alternative interaction style to API schemas.
+
+With an API schema, the entire available interface is presented up-front
+as a single endpoint. Responses to individual API endpoints are then typically
+presented as plain data, without any further interactions contained in each
+response.
+
+With Hypermedia, the client is instead presented with a document containing
+both data and available interactions. Each interaction results in a new
+document, detailing both the current state and the available interactions.
+
+Further information and support on building Hypermedia APIs with REST framework
+is planned for a future version.
+
+---
+
+# Adding a schema
+
+You'll need to install the `coreapi` package in order to add schema support
+for REST framework.
+
+    pip install coreapi
+
+REST framework includes functionality for auto-generating a schema,
+or allows you to specify one explicitly. There are a few different ways to
+add a schema to your API, depending on exactly what you need.
+
+## Using DefaultRouter
+
+If you're using `DefaultRouter` then you can include an auto-generated schema,
+simply by adding a `schema_title` argument to the router.
+
+    router = DefaultRouter(schema_title='Server Monitoring API')
+
+The schema will be included in by the root URL, `/`, and presented to clients
+that include the Core JSON media type in their `Accept` header.
+
+    $ http http://127.0.0.1:8000/ Accept:application/vnd.coreapi+json
+    HTTP/1.0 200 OK
+    Allow: GET, HEAD, OPTIONS
+    Content-Type: application/vnd.coreapi+json
+
+    {
+        "_meta": {
+            "title": "Server Monitoring API"
+        },
+        "_type": "document",
+        ...
+    }
+
+This is a great zero-configuration option for when you want to get up and
+running really quickly. If you want a little more flexibility over the
+schema output then you'll need to consider using `SchemaGenerator` instead.
+
+## Using SchemaGenerator
+
+The most common way to add a schema to your API is to use the `SchemaGenerator`
+class to auto-generate the `Document` instance, and to return that from a view.
+
+This option gives you the flexibility of setting up the schema endpoint
+with whatever behaviour you want. For example, you can apply different
+permission, throttling or authentication policies to the schema endpoint.
+
+Here's an example of using `SchemaGenerator` together with a view to
+return the schema.
+
+**views.py:**
+
+    from rest_framework.decorators import api_view, renderer_classes
+    from rest_framework import renderers, schemas
+
+    generator = schemas.SchemaGenerator(title='Bookings API')
+
+    @api_view()
+    @renderer_classes([renderers.CoreJSONRenderer])
+    def schema_view(request):
+        return generator.get_schema()
+
+**urls.py:**
+
+    urlpatterns = [
+        url('/', schema_view),
+        ...
+    ]
+
+You can also serve different schemas to different users, depending on the
+permissions they have available. This approach can be used to ensure that
+unauthenticated requests are presented with a different schema to
+authenticated requests, or to ensure that different parts of the API are
+made visible to different users depending on their role.
+
+In order to present a schema with endpoints filtered by user permissions,
+you need to pass the `request` argument to the `get_schema()` method, like so:
+
+    @api_view()
+    @renderer_classes([renderers.CoreJSONRenderer])
+    def schema_view(request):
+        return generator.get_schema(request=request)
+
+## Explicit schema definition
+
+An alternative to the auto-generated approach is to specify the API schema
+explicitly, by declaring a `Document` object in your codebase. Doing so is a
+little more work, but ensures that you have full control over the schema
+representation.
+
+    import coreapi
+    from rest_framework.decorators import api_view, renderer_classes
+    from rest_framework import renderers
+
+    schema = coreapi.Document(
+        title='Bookings API',
+        content={
+            ...
+        }
+    )
+
+    @api_view()
+    @renderer_classes([renderers.CoreJSONRenderer])
+    def schema_view(request):
+        return schema
+
+## Static schema file
+
+A final option is to write your API schema as a static file, using one
+of the available formats, such as Core JSON or Open API.
+
+You could then either:
+
+* Write a schema definition as a static file, and [serve the static file directly][static-files].
+* Write a schema definition that is loaded using `Core API`, and then
+  rendered to one of many available formats, depending on the client request.
+
+---
+
+# API Reference
+
+## SchemaGenerator
+
+A class that deals with introspecting your API views, which can be used to
+generate a schema.
+
+Typically you'll instantiate `SchemaGenerator` with a single argument, like so:
+
+    generator = SchemaGenerator(title='Stock Prices API')
+
+Arguments:
+
+* `title` - The name of the API. **required**
+* `patterns` - A list of URLs to inspect when generating the schema. Defaults to the project's URL conf.
+* `urlconf` - A URL conf module name to use when generating the schema. Defaults to `settings.ROOT_URLCONF`.
+
+### get_schema()
+
+Returns a `coreapi.Document` instance that represents the API schema.
+
+    @api_view
+    @renderer_classes([renderers.CoreJSONRenderer])
+    def schema_view(request):
+        return generator.get_schema()
+
+Arguments:
+
+* `request` - The incoming request. Optionally used if you want to apply per-user permissions to the schema-generation.
+
+---
+
+## Core API
+
+This documentation gives a brief overview of the components within the `coreapi`
+package that are used to represent an API schema.
+
+Note that these classes are imported from the `coreapi` package, rather than
+from the `rest_framework` package.
+
+### Document
+
+Represents a container for the API schema.
+
+#### `title`
+
+A name for the API.
+
+#### `url`
+
+A canonical URL for the API.
+
+#### `content`
+
+A dictionary, containing the `Link` objects that the schema contains.
+
+In order to provide more structure to the schema, the `content` dictionary
+may be nested, typically to a second level. For example:
+
+    content={
+        "bookings": {
+            "list": Link(...),
+            "create": Link(...),
+            ...
+        },
+        "venues": {
+            "list": Link(...),
+            ...
+        },
+        ...
+    }
+
+### Link
+
+Represents an individual API endpoint.
+
+#### `url`
+
+The URL of the endpoint. May be a URI template, such as `/users/{username}/`.
+
+#### `action`
+
+The HTTP method associated with the endpoint. Note that URLs that support
+more than one HTTP method, should correspond to a single `Link` for each.
+
+#### `fields`
+
+A list of `Field` instances, describing the available parameters on the input.
+
+#### `description`
+
+A short description of the meaning and intended usage of the endpoint.
+
+### Field
+
+Represents a single input parameter on a given API endpoint.
+
+#### `name`
+
+A descriptive name for the input.
+
+#### `required`
+
+A boolean, indicated if the client is required to included a value, or if
+the parameter can be omitted.
+
+#### `location`
+
+Determines how the information is encoded into the request. Should be one of
+the following strings:
+
+**"path"**
+
+Included in a templated URI. For example a `url` value of `/products/{product_code}/` could be used together with a `"path"` field, to handle API inputs in a URL path such as `/products/slim-fit-jeans/`.
+
+These fields will normally correspond with [named arguments in the project URL conf][named-arguments].
+
+**"query"**
+
+Included as a URL query parameter. For example `?search=sale`. Typically for `GET` requests.
+
+These fields will normally correspond with pagination and filtering controls on a view.
+
+**"form"**
+
+Included in the request body, as a single item of a JSON object or HTML form. For example `{"colour": "blue", ...}`. Typically for `POST`, `PUT` and `PATCH` requests. Multiple `"form"` fields may be included on a single link.
+
+These fields will normally correspond with serializer fields on a view.
+
+**"body"**
+
+Included as the complete request body. Typically for `POST`, `PUT` and `PATCH` requests. No more than one `"body"` field may exist on a link. May not be used together with `"form"` fields.
+
+These fields will normally correspond with views that use `ListSerializer` to validate the request input, or with file upload views.
+
+#### `description`
+
+A short description of the meaning and intended usage of the input field.
+
+
+[cite]: https://blog.heroku.com/archives/2014/1/8/json_schema_for_heroku_platform_api
+[coreapi]: http://www.coreapi.org/
+[corejson]: http://www.coreapi.org/specification/encoding/#core-json-encoding
+[open-api]: https://openapis.org/
+[json-hyperschema]: http://json-schema.org/latest/json-schema-hypermedia.html
+[api-blueprint]: https://apiblueprint.org/
+[static-files]: https://docs.djangoproject.com/en/dev/howto/static-files/
+[named-arguments]: https://docs.djangoproject.com/en/dev/topics/http/urls/#named-groups

docs/index.md

@@ -191,6 +191,7 @@ The API guide is your complete reference manual to all the functionality provide
 * [Versioning][versioning]
 * [Content negotiation][contentnegotiation]
 * [Metadata][metadata]
+* [Schemas][schemas]
 * [Format suffixes][formatsuffixes]
 * [Returning URLs][reverse]
 * [Exceptions][exceptions]
@@ -314,6 +315,7 @@ OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 [versioning]: api-guide/versioning.md
 [contentnegotiation]: api-guide/content-negotiation.md
 [metadata]: api-guide/metadata.md
+[schemas]: 'api-guide/schemas.md'
 [formatsuffixes]: api-guide/format-suffixes.md
 [reverse]: api-guide/reverse.md
 [exceptions]: api-guide/exceptions.md

mkdocs.yml

@@ -42,6 +42,7 @@ pages:
     - 'Versioning': 'api-guide/versioning.md'
     - 'Content negotiation': 'api-guide/content-negotiation.md'
     - 'Metadata': 'api-guide/metadata.md'
+    - 'Schemas': 'api-guide/schemas.md'
     - 'Format suffixes': 'api-guide/format-suffixes.md'
     - 'Returning URLs': 'api-guide/reverse.md'
     - 'Exceptions': 'api-guide/exceptions.md'

rest_framework/filters.py

@@ -71,6 +71,9 @@ class BaseFilterBackend(object):
         """
         raise NotImplementedError(".filter_queryset() must be overridden.")
 
+    def get_fields(self):
+        return []
+
 
 class DjangoFilterBackend(BaseFilterBackend):
     """
@@ -127,6 +130,17 @@ class DjangoFilterBackend(BaseFilterBackend):
         template = loader.get_template(self.template)
         return template_render(template, context)
 
+    def get_fields(self):
+        filter_class = getattr(view, 'filter_class', None)
+        if filter_class:
+            return list(filter_class().filters.keys())
+
+        filter_fields = getattr(view, 'filter_fields', None)
+        if filter_fields:
+            return filter_fields
+
+        return []
+
 
 class SearchFilter(BaseFilterBackend):
     # The URL query parameter used for the search.
@@ -191,6 +205,9 @@ class SearchFilter(BaseFilterBackend):
         template = loader.get_template(self.template)
         return template_render(template, context)
 
+    def get_fields(self):
+        return [self.search_param]
+
 
 class OrderingFilter(BaseFilterBackend):
     # The URL query parameter used for the ordering.
@@ -304,6 +321,9 @@ class OrderingFilter(BaseFilterBackend):
         context = self.get_template_context(request, queryset, view)
         return template_render(template, context)
 
+    def get_fields(self):
+        return [self.ordering_param]
+
 
 class DjangoObjectPermissionsFilter(BaseFilterBackend):
     """

rest_framework/pagination.py

@@ -157,6 +157,9 @@ class BasePagination(object):
     def get_results(self, data):
         return data['results']
 
+    def get_fields(self):
+        return []
+
 
 class PageNumberPagination(BasePagination):
     """
@@ -280,6 +283,11 @@ class PageNumberPagination(BasePagination):
         context = self.get_html_context()
         return template_render(template, context)
 
+    def get_fields(self):
+        if self.page_size_query_param is None:
+            return [self.page_query_param]
+        return [self.page_query_param, self.page_size_query_param]
+
 
 class LimitOffsetPagination(BasePagination):
     """
@@ -404,6 +412,10 @@ class LimitOffsetPagination(BasePagination):
         context = self.get_html_context()
         return template_render(template, context)
 
+    def get_fields(self):
+        return [self.limit_query_param, self.offset_query_param]
+
+
 
 class CursorPagination(BasePagination):
     """
@@ -706,3 +718,6 @@ class CursorPagination(BasePagination):
         template = loader.get_template(self.template)
         context = self.get_html_context()
         return template_render(template, context)
+
+    def get_fields(self):
+        return [self.cursor_query_param]

rest_framework/routers.py

@@ -270,6 +270,7 @@ class DefaultRouter(SimpleRouter):
     include_root_view = True
     include_format_suffixes = True
     root_view_name = 'api-root'
+    schema_renderers = [renderers.CoreJSONRenderer]
 
     def __init__(self, *args, **kwargs):
         self.schema_title = kwargs.pop('schema_title', None)
@@ -287,20 +288,29 @@ class DefaultRouter(SimpleRouter):
         view_renderers = list(api_settings.DEFAULT_RENDERER_CLASSES)
 
         if schema_urls and self.schema_title:
-            view_renderers += [renderers.CoreJSONRenderer]
+            view_renderers += list(self.schema_renderers)
-            schema_generator = SchemaGenerator(patterns=schema_urls)
+            schema_generator = SchemaGenerator(
+                title=self.schema_title,
+                patterns=schema_urls
+            )
+            schema_media_types = [
+                renderer.media_type
+                for renderer in self.schema_renderers
+            ]
 
         class APIRoot(views.APIView):
             _ignore_model_permissions = True
             renderer_classes = view_renderers
 
             def get(self, request, *args, **kwargs):
-                if request.accepted_renderer.format == 'corejson':
+                if request.accepted_renderer.media_type in schema_media_types:
+                    # Return a schema response.
                     schema = schema_generator.get_schema(request)
                     if schema is None:
                         raise exceptions.PermissionDenied()
                     return Response(schema)
 
+                # Return a plain {"name": "hyperlink"} response.
                 ret = OrderedDict()
                 namespace = request.resolver_match.namespace
                 for key, url_name in api_root_dict.items():

rest_framework/schemas.py

@@ -11,6 +11,32 @@ from rest_framework.request import clone_request
 from rest_framework.views import APIView
 
 
+def is_api_view(callback):
+    """
+    Return `True` if the given view callback is a REST framework view/viewset.
+    """
+    cls = getattr(callback, 'cls', None)
+    return (cls is not None) and issubclass(cls, APIView)
+
+
+def insert_into(target, keys, item):
+    """
+    Insert `item` into the nested dictionary `target`.
+
+    For example:
+
+        target = {}
+        insert_into(target, ('users', 'list'), Link(...))
+        insert_into(target, ('users', 'detail'), Link(...))
+        assert target == {'users': {'list': Link(...), 'detail': Link(...)}}
+    """
+    for key in keys[:1]:
+        if key not in target:
+            target[key] = {}
+        target = target[key]
+    target[keys[-1]] = item
+
+
 class SchemaGenerator(object):
     default_mapping = {
         'get': 'read',
@@ -20,7 +46,7 @@ class SchemaGenerator(object):
         'delete': 'destroy',
     }
 
-    def __init__(self, schema_title=None, patterns=None, urlconf=None):
+    def __init__(self, title=None, patterns=None, urlconf=None):
         assert coreapi, '`coreapi` must be installed for schema support.'
 
         if patterns is None and urlconf is not None:
@@ -33,7 +59,7 @@ class SchemaGenerator(object):
             urls = import_module(settings.ROOT_URLCONF)
             patterns = urls.urlpatterns
 
-        self.schema_title = schema_title
+        self.title = title
         self.endpoints = self.get_api_endpoints(patterns)
 
     def get_schema(self, request=None):
@@ -61,15 +87,10 @@ class SchemaGenerator(object):
         # ('users', 'list'), Link -> {'users': {'list': Link()}}
         content = {}
         for key, link, callback in endpoints:
-            insert_into = content
+            insert_into(content, key, link)
-            for item in key[:1]:
-                if item not in insert_into:
-                    insert_into[item] = {}
-                insert_into = insert_into[item]
-            insert_into[key[-1]] = link
 
         # Return the schema document.
-        return coreapi.Document(title=self.schema_title, content=content)
+        return coreapi.Document(title=self.title, content=content)
 
     def get_api_endpoints(self, patterns, prefix=''):
         """
@@ -83,7 +104,7 @@ class SchemaGenerator(object):
             if isinstance(pattern, RegexURLPattern):
                 path = self.get_path(path_regex)
                 callback = pattern.callback
-                if self.include_endpoint(path, callback):
+                if self.should_include_endpoint(path, callback):
                     for method in self.get_allowed_methods(callback):
                         key = self.get_key(path, method, callback)
                         link = self.get_link(path, method, callback)
@@ -107,19 +128,18 @@ class SchemaGenerator(object):
         path = path.replace('<', '{').replace('>', '}')
         return path
 
-    def include_endpoint(self, path, callback):
+    def should_include_endpoint(self, path, callback):
         """
-        Return True if the given endpoint should be included.
+        Return `True` if the given endpoint should be included.
         """
-        cls = getattr(callback, 'cls', None)
+        if not is_api_view(callback):
-        if (cls is None) or not issubclass(cls, APIView):
+            return False  # Ignore anything except REST framework views.
-            return False
 
         if path.endswith('.{format}') or path.endswith('.{format}/'):
-            return False
+            return False  # Ignore .json style URLs.
 
         if path == '/':
-            return False
+            return False  # Ignore the root endpoint.
 
         return True
 
@@ -153,25 +173,77 @@ class SchemaGenerator(object):
             return (category, action)
         return (action,)
 
+    # Methods for generating each individual `Link` instance...
+
     def get_link(self, path, method, callback):
         """
         Return a `coreapi.Link` instance for the given endpoint.
         """
         view = callback.cls()
+        fields = self.get_path_fields(path, method, callback, view)
+        fields += self.get_serializer_fields(path, method, callback, view)
+        fields += self.get_pagination_fields(path, method, callback, view)
+        fields += self.get_filter_fields(path, method, callback, view)
+        return coreapi.Link(url=path, action=method.lower(), fields=fields)
+
+    def get_path_fields(self, path, method, callback, view):
+        """
+        Return a list of `coreapi.Field` instances corresponding to any
+        templated path variables.
+        """
         fields = []
 
         for variable in uritemplate.variables(path):
             field = coreapi.Field(name=variable, location='path', required=True)
             fields.append(field)
 
-        if method in ('PUT', 'PATCH', 'POST'):
+        return fields
-            serializer_class = view.get_serializer_class()
-            serializer = serializer_class()
-            for field in serializer.fields.values():
-                if field.read_only:
-                    continue
-                required = field.required and method != 'PATCH'
-                field = coreapi.Field(name=field.source, location='form', required=required)
-                fields.append(field)
 
-        return coreapi.Link(url=path, action=method.lower(), fields=fields)
+    def get_serializer_fields(self, path, method, callback, view):
+        """
+        Return a list of `coreapi.Field` instances corresponding to any
+        request body input, as determined by the serializer class.
+        """
+        if method not in ('PUT', 'PATCH', 'POST'):
+            return []
+
+        fields = []
+
+        serializer_class = view.get_serializer_class()
+        serializer = serializer_class()
+        for field in serializer.fields.values():
+            if field.read_only:
+                continue
+            required = field.required and method != 'PATCH'
+            field = coreapi.Field(name=field.source, location='form', required=required)
+            fields.append(field)
+
+        return fields
+
+    def get_pagination_fields(self, path, method, callback, view):
+        if method != 'GET':
+            return []
+
+        if hasattr(callback, 'actions') and ('list' not in callback.actions.values()):
+            return []
+
+        if not hasattr(view, 'pagination_class'):
+            return []
+
+        paginator = view.pagination_class()
+        return paginator.get_fields()
+
+    def get_filter_fields(self, path, method, callback, view):
+        if method != 'GET':
+            return []
+
+        if hasattr(callback, 'actions') and ('list' not in callback.actions.values()):
+            return []
+
+        if not hasattr(view, 'filter_backends'):
+            return []
+
+        fields = []
+        for filter_backend in view.filter_backends:
+            fields += filter_backend().get_fields()
+        return fields