Merge branch 'master' of github.com:encode/django-rest-framework into update_compat

docs/api-guide/schemas.md

@@ -765,10 +765,21 @@ Valid only if a `location="body"` field is included on the `Link`.
 A short description of the meaning and intended usage of the input field.
 
 
+---
+
+# Third party packages
+
+## DRF OpenAPI
+
+[DRF OpenAPI][drf-openapi] renders the schema generated by Django Rest Framework
+in [OpenAPI][open-api] format.
+
+
 [cite]: https://blog.heroku.com/archives/2014/1/8/json_schema_for_heroku_platform_api
 [coreapi]: http://www.coreapi.org/
 [corejson]: http://www.coreapi.org/specification/encoding/#core-json-encoding
 [open-api]: https://openapis.org/
+[drf-openapi]: https://github.com/limdauto/drf_openapi
 [json-hyperschema]: http://json-schema.org/latest/json-schema-hypermedia.html
 [api-blueprint]: https://apiblueprint.org/
 [static-files]: https://docs.djangoproject.com/en/stable/howto/static-files/

docs/api-guide/serializers.md

@@ -993,7 +993,7 @@ The following class is an example of a generic serializer that can handle coerci
 
 ## Overriding serialization and deserialization behavior
 
-If you need to alter the serialization, deserialization or validation of a serializer class you can do so by overriding the `.to_representation()` or `.to_internal_value()` methods.
+If you need to alter the serialization or deserialization behavior of a serializer class, you can do so by overriding the `.to_representation()` or `.to_internal_value()` methods.
 
 Some reasons this might be useful include...
 
@@ -1011,7 +1011,7 @@ Takes the object instance that requires serialization, and should return a primi
 
 Takes the unvalidated incoming data as input and should return the validated data that will be made available as `serializer.validated_data`. The return value will also be passed to the `.create()` or `.update()` methods if `.save()` is called on the serializer class.
 
-If any of the validation fails, then the method should raise a `serializers.ValidationError(errors)`. Typically the `errors` argument here will be a dictionary mapping field names to error messages.
+If any of the validation fails, then the method should raise a `serializers.ValidationError(errors)`. The `errors` argument should be a dictionary mapping field names (or `settings.NON_FIELD_ERRORS_KEY`) to a list of error messages. If you don't need to alter deserialization behavior and instead want to provide object-level validation, it's recommended that you intead override the [`.validate()`](#object-level-validation) method.
 
 The `data` argument passed to this method will normally be the value of `request.data`, so the datatype it provides will depend on the parser classes you have configured for your API.
 

docs/topics/documenting-your-api.md

@@ -75,6 +75,23 @@ When using viewsets, you should use the relevant action names as delimiters.
 
 There are a number of mature third-party packages for providing API documentation.
 
+#### DRF OpenAPI
+
+[DRF OpenAPI][drf-openapi] bridges the gap between OpenAPI specification and tool chain with the schema exposed
+out-of-the-box by Django Rest Framework. Its goals are:
+
+  * To be dropped into any existing DRF project without any code change necessary.
+  * Provide clear disctinction between request schema and response schema.
+  * Provide a versioning mechanism for each schema. Support defining schema by version range syntax, e.g. >1.0, <=2.0
+  * Support multiple response codes, not just 200
+  * All this information should be bound to view methods, not view classes.
+
+It also tries to stay current with the maturing schema generation mechanism provided by DRF.
+
+![Screenshot - DRF OpenAPI][image-drf-openapi]
+
+---
+
 #### DRF Docs
 
 [DRF Docs][drfdocs-repo] allows you to document Web APIs made with Django REST Framework and it is authored by Emmanouil Konstantinidis. It's made to work out of the box and its setup should not take more than a couple of minutes. Complete documentation can be found on the [website][drfdocs-website] while there is also a [demo][drfdocs-demo] available for people to see what it looks like. **Live API Endpoints** allow you to utilize the endpoints from within the documentation in a neat way.
@@ -197,6 +214,8 @@ In this approach, rather than documenting the available API endpoints up front,
 To implement a hypermedia API you'll need to decide on an appropriate media type for the API, and implement a custom renderer and parser for that media type.  The [REST, Hypermedia & HATEOAS][hypermedia-docs] section of the documentation includes pointers to background reading, as well as links to various hypermedia formats.
 
 [cite]: http://roy.gbiv.com/untangled/2008/rest-apis-must-be-hypertext-driven
+[drf-openapi]: https://github.com/limdauto/drf_openapi/
+[image-drf-openapi]: ../img/drf-openapi.png
 [drfdocs-repo]: https://github.com/ekonstantinidis/django-rest-framework-docs
 [drfdocs-website]: http://www.drfdocs.com/
 [drfdocs-demo]: http://demo.drfdocs.com/

docs/tutorial/6-viewsets-and-routers.md

@@ -26,6 +26,7 @@ Here we've used the `ReadOnlyModelViewSet` class to automatically provide the de
 Next we're going to replace the `SnippetList`, `SnippetDetail` and `SnippetHighlight` view classes.  We can remove the three views, and again replace them with a single class.
 
     from rest_framework.decorators import detail_route
+    from rest_framework.response import Response
 
     class SnippetViewSet(viewsets.ModelViewSet):
         """

rest_framework/compat.py

@@ -252,6 +252,7 @@ try:
         md = markdown.Markdown(
             extensions=extensions, extension_configs=extension_configs
         )
+        md_filter_add_syntax_highlight(md)
         return md.convert(text)
 except ImportError:
     apply_markdown = None
@@ -281,6 +282,38 @@ except ImportError:
     def pygments_css(style):
         return None
 
+if markdown is not None and pygments is not None:
+    # starting from this blogpost and modified to support current markdown extensions API
+    # https://zerokspot.com/weblog/2008/06/18/syntax-highlighting-in-markdown-with-pygments/
+
+    from markdown.preprocessors import Preprocessor
+    import re
+
+    class CodeBlockPreprocessor(Preprocessor):
+        pattern = re.compile(
+            r'^\s*@@ (.+?) @@\s*(.+?)^\s*@@', re.M|re.S)
+
+        formatter = HtmlFormatter()
+
+        def run(self, lines):
+            def repl(m):
+                try:
+                    lexer = get_lexer_by_name(m.group(1))
+                except (ValueError, NameError):
+                    lexer = TextLexer()
+                code = m.group(2).replace('\t','    ')
+                code = pygments.highlight(code, lexer, self.formatter)
+                code = code.replace('\n\n', '\n&nbsp;\n').replace('\n', '<br />').replace('\\@','@')
+                return '\n\n%s\n\n' % code
+            ret = self.pattern.sub(repl, "\n".join(lines))
+            return ret.split("\n")
+
+    def md_filter_add_syntax_highlight(md):
+        md.preprocessors.add('highlight', CodeBlockPreprocessor(), "_begin")
+        return True
+else:
+    def md_filter_add_syntax_highlight(md):
+        return False
 
 try:
     import pytz

rest_framework/templates/rest_framework/docs/document.html

@@ -13,8 +13,8 @@
     {% if 'javascript' in langs %}{% include "rest_framework/docs/langs/javascript-intro.html" %}{% endif %}
 </div>
 </div>
-{% if document.data %}
-{% for section_key, section in document.data|items %}
+{% if document|data %}
+{% for section_key, section in document|data|items %}
 {% if section_key %}
     <h2 id="{{ section_key }}" class="coredocs-section-title">{{ section_key }} <a href="#{{ section_key }}"><i class="fa fa-link" aria-hidden="true"></i>
 </a></h2>

rest_framework/templates/rest_framework/docs/sidebar.html

@@ -5,8 +5,8 @@
     <i class="fa fa-bars fa-2x toggle-btn" data-toggle="collapse" data-target="#menu-content"></i>
     <div class="menu-list">
         <ul id="menu-content" class="menu-content collapse out">
-            {% if document.data %}
-            {% for section_key, section in document.data|items %}
+            {% if document|data %}
+            {% for section_key, section in document|data|items %}
             <li data-toggle="collapse" data-target="#{{ section_key }}-dropdown" class="collapsed">
                 <a><i class="fa fa-dot-circle-o fa-lg"></i> {% if section_key %}{{ section_key }}{% else %}API Endpoints{% endif %} <span class="arrow"></span></a>
                 <ul class="sub-menu {% if section_key %}collapse{% endif %}" id="{{ section_key }}-dropdown">

rest_framework/templatetags/rest_framework.py

@@ -11,7 +11,8 @@ from django.utils.html import escape, format_html, smart_urlquote
 from django.utils.safestring import SafeData, mark_safe
 
 from rest_framework.compat import (
-    NoReverseMatch, markdown, pygments_highlight, reverse, template_render
+    NoReverseMatch, apply_markdown, pygments_highlight, reverse,
+    template_render
 )
 from rest_framework.renderers import HTMLFormRenderer
 from rest_framework.utils.urls import replace_query_param
@@ -68,9 +69,9 @@ def form_for_link(link):
 
 @register.simple_tag
 def render_markdown(markdown_text):
-    if not markdown:
+    if apply_markdown is None:
         return markdown_text
-    return mark_safe(markdown.markdown(markdown_text))
+    return mark_safe(apply_markdown(markdown_text))
 
 
 @register.simple_tag
@@ -244,6 +245,20 @@ def items(value):
     return value.items()
 
 
+@register.filter
+def data(value):
+    """
+    Simple filter to access `data` attribute of object,
+    specifically coreapi.Document.
+
+    As per `items` filter above, allows accessing `document.data` when
+    Document contains Link keyed-at "data".
+
+    See issue #5395
+    """
+    return value.data
+
+
 @register.filter
 def schema_links(section, sec_key=None):
     """

tests/conftest.py

@@ -26,6 +26,9 @@ def pytest_configure():
             {
                 'BACKEND': 'django.template.backends.django.DjangoTemplates',
                 'APP_DIRS': True,
+                'OPTIONS': {
+                    "debug": True,  # We want template errors to raise
+                }
             },
         ],
         MIDDLEWARE=MIDDLEWARE,

tests/test_description.py

@@ -24,11 +24,36 @@ another header
 
 indented
 
-# hash style header #"""
+# hash style header #
+
+@@ json @@
+[{
+    "alpha": 1,
+    "beta: "this is a string"
+}]
+@@"""
 
 # If markdown is installed we also test it's working
 # (and that our wrapped forces '=' to h2 and '-' to h3)
 
+MARKED_DOWN_HILITE = """
+<div class="highlight"><pre><span></span><span \
+class="p">[{</span><br />    <span class="nt">&quot;alpha&quot;</span><span\
+ class="p">:</span> <span class="mi">1</span><span class="p">,</span><br />\
+    <span class="nt">&quot;beta: &quot;</span><span class="err">this</span>\
+ <span class="err">is</span> <span class="err">a</span> <span class="err">\
+string&quot;</span><br /><span class="p">}]</span><br /></pre></div>
+
+<p><br /></p>"""
+
+MARKED_DOWN_NOT_HILITE = """
+<p>@@ json @@
+[{
+    "alpha": 1,
+    "beta: "this is a string"
+}]
+@@</p>"""
+
 # We support markdown < 2.1 and markdown >= 2.1
 MARKED_DOWN_lt_21 = """<h2>an example docstring</h2>
 <ul>
@@ -39,7 +64,7 @@ MARKED_DOWN_lt_21 = """<h2>an example docstring</h2>
 <pre><code>code block
 </code></pre>
 <p>indented</p>
-<h2 id="hash_style_header">hash style header</h2>"""
+<h2 id="hash_style_header">hash style header</h2>%s"""
 
 MARKED_DOWN_gte_21 = """<h2 id="an-example-docstring">an example docstring</h2>
 <ul>
@@ -50,7 +75,7 @@ MARKED_DOWN_gte_21 = """<h2 id="an-example-docstring">an example docstring</h2>
 <pre><code>code block
 </code></pre>
 <p>indented</p>
-<h2 id="hash-style-header">hash style header</h2>"""
+<h2 id="hash-style-header">hash style header</h2>%s"""
 
 
 class TestViewNamesAndDescriptions(TestCase):
@@ -78,7 +103,14 @@ class TestViewNamesAndDescriptions(TestCase):
 
             indented
 
-            # hash style header #"""
+            # hash style header #
+
+            @@ json @@
+            [{
+                "alpha": 1,
+                "beta: "this is a string"
+            }]
+            @@"""
 
         assert MockView().get_view_description() == DESCRIPTION
 
@@ -118,8 +150,16 @@ class TestViewNamesAndDescriptions(TestCase):
         Ensure markdown to HTML works as expected.
         """
         if apply_markdown:
-            gte_21_match = apply_markdown(DESCRIPTION) == MARKED_DOWN_gte_21
-            lt_21_match = apply_markdown(DESCRIPTION) == MARKED_DOWN_lt_21
+            gte_21_match = (
+                apply_markdown(DESCRIPTION) == (
+                    MARKED_DOWN_gte_21 % MARKED_DOWN_HILITE) or
+                apply_markdown(DESCRIPTION) == (
+                    MARKED_DOWN_gte_21 % MARKED_DOWN_NOT_HILITE))
+            lt_21_match = (
+                apply_markdown(DESCRIPTION) == (
+                    MARKED_DOWN_lt_21 % MARKED_DOWN_HILITE) or
+                apply_markdown(DESCRIPTION) == (
+                    MARKED_DOWN_lt_21 % MARKED_DOWN_NOT_HILITE))
             assert gte_21_match or lt_21_match
 
 

tests/test_renderers.py

@@ -14,9 +14,10 @@ from django.utils import six
 from django.utils.safestring import SafeText
 from django.utils.translation import ugettext_lazy as _
 
+import coreapi
 from rest_framework import permissions, serializers, status
 from rest_framework.renderers import (
-    AdminRenderer, BaseRenderer, BrowsableAPIRenderer,
+    AdminRenderer, BaseRenderer, BrowsableAPIRenderer, DocumentationRenderer,
     HTMLFormRenderer, JSONRenderer, StaticHTMLRenderer
 )
 from rest_framework.request import Request
@@ -706,3 +707,32 @@ class AdminRendererTests(TestCase):
         response = view(request)
         response.render()
         self.assertInHTML('<tr><th>Iteritems</th><td>a string</td></tr>', str(response.content))
+
+
+class TestDocumentationRenderer(TestCase):
+
+    def test_document_with_link_named_data(self):
+        """
+        Ref #5395: Doc's `document.data` would fail with a Link named "data".
+            As per #4972, use templatetag instead.
+        """
+        document = coreapi.Document(
+            title='Data Endpoint API',
+            url='https://api.example.org/',
+            content={
+                'data': coreapi.Link(
+                    url='/data/',
+                    action='get',
+                    fields=[],
+                    description='Return data.'
+                )
+            }
+        )
+
+        factory = APIRequestFactory()
+        request = factory.get('/')
+
+        renderer = DocumentationRenderer()
+
+        html = renderer.render(document, accepted_media_type="text/html", renderer_context={"request": request})
+        assert '<h1>Data Endpoint API</h1>' in html

tests/urls.py

@@ -1,4 +1,11 @@
 """
-Blank URLConf just to keep the test suite happy
+URLConf for test suite.
+
+We need only the docs urls for DocumentationRenderer tests.
 """
-urlpatterns = []
+from django.conf.urls import url
+from rest_framework.documentation import include_docs_urls
+
+urlpatterns = [
+    url(r'^docs/', include_docs_urls(title='Test Suite API')),
+]