From 08ecffe107fbee60bb53502d16665178547feebb Mon Sep 17 00:00:00 2001 From: Andrew Godwin Date: Wed, 8 Jun 2016 23:14:45 +0000 Subject: [PATCH] Update ASGI spec with single-reader channel --- docs/asgi.rst | 69 +++++++++++++++++++++++++++++++++++---------------- 1 file changed, 47 insertions(+), 22 deletions(-) diff --git a/docs/asgi.rst b/docs/asgi.rst index c37fadc..c4e59d5 100644 --- a/docs/asgi.rst +++ b/docs/asgi.rst @@ -100,7 +100,7 @@ contain only the following types to ensure serializability: Channels are identified by a unicode string name consisting only of ASCII letters, ASCII numerical digits, periods (``.``), dashes (``-``) and -underscores (``_``), plus an optional prefix character (see below). +underscores (``_``), plus an optional type character (see below). Channels are a first-in, first out queue with at-most-once delivery semantics. They can have multiple writers and multiple readers; only a single @@ -111,8 +111,26 @@ this is necessary to achieve this restriction. In order to aid with scaling and network architecture, a distinction is made between channels that have multiple readers (such as the ``http.request`` channel that web applications would listen on from every -application worker process) and *process-specific channels* -(such as a ``http.response!ABCDEF`` channel tied to a client socket). +application worker process), *single-reader channels* that are read from a +single unknown location (such as ``http.request.body?ABCDEF``), and +*process-specific channels* (such as a ``http.response!ABCDEF`` channel +tied to a client socket). + +*Normal channel* names contain no type characters, and can be routed however +the backend wishes; in particular, they do not have to appear globally +consistent, and backends may shard their contents out to different servers +so that a querying client only sees some portion of the messages. Calling +``receive_many`` on these channels does not guarantee that you will get the +messages in order or that you will get anything if the channel is non-empty. + +*Single-reader channel* names contain an exclamation mark +(``?``) character in order to indicate to the channel layer that it must make +these channels appear globally consistent. The ``?`` is always preceded by +the main channel name (e.g. ``http.response.body``) and followed by a +random portion. Channel layers may use the random portion to help pin the +channel to a server, but reads from this channel by a single process must +always be in-order and return messages if the channel is non-empty. These names +must be generated by the ``new_channel`` call. *Process-specific channel* names contain an exclamation mark (``!``) character in order to indicate to the channel layer that it may @@ -124,16 +142,19 @@ per-client/random portion - channel layers can split on the ``!`` and use just the right hand part to route if they desire, or can ignore it if they don't need to use different routing rules. Even if the right hand side contains client routing information, it must still contain random parts too so that -each call to ``new_channel`` returns a new, unused name. +each call to ``new_channel`` returns a new, unused name. These names +must be generated by the ``new_channel`` call; they are guaranteed to only +be read from the same process that calls ``new_channel``. Messages should expire after a set time sitting unread in a channel; the recommendation is one minute, though the best value depends on the channel layer and the way it is deployed. -The maximum message size is 1MB; if more data than this needs to be transmitted -it must be chunked or placed onto its own process-specific channel (see how -HTTP request bodies are done, for example). All channel layers must support -messages up to this size. +The maximum message size is 1MB if the message were encoded as JSON; +if more data than this needs to be transmitted it must be chunked or placed +onto its own single-reader or process-specific channel (see how HTTP request +bodies are done, for example). All channel layers must support messages up +to this size, but protocol specifications are encouraged to keep well below it. Handling Protocols @@ -253,9 +274,9 @@ and the vast majority of application code will not need to deal with this problem. If ordering of incoming packets matters for a protocol, they should be annotated with a packet number (as WebSocket is in this specification). -Single-reader channels, such as those used for response channels back to -clients, are not subject to this problem; a single reader must always -receive messages in channel order. +Single-reader and process-specific channels, such as those used for response +channels back to clients, are not subject to this problem; a single reader +on these must always receive messages in channel order. Capacity @@ -299,14 +320,13 @@ A *channel layer* must provide an object with these attributes * ``new_channel(pattern)``, a callable that takes a unicode string pattern, and returns a new valid channel name that does not already exist, by - adding a unicode string after the ``!`` character in ``pattern``, + adding a unicode string after the ``!`` or ``?`` character in ``pattern``, and checking for existence of that name in the channel layer. The ``pattern`` - MUST end with ``!`` or this function must error. This is not always called - prior to a message being sent on a channel, and cannot be used for - channel initialization. ``new_channel`` must be called on the same channel - layer that intends to read the channel with ``receive_many``; any other - channel layer instance may not receive messages on this channel due to - client-routing portions of the appended string. + MUST end with ``!`` or ``?`` or this function must error. If the character + is ``!``, making it a process-specific channel, ``new_channel`` must be + called on the same channel layer that intends to read the channel with + ``receive_many``; any other channel layer instance may not receive + messages on this channel due to client-routing portions of the appended string. * ``MessageTooLarge``, the exception raised when a send operation fails because the encoded message is over the layer's size limit. @@ -366,7 +386,7 @@ Channel Semantics Channels **must**: * Preserve ordering of messages perfectly with only a single reader - and writer, and preserve as much as possible in other cases. + and writer if the channel is a *single-reader* or *process-specific* channel. * Never deliver a message more than once. @@ -379,6 +399,9 @@ Channels **must**: * Have a maximum name length of at least 100 bytes. +They should attempt to preserve ordering in all cases as much as possible, +but perfect global ordering is obviously not possible in the distributed case. + They are not expected to deliver all messages, but a success rate of at least 99.99% is expected under normal circumstances. Implementations may want to have a "resilience testing" mode where they deliberately drop more messages @@ -520,7 +543,7 @@ Keys: If ``more_body`` is set, treat as start of body and concatenate on further chunks. -* ``more_body``: Channel name that contains +* ``more_body``: Name of a single-reader channel (containing ``?``) that contains Request Body Chunk messages representing a large request body. Optional, defaults to ``None``. Chunks append to ``body`` if set. Presence of a channel indicates at least one Request Body Chunk message needs to be read, @@ -541,7 +564,7 @@ Request Body Chunk Must be sent after an initial Response. -Channel: ``http.request.body!`` +Channel: ``http.request.body?`` Keys: @@ -845,7 +868,9 @@ limitation that they only use the following characters: * Hyphen ``-`` * Underscore ``_`` * Period ``.`` -* Exclamation mark ``!`` (only to deliniate process-specific channel names, +* Question mark ``?`` (only to delineiate single-reader channel names, + and only one per name) +* Exclamation mark ``!`` (only to delineate process-specific channel names, and only one per name)