An HTTP/2 extension for bidirectional messaging communication

Introduction HTTP/2 is the de facto application protocol in Internet. The optimizations developed in HTTP/2, like stream multiplexing, header compression, and binary message framing are very generic. They can be useful in non web browsing applications, for example, Publish/Subscribe, RPC. However, the request/response from client to server communication pattern limits HTTP/2 from wider use in these applications. This draft proposes a HTTP/2 protocol extension, which enables bidirectional messaging between client and server. The only mechanism HTTP/2 provides for server to client communication is PUSHPROMISE. While this satisfies some use-cases, it is unidirectional, i.g. the client cannot respond. In this draft, a new frame is introduced which has the routing properties of PUSHPROMISE and the bi-directionality of HEADERS. Further, clients are also able group streams together for routing purposes, such that each individual stream does not need to carry additional routing information.

Conventions and Terminology The keywords MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT, SHOULD, SHOULD NOT, RECOMMENDED, MAY, and OPTIONAL, when they appear in this document, are to be interpreted as described in . All the terms defined in the Conventions and Terminology section in apply to this document.

Solution Overview

Routing Stream and ExStream A routing stream (RStream) is a long lived HTTP/2 stream in nature. RStreams are initiated by clients, and can be routed independently by any intermediaries. Though an RStream is effectively a regular HTTP/2 stream, RStreams are recommended for exchanging metadata, but not user data. A new HTTP/2 stream called ExStream is introduced for exchanging user data. ExStreams are recommended for short lived transactions, so intermediaries and servers can gracefully shutdown ExStreams within a short time. The typical use case can be a subscription or publish request/response in Publish/Subscribe use case, or an RPC call between two endpoints. An ExStream is opened by an EX_HEADERS frame, and continued by CONTINUATION and DATA frames. An ExStream MUST be associated with an open RStream, and MUST NOT be associated with any other ExStream. ExStreams are routed according to their RStreams by intermediaries and servers. Effectively, all ExStreams with the same RStream form a logical stream group, and are routed to the same endpoint.

Bidirectional Messaging Communication With RStreams and ExStreams, HTTP/2 can be used for bidirectional messaging communication. As shown in the follow diagrams, after an RStream is open from client to server, either endpoint can initiate an ExStreams to its peer.

Server initiates the ExStream to client, after an RStream is open. +--------+ RStream (5) +---------+ RStream (1) +--------+ | client |>--------------->| proxy |>---------------->| server | +--------+ +---------+ +--------+ ^ v ^ v | ExStream(4, RS=5) | | ExStream(2, RS=1) | +------------------------+ +------------------------+ Beyond that, clients can multiplex RStreams, ExStreams and regular HTTP/2 streams into one HTTP/2 connection. This enables clients to access different services without initiating new TCP connections. This avoids the latency cost of setting up new connections. This is more desirable for mobile devices because they usually have longer RTT and battery constraints. Multiplexing these services also allows them to share a single TCP connection congestion control context. As shown in the following diagram, the client can exchange data with PubSub, RPC and CDN three different services with one TCP connection.

Client opens multiple RStreams and a HTTP/2 stream within one HTTP/2 connection. +--------+ RStream (5) +---------+ RStream (1) +----------+ | client |>--------------->| proxy |>---------------->| PUBSUB | +--------+ +---------+ +----------+ v v ^ ^ v v | | RStream (7) / | | \ RStream (5) +----------+ | +-------------------+ | | +---------------->| RPC | | | | +----------+ | | | | Stream (9) | | Stream (7) +----------+ +---------------------------+ +--------------------->| CDN | +----------+

States of RStream and ExStream RStreams are regular HTTP/2 streams that follow the stream lifecycle in , section 5.1. ExStreams use the same lifecycle as regular HTTP/2 streams, but also depend on their RStreams. If a RStream is reset, endpoints MUST reset the ExStreams associated with that RStream. If the RStream is closed, endpoints SHOULD allow existing ExStreams complete normally. The RStream SHOULD remain open while communication is ongoing. Endpoints SHOULD refresh any timeouts on the RStream while associated ExStreams are open. A sender MUST NOT initiate new ExStreams if on an RStream that is in the open or half closed (remote) state. Endpoints process new ExStreams only when the RStream is open or half closed (local) state. If an endpoint receives an EXHEADERS frame specifying an RStream in the closed or haf closed (remote) state, it MUST respond with a connection error of type ROUTINGSTREAM_ERROR.

Negotiate the Extension through SETTINGS frame The extension SHOULD be disabled by default. Endpoints can negotiate the use of the extension through the SETTINGS frame. If an implementation supports the extension, it is RECOMMENDED to include the ENABLEEXHEADERS setting in the initial SETTINGS frame. HTTP/2 compliant implementations will ignore the setting if it is unknown. An endpoint can send EXHEADERS frames immediately upon receiving a SETTINGS frame with ENABLEEX_HEADERS=1. Endpoints MUST NOT send out EXHEADERS before receiving a SETTINGS frame with the ENABLEEXHEADERS=1. If a remote endpoint does not support this extension, the EXHEADERS will be ignored, making the header compression contexts inconsistent between sender and receiver. If an endpoint supports this extension, but receives EXHEADERS before ENABLEEXHEADERS, it MUST respond with a connection error EXHEADERNOTENABLED_ERROR. Intermediaries SHOULD send the ENABLEEXHEADERS setting to clients, only if intermediaries and their upstream servers can support this extension. If an intermediary receives an ExStream but discovers the destination endpoint does not support the extension, it MUST reset the stream with EXHEADERNOTENABLEDERROR.

Interaction with standard http2 features The extension implementation should apply stream and connection level flow control, maximum concurrent streams limit, GOAWAY logic to both RStreams and ExStreams.

HTTP2 EX_HEADERS Frame The EXHEADERS frame (type=0xfb) has all the fields and frame header flags defined by HEADERS frame in HEADERS . Moreover, EXHEADERS has one extra field, RStream ID. It is used to open an ExStream, and additionally carries a header block fragment. EX_HEADERS frames can be sent on a stream in the "idle", "open", or "half-closed (remote)" state. Like HEADERS, the CONTINUATION frame (type=0x9) is used to continue a sequence of header block fragments, if the headers do not fit into one EX_HEADERS frame.

EX_HEADERS Frame Payload +---------------+ |Pad Length? (8)| +-+-------------+-----------------------------------------------+ |E| Stream Dependency? (31) | +-+-------------+-----------------------------------------------+ | Weight? (8) | +-+-------------+-----------------------------------------------+ |R| Routing Stream ID (31) | +-+-------------+-----------------------------------------------+ | Header Block Fragment (*) ... +---------------------------------------------------------------+ | Padding (*) ... +---------------------------------------------------------------+ The RStream specified in EXHEADERS frames MUST be an open stream. The recipient MUST respond with a connection error ROUTINGSTREAMERROR PROTOCOLERROR, if the specified RStream is missing; or is an ExStream rather than a stream; or is closed or half-closed (remote). Otherwise, the states maintained for header compression or flow control) may be out of sync.

IANA Considerations This document establishes a registry for a new frame type, setting, and error code.