mnot@mnot.net https://www.mnot.net/

General Internet-Draft This document specifies an alternative way for Web sites to send HTTP response header fields that apply to large numbers of resources, to improve efficiency.

HTTP response headers are being used for an increasing amount of metadata that applies to an entire site, or large portions of it. For example, Strict-Transport-Security and Public-Key-Pins both define headers that are explicitly scoped to an entire origin , and number of similar headers are under consideration. Likewise, some HTTP header fields only sensibly have a single value per origin; for example, Server. Furthermore, some headers are used uniformly across an origin. For example, a site might have a Content-Security-Policy header that doesn’t vary across the site, or only varies slightly from resource to resource. HTTP/2’s HPACK header compression mechanism was designed to reduce bandwidth usage for often-repeated headers, both in responses and requests. However, it limits the amount of compression contents usable for a connection (by default, 4K), which sites are beginning to exceed, thereby reducing the efficiency of HPACK itself. For example, it is not uncommon for a CSP response header field to exceed 1K (and has been observed to be greater than 3K on popular sites). This forces site administrators to make an awkward choice; put the large header in the HPACK table, thereby crowding out other headers, or omit it, requiring its full content to be sent on every applicable response. This document defines a way to specify one or more sets of HTTP response header fields in a well-known resource that, when their use is negotiated, are appended to HTTP responses by the user agent. This allows common response headers to be omitted both from on-the-wire responses and the HPACK compression table, making both more efficient. This approach is preferable to increasing the HTTP/2 SETTINGS_HEADER_TABLE_SIZE (, Section 6.5.2), because increasing that setting incurs a per-connection overhead on the server, whereas using the technique documented here does not.

If a user agent has a fresh copy of the well-known resource for an origin (see ), because either it performed a GET, or HTTP/2 Server Push was used:

and the user agent makes the request:

this indicates that the user agent has processed the well-known resource, and therefore that the server can omit the nominated response header fields on the wire, instead referring to them with the HS response header field:

Upon receipt of that response, the user agent will consider it equivalent to:

If a request omits the SM header field, or its field-value does not match the current ETag of the well-known resource, all of the header fields above will be sent by the server in the response.

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in . This document uses the following ABNF rules from : DQUOTE, ALPHA. From : OWS, RWS, CRLF, header-field. From : entity-tag.

When a server wishes to use site-wide HTTP headers, it places a file in the format specified in at the well-known URI specified in . Then, when a request has a SM request header field (as per ) that matches the current ETag of the well-known resource, the set of response header fields referred to by the HS response header field (see ) for the requested resource are omitted from the corresponding response. Servers SHOULD include SM in the field-value of the Vary response header field for all cacheable (as per ) responses of resources that behave in this manner, whether or not headers have been actually appended. This assures correct cache operation, and also advertises support for this specification. Servers MAY use HTTP/2 Server Push (, Section 8.2) to proactively send the well-known resource to user agents (e.g., if they emit SM: *, indicating that they do not have a fresh copy of the well-known resource).

Because this mechanism effectively hides response header fields from intermediaries that do not implement it, care ought to be take in selecting the headers to use it upon. For example, the Cache-Control and Vary headers are poor candidates, because they are often used by intermediaries for HTTP caching . Likewise, HTTP/1 headers that affect message framing and connection behaviour (e.g., Content-Length, Transfer-Encoding, Connection) MUST NOT be included in the well-known resource.

The HS HTTP response header field indicates the header set in the well-known location file (see ) that should be applied to the response it occurs within.

For example:

User agents that support this specification SHOULD always emit a SM header field in requests, carrying either the ETag of the well-known resource currently held for the origin, or * to indicate that they support this specification, but do not have a fresh (as per ) copy of it. User agents might discover that an origin supports this specification when it returns a response containing the HS response header field, or they might learn of it when the well-known location’s current contents are sent via a HTTP/2 Server Push. In either case, user agents SHOULD send a SM request header field on all requests to such an origin. Upon receiving a response to such a request containing the HS response header field, user agents MUST locate the header-set referred to by its field-value in the stored well-known response, remove any surrounding white space, and append it to the response headers, stripping the HS response header field. If the corresponding header-set cannot be found in the well-known location, the response MUST be considered invalid and MUST NOT be used; the user agent MAY retry the request without the SM request header field if its method was safe, or may take alternative recovery strategies.

The SM HTTP request header field indicates that the user agent has a fresh (as per ) copy of the well-known resource (see ) for the request’s origin ().

Its value is the entity-tag of the freshest valid well-known location response held by the user agent. If none is held, it should be * (without quotes). For example:

The well-known URI “site-headers” is a resource that, when fetched, returns a file in the “text/site-headers” format (see ). Its media type SHOULD be generated as text/site-headers, although user agents SHOULD NOT reject responses with other types (particularly, application/octet-stream and text/plain). Its representation MUST contain an ETag response header . User agents SHOULD consider it to be valid for its freshness lifetime (as per ). If it does not have an explicit freshness lifetime, they SHOULD consider it to have a heuristic freshness lifetime of 60 seconds.

The text/site-headers media type is used to indicate that a file contains one or more sets of HTTP header fields, as defined in , Section 3.

Each set of HTTP header fields is started by a header-header, which is indicated by an octothorp (“#”) followed by the name of the header set. The following lines, up until the next line beginning with an octothorp or the end of the file are considered to be the header-set’s contents. As in HTTP itself, implementations need to be forgiving about line endings; specifically, bare CR MUST be considered to be a line ending. For example:

This file specifies two sets of HTTP headers, “foo” and “bar”. Note that the Public-Key-Pins and Content-Security-Policy header fields are line-folded; as in HTTP, this form of header is deprecated in this format, and SHOULD NOT be used (except in documentation, as we see here).

Given a stream of Unicode characters: Let header-sets be an empty mapping. Consume all characters from up to and including the first octothorp (“#”). Consume all WSP characters. Let set-name be all characters up to but not including the next WSP, CR or LF. Consume all WSP, CR and LF characters. Let header-set be all characters up to but not including the next CR or LF character followed by an octothorp (“#”), or the end of the file. Trim all WSP from the end of header-set. Let the value of the set-name entry in header-sets be header-set (removing any existing value). If there is more input, return to step 2. Otherwise, return header-sets. This returns a mapping of set-name to a HTTP header-set, as defined in , Section 3. It SHOULD be parsed as defined there.

TBD

Site-wide headers allow a single resource to inject HTTP response headers for an entire origin. Accordingly, the ability to write to that resource needs to be carefully controlled by the origin server.

As noted in , there are a variety of HTTP response headers which are inappropriate for use as site-wide headers, and some (e.g., Content-Length) can cause both interoperability and security issues.

Because headers sent via this mechanism will not be seen by user agents and intermediaries that do not implement this specification, they will potentially have a different view of the response headers.

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. This document defines the concept of an "origin", which is often used as the scope of authority or privilege by user agents. Typically, user agents isolate content retrieved from different origins to prevent malicious web site operators from interfering with the operation of benign web sites. In addition to outlining the principles that underlie the concept of origin, this document details how to determine the origin of a URI and how to serialize an origin into a string. It also defines an HTTP header field, named "Origin", that indicates which origins are associated with an HTTP request. [STANDARDS-TRACK] This memo defines a path prefix for "well-known locations", "/.well-known/", in selected Uniform Resource Identifier (URI) schemes. [STANDARDS-TRACK] Internet technical specifications often need to define a formal syntax. Over the years, a modified version of Backus-Naur Form (BNF), called Augmented BNF (ABNF), has been popular among many Internet specifications. The current specification documents ABNF. It balances compactness and simplicity with reasonable representational power. The differences between standard BNF and ABNF involve naming rules, repetition, alternatives, order-independence, and value ranges. This specification also supplies additional rule definitions and encoding for a core lexical analyzer of the type common to several Internet specifications. [STANDARDS-TRACK] The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations. The Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP/1.1 conditional requests, including metadata header fields for indicating state changes, request header fields for making preconditions on such state, and rules for constructing the responses to a conditional request when one or more preconditions evaluate to false. The Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines HTTP caches and the associated header fields that control cache behavior or indicate cacheable response messages. This specification defines a mechanism enabling web sites to declare themselves accessible only via secure connections and/or for users to be able to direct their user agent(s) to interact with given sites only over secure connections. This overall policy is referred to as HTTP Strict Transport Security (HSTS). The policy is declared by web sites via the Strict-Transport-Security HTTP response header field and/or by other means, such as user agent configuration, for example. [STANDARDS-TRACK] This document defines a new HTTP header that allows web host operators to instruct user agents to remember ("pin") the hosts' cryptographic identities over a period of time. During that time, user agents (UAs) will require that the host presents a certificate chain including at least one Subject Public Key Info structure whose fingerprint matches one of the pinned fingerprints for that host. By effectively reducing the number of trusted authorities who can authenticate the domain during the lifetime of the pin, pinning may reduce the incidence of man-in-the-middle attacks due to compromised Certification Authorities. This specification defines HPACK, a compression format for efficiently representing HTTP header fields, to be used in HTTP/2. This specification describes an optimized expression of the semantics of the Hypertext Transfer Protocol (HTTP), referred to as HTTP version 2 (HTTP/2). HTTP/2 enables a more efficient use of network resources and a reduced perception of latency by introducing header field compression and allowing multiple concurrent exchanges on the same connection. It also introduces unsolicited push of representations from servers to clients.This specification is an alternative to, but does not obsolete, the HTTP/1.1 message syntax. HTTP's existing semantics remain unchanged.