Mozilla

martin.thomson@gmail.com

General Internet-Draft This memo introduces a content-coding for HTTP that provides progressive integrity for message contents. This integrity protection can be evaluated on a partial representation, allowing a recipient to process a message as it is delivered while retaining strong integrity protection.

Integrity protection for HTTP content is highly valuable. HTTPS is the most common form of integrity protection deployed, but that requires a direct TLS connection to a host. However, additional integrity protection might be desirable for some use cases. This might be for additional protection against failures or attack (see ) or because content needs to remain unmodified throughout multiple HTTPS-protected exchanges. This document describes a “mi-sha256” content-encoding (see ) that is a progressive, hash-based integrity check based on Merkle Hash Trees . The means of conveying the root integrity proof used by this content encoding will depend on deployment requirements. This document defines an MI header field (see ) that can carry an integrity proof.

The key words “MUST”, “MUST NOT”, “REQUIRED”, “SHALL”, “SHALL NOT”, “SHOULD”, “SHOULD NOT”, “RECOMMENDED”, “MAY”, and “OPTIONAL” in this document are to be interpreted as described in .

A Merkle Hash Tree is a structured integrity mechanism that collates multiple integrity checks into a tree. The leaf nodes of the tree contain data (or hashes of data) and non-leaf nodes contain hashes of the nodes below them. A balanced Merkle Hash Tree is used to efficiently prove membership in large sets (such as in ). However, in this case, a right-skewed tree is used to provide a progressive integrity proof. This integrity proof is used to establish that a given record is part of a message. The hash function used for “mi-sha256” content encoding is SHA-256 . The integrity proof for all records other than the last is the hash of the concatenation of the record, the integrity proof of all subsequent records, and a single octet with a value of 0x1:

The integrity proof for the final record is the hash of the record with a single octet with a value 0x0 appended:

shows the structure of the integrity proofs for a message that is split into 4 blocks: A, B, C, D). As shown, the integrity proof for the entire message (that is, proof(A)) is derived from the content of the first block (A), plus the value of the proof for the second and subsequent blocks.

The final encoded message is formed from the first record, followed by an arbitrary number of tuples of the integrity proof of the next record and then the record itself. Thus, in , the body is:

A message that has a content length less than or equal to the content size does not include any inline proofs. The proof for a message with a single record is simply the hash of the body plus a trailing zero octet.

In order to produce the final content encoding the content of the message is split into equal-sized records. The final record can contain less than the defined record size. The default record size for the “mi-sha256” content encoding is 4096 octets. This refers to the length of each data block. The MI header field MAY contain an “rs” parameter that describes a different record size. The final encoded stream comprises of a record (“rs” octets in length), followed by the proof for the following record (32 octets). This allows a receiver to validate and act upon each record after receiving the proof that precedes it. The final record is not followed by a proof. This content encoding increases the size of a message by 32 octets times the length of the message divided by the record size, rounded up, less one. That is, 32 * (ceil(length / rs) - 1). Constructing a message with the “mi-sha256” content encoding requires processing of the records in reverse order, inserting the proof derived from each record before that record. This structure permits the use of range requests . However, to validate a given record, a contiguous sequence of records back to the start of the message is needed.

A receiver of a message with the “mi-sha256” content-encoding applied first attempts to acquire the integrity proof for the first record. If the MI header field is present, a value might be included there. Then, the message is read into records of size “rs” (based on the value in the MI header field) plus 32 octets. The last record is between 1 and “rs” octets in length, if not then validation fails. For each record: Hash the record using SHA-256 with a single octet appended: a. All records other than the last have an octet with a value of 0x1 appended. b. The last record has an octet with a value of 0x0 appended. Compare the hash with the expected value: a. For the first record, the expected value might found in the MI header field and is otherwise provided through some external means. b. For records after the first, the expected value is the last 32 octets of the previous record. If the hash is different, then this record and all subsequent records do not have integrity protection and this process ends. If a record is valid, up to “rs” octets is passed on for processing. In other words, the trailing 32 octets is removed from every record other than the last before being used. If an integrity check fails, the message SHOULD be discarded and the exchange treated as an error unless explicitly configured otherwise. For clients, treat this as equivalent to a server error; servers SHOULD generate a 400 or other 4xx status code. However, if the integrity proof for the first record is not known, this check SHOULD NOT fail unless explicitly configured.

The MI HTTP header field describes the message integrity content encoding(s) that have been applied to a payload body, and therefore how those content encoding(s) can be removed. The MI header field uses the extended ABNF syntax defined in Section 1.2 of and the parameter rule from :

If the payload is encoded more than once (as reflected by having multiple content-codings that use the message integrity header field), each application of the content encoding is reflected in the MI header field in the order in which they were applied. The MI header MAY be omitted if the sender intends for the receiver to acquire the integrity proof for the first record by other means.

The following parameters are used in validating content encoded with the “mi-sha256” content encoding: The “p” parameter carries an integrity proof for the first record of the message. This provides integrity for the entire message body. This value is encoded using base64url encoding . The “rs” parameter contains a positive decimal integer that describes the record size in octets. This value MUST be greater than 0. If the “rs” parameter is absent, the record size defaults to 4096 octets.

The following example contains a short message. This contains just a single record, so there are no inline integrity proofs, just a single value in a MI header field.

This example shows the same message as above, but with a smaller record size (16 octets). This results in two integrity proofs being included in the representation.

Since the inline integrity proofs contain non-printing characters, these are shown here using the base64url Encoding with new lines between the original text and integrity proofs. Note that there is a single trailing space (0x20) on the first line.

The integrity of an entire message body depends on the means by which the integrity proof for the first record is protected. If this value comes from the same place as the message, then this provides only limited protection against transport-level errors (something that TLS provides adequate protection against). Separate protection for header fields might be provided by other means if the first record retrieved is the first record in the message, but range requests do not allow for this option.

This integrity scheme permits the detection of truncated messages. However, it enables and even encourages processing of messages prior to receiving an complete message. Actions taken on a partial message can produce incorrect results. For example, a message could say “I need some 2mm copper cable, please send 100mm for evaluation purposes” then be truncated to “I need some 2mm copper cable, please send 100m”. A network-based attacker might be able to force this sort of truncation by delaying packets that contain the remainder of the message. Whether it is safe to act on partial messages will depend on the nature of the message and the processing that is performed.

A new content encoding type is needed in order to define the use of a hash function other than SHA-256.

This memo registers the “mi-sha256” HTTP content-coding in the HTTP Content Codings Registry, as detailed in . Name: mi-sha256 Description: A Merkle Hash Tree based content encoding that provides progressive integrity. Reference: this specification

This memo registers the “MI” HTTP header field in the Permanent Message Header Registry, as detailed in . Field name: MI Protocol: HTTP Status: Standard Reference: this specification Notes:

This memo establishes a registry for parameters used by the “MI” header field under the “Hypertext Transfer Protocol (HTTP) Parameters” grouping. The “Hypertext Transfer Protocol (HTTP) MI Parameters” registry operates under an “Specification Required” policy . Entries in this registry are expected to include the following information: Parameter Name: The name of the parameter. Purpose: A brief description of the purpose of the parameter. Reference: A reference to a specification that defines the semantics of the parameter. The initial contents of this registry are:

Parameter Name: p Purpose: The value of the integrity proof for the first record. Reference: this document

Parameter Name: rs Purpose: The size of the records used for progressive integrity protection. Reference: this document

In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Many protocols make use of identifiers consisting of constants and other well-known values. Even after a protocol has been defined and deployment has begun, new values may need to be assigned (e.g., for a new option type in DHCP, or a new encryption or authentication transform for IPsec). To ensure that such quantities have consistent values and interpretations across all implementations, their assignment must be administered by a central authority. For IETF protocols, that role is provided by the Internet Assigned Numbers Authority (IANA).In order for IANA to manage a given namespace prudently, it needs guidelines describing the conditions under which new values can be assigned or when modifications to existing values can be made. If IANA is expected to play a role in the management of a namespace, IANA must be given clear and concise instructions describing that role. This document discusses issues that should be considered in formulating a policy for assigning values to a namespace and provides guidelines for authors on the specific text that must be included in documents that place demands on IANA.This document obsoletes RFC 2434. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document provides an overview of HTTP architecture and its associated terminology, defines the "http" and "https" Uniform Resource Identifier (URI) schemes, defines the HTTP/1.1 message syntax and parsing requirements, and describes related security concerns for implementations. The Hypertext Transfer Protocol (HTTP) is a stateless \%application- level protocol for distributed, collaborative, hypertext information systems. This document defines the semantics of HTTP/1.1 messages, as expressed by request methods, request header fields, response status codes, and response header fields, along with the payload of messages (metadata and body content) and mechanisms for content negotiation. JSON Web Signature (JWS) represents content secured with digital signatures or Message Authentication Codes (MACs) using JSON-based data structures. Cryptographic algorithms and identifiers for use with this specification are described in the separate JSON Web Algorithms (JWA) specification and an IANA registry defined by that specification. Related encryption capabilities are described in the separate JSON Web Encryption (JWE) specification. This memo describes how to use Transport Layer Security (TLS) to secure Hypertext Transfer Protocol (HTTP) connections over the Internet. This memo provides information for the Internet community. This document specifies Version 1.2 of the Transport Layer Security (TLS) protocol. The TLS protocol provides communications security over the Internet. The protocol allows client/server applications to communicate in a way that is designed to prevent eavesdropping, tampering, or message forgery. [STANDARDS-TRACK] This document describes an experimental protocol for publicly logging the existence of Transport Layer Security (TLS) certificates as they are issued or observed, in a manner that allows anyone to audit certificate authority (CA) activity and notice the issuance of suspect certificates as well as to audit the certificate logs themselves. The intent is that eventually clients would refuse to honor certificates that do not appear in a log, effectively forcing CAs to add all issued certificates to the logs.Logs are network services that implement the protocol operations for submissions and queries that are defined in this document. The Hypertext Transfer Protocol (HTTP) is a stateless application- level protocol for distributed, collaborative, hypertext information systems. This document defines range requests and the rules for constructing and combining responses to those requests.

David Benjamin and Erik Nygren both separately suggested that something like this might be valuable. Eric Rescorla provided useful feedback.

Why not include the first proof in the encoding? The requirements for the integrity proof for the first record require a great deal more flexibility than this allows for. Transferring the proof separately is sometimes necessary. Separating the value out allows for that to happen more easily. Why do messages have to be processed in reverse to construct them? The final integrity value, no matter how it is derived, has to depend on every bit of the message. That means that there are three choices: both sender and receiver have to process the whole message, the sender has to work backwards, or the receiver has to work backwards. The current form is the best option of the three. The expectation is that this will be useful for content that is generated once and sent multiple times, since the onerous backwards processing requirement can be amortized.