Encapsulations for In-situ OAM Data

This document discusses transport mechanisms for "in-situ" Operations, Administration, and Maintenance (OAM) data fields. In-situ OAM records OAM information within the packet while the packet traverses a particular network domain. The term "in-situ" refers to the fact that the OAM data is added to the data packets rather than is being sent within packets specifically dedicated to OAM. A discussion of the motivation and requirements for in-situ OAM can be found in . Data types and data formats for in-situ OAM are defined in . This document outlines transport encapsulations for the in-situ OAM data defined in . This document is to serve informational purposes only. As part of an in-situ OAM implementation study different protocol encapsulations for in-situ OAM data are being explored. Once data formats and encapsulation approaches are settled, protocol specific specifications for in-situ OAM data transport will address the standardization aspect. The data for in-situ OAM defined in can be carried in a variety of protocols based on the deployment needs. This document discusses transport of in-situ OAM data for the following protocols: IPv6 IPv4 VXLAN-GPE NSH Segment Routing (IPv6 and MPLS) This list is non-exhaustive, as it is possible to carry the in-situ OAM data in several other protocols and transports. A feasibility study of in-situ OAM is currently underway as part of the FD.io project . The in-situ OAM implementation study should be considered as a "tool box" to showcase how "in-situ" OAM can complement probe-packet based OAM mechanisms for different deployments and packet transport formats. For details, see the open source code in the FD.io .

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in . Abbreviations used in this document: In-situ Operations, Administration, and Maintenance Maximum Transmit Unit Network Service Header Operations, Administration, and Maintenance Proof of Transit Service Function Chain Segment Identifier Segment Routing Virtual eXtensible Local Area Network, Generic Protocol Extension

This mechanisms of in-situ OAM in IPv6 complement others proposed to enhance diagnostics of IPv6 networks, such as the IPv6 Performance and Diagnostic Metrics Destination Option described in . The IP Performance and Diagnostic Metrics Destination Option is destination focused and specific to IPv6, whereas in-situ OAM is performed between end-points of the network or a network domain where it is enabled and used. A historical note: The idea of IPv6 route recording was originally introduced by back in year 2000. With IPv6 now being generally deployed and new concepts such as Segment Routing being introduced, it is imperative to further mature the Operations, Administration, and Maintenance mechanisms available to IPv6 networks. The in-situ OAM options translate into options for an IPv6 hop by hop extension header. The extension header would be inserted by either a host source of the packet, or by a transit/domain-edge node. If the addition of the in-situ OAM Hop-by-Hop Option header would lead to the packet exceeding the MTU of the domain an error should be reported. The methods and procedures of how the error is reported are outside the scope of this document. Likewise if an ICMPv6 forwarding error occurs between encapsulating and decapsulating nodes, the node generating the ICMPv6 error should strip the in-situ OAM Hop-by-Hop Option header before sending the ICMPv6 message to the source.

This section defines in-situ OAM for IPv6 transport. In-situ OAM Options are transported in IPv6 hop-by-hop extension header.

IPv6 hop-by-hop option format for carrying in-situ OAM data fields:

In-situ OAM Options are inserted as Option data as follows: Pre-allocated Tracing Option: The in-situ OAM Preallocated Tracing option defined in is represented as a IPv6 option in hop by hop extension header by allocating following type: 001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_PRE_TRACE_OPTION_IPV6. Incremental Tracing Option: The in-situ OAM Incremental Tracing option defined in is represented as a IPv6 option in hop by hop extension header by allocating following type: 001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INCR_TRACE_OPTION_IPV6. Proof of Transit Option: The in-situ OAM POT option defined in is represented as a IPv6 option in hop by hop extension header by allocating following type: 001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_POT_OPTION_IPV6. Edge to Edge Option: The in-situ OAM E2E option defined in is represented as a IPv6 option in hop by hop extension header by allocating following type: 000xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_E2E_OPTION_IPV6.

Transport of in-situ OAM data in IPv4 will use GRE encapsulation. GRE encapsulation is defined in . IOAM is defined as a "set of Protocol Types" TBD_IANA_ETHERNET_NUMBER_IOAM_* and follows GRE header. These Protocol Types are defined in as "ETHER TYPES" and in . The different IOAM data fields defined in are added as TLVs following the GRE header. In an administrative domain where IOAM is used, insertion of the IOAM protocol header in GRE is enabled at the GRE tunnel endpoints which also serve as IOAM encapsulating/decapsulating nodes by means of configuration. For IOAM the following new GRE protocol types are requested: IOAM_Trace_Preallocated: TBD_IANA_ETHERNET_NUMBER_IOAM_TRACE_PREALLOCATED IOAM_Trace_Incremental: TBD_IANA_ETHERNET_NUMBER_IOAM_TRACE_INCREMENTAL IOAM_POT: TBD_IANA_ETHERNET_NUMBER_IOAM_POT IOAM_End-to_End: TBD_IANA_ETHERNET_NUMBER_IOAM_E2E

The packet formats of the pre-allocated IOAM trace and incremental IOAM trace when transported using GRE are defined as below. See for details about pre-allocated and incremental IOAM trace options.

The GRE header and fields are defined in with Protocol Type set to TBD_IANA_ETHERNET_NUMBER_IOAM_TRACE. IOAM specific fields and header are defined here: 8-bit unsigned integer defining IOAM header type IOAM_TRACE_Preallocated or IOAM_Trace_Incremental are defined here. 8 bits Length field contains the length of the variable metadata octets. 16 bits Next Protocol Type field contains the protocol type of the packet following IOAM protocol header. These Protocol Types are defined in as "ETHER TYPES" and in . An implementation receiving a packet containing a Protocol Type which is not listed in or SHOULD discard the packet. 16-bit identifier of IOAM Trace Type as defined in IOAM-Trace-Types. 4-bit unsigned integer as defined in . 5-bit field as defined in . 7-bit unsigned integer as defined in . 7-bit unsigned integer as defined in . Variable-length field as defined in .

The GRE header and fields are defined in with Protocol Type set to TBD_IANA_ETHERNET_NUMBER_IOAM_POT. IOAM specific fields and header are defined here: 7-bit identifier of a particular POT variant that dictates the POT data that is included as defined in . 1-bit as defined in IOAM POT Option. 8 bits Length field contains the length of the variable metadata octets. 16 bits Next Protocol Type field contains the protocol type of the packet following IOAM protocol header. These Protocol Types are defined in as "ETHER TYPES" and in . An implementation receiving a packet containing a Protocol Type which is not listed in or SHOULD discard the packet. 64-bit Per-packet random number. 64-bit Cumulative value that is updated by the Service Functions.

8-bit identifier of a particular E2E variant that dictates the E2E data that is included as defined in . 8 bits Length field contains the length of the variable metadata octets. 16 bits Next Protocol Type field contains the protocol type of the packet following IOAM protocol header. These Protocol Types are defined in as "ETHER TYPES" and in . An implementation receiving a packet containing a Protocol Type which is not listed in or SHOULD discard the packet. Variable length field as defined in IOAM E2E Option.

VXLAN-GPE encapsulation is somewhat similar to IPv6 extension headers in that a series of headers can be contained in the header as a linked list. The different iIOAM types are added as options within a new IOAM protocol header in VXLAN GPE. In an administrative domain where IOAM is used, insertion of the IOAM protocol header in VXLAN GPE is enabled at the VXLAN GPE tunnel endpoint which also serve as IOAM encapsulating/decapsulating nodes by means of configuration.

The packet formats of the pre-allocated IOAM trace and incremental IOAM trace when transported in VXLAN-GPE are defined as below. See for details about pre-allocated and incremental IOAM trace options. The VXLAN-GPE header and fields are defined in . IOAM specific fields and header are defined here:

8-bit unsigned integer defining IOAM header type IOAM_TRACE_Preallocated or IOAM_Trace_Incremental are defined here. 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units. 8-bit reserved field MUST be set to zero. 8-bit unsigned integer that determines the type of header following IOAM protocol. The value is from the IANA registry setup for VXLAN GPE Next Protocol defined in . 16-bit identifier of IOAM Trace Type as defined in IOAM-Trace-Types. 4-bit unsigned integer as defined in . 5-bit field as defined in . 7-bit unsigned integer as defined in . 7-bit unsigned integer as defined in . Variable-length field as defined in .

The VXLAN-GPE header and fields are defined in . IOAM specific fields and header are defined here:

7-bit identifier of a particular POT variant that dictates the POT data that is included as defined in . 1-bit as defined in IOAM POT Option. 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units 8-bit reserved field MUST be set to zero. 8-bit unsigned integer that determines the type of header following IOAM protocol. The value is from the IANA registry setup for VXLAN GPE Next Protocol defined in . 64-bit Per-packet random number. 64-bit Cumulative value that is updated by the Service Functions.

8-bit identifier of a particular E2E variant that dictates the E2E data that is included as defined in . 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units 8-bit reserved field MUST be set to zero. 8-bit unsigned integer that determines the type of header following IOAM protocol. The value is from the IANA registry setup for VXLAN GPE Next Protocol defined in . Variable length field as defined in IOAM E2E Option.

The packet formats of the pre-allocated IOAM trace and incremental IOAM trace when transported in NSH are defined as below. See for details about pre-allocated and incremental IOAM trace options. In Service Function Chaining (SFC) , the Network Service Header (NSH) already includes path tracing capabilities . Tracing information can be carried in-situ as IOAM data fields following NSH MDx metadata TLVs.

TBD value for IOAM_Trace. 8-bit unsigned integer defining IOAM header type IOAM_TRACE_Preallocated or IOAM_Trace_Incremental are defined here. 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units. Reserved bits are present for future use. The reserved bits MUST be set to 0x0. 8-bit unsigned integer that determines the type of header following IOAM protocol. 16-bit identifier of IOAM Trace Type as defined in IOAM-Trace-Types. 4-bit unsigned integer as defined in . 5-bit field as defined in . 7-bit unsigned integer as defined in . 7-bit unsigned integer as defined in . Variable-length field as defined in .

The "Proof of Transit" capabilities (see and ) of in-situ OAM can be leveraged within NSH. In an administrative domain where in-situ OAM is used, insertion of the in-situ OAM data into the NSH header is enabled at the required nodes (i.e. at the in-situ OAM encapsulating/decapsulating nodes) by means of configuration. Proof of transit in-situ OAM data is added as NSH Type 2 metadata:

TBD value for IOAM_POT. 7-bit identifier of a particular POT variant that dictates the POT data that is included as defined in . 1-bit as defined in IOAM POT Option. 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units Reserved bits are present for future use. The reserved bits MUST be set to 0x0. 8-bit unsigned integer that determines the type of header following IOAM protocol. 64-bit Per-packet random number. 64-bit Cumulative value that is updated by the Service Functions.

The "Edge-to-Edge" capabilities (see ) of in-situ OAM can be leveraged within NSH. In an administrative domain where in-situ OAM is used, insertion of the in-situ OAM data into the NSH header is enabled at the required nodes (i.e. at the in-situ OAM encapsulating/decapsulating nodes) by means of configuration. Edge-to-Edge in-situ OAM data is added as a TLV following NSH MDx metadata:

TBD value for IOAM_E2E. 8-bit identifier of a particular E2E variant that dictates the IOAM E2E data that is included as defined in . 8-bit unsigned integer. Length of the in-situ OAM HDR in 8-octet units Reserved bits are present for future use. The reserved bits MUST be set to 0x0. 8-bit unsigned integer that determines the type of header following IOAM protocol. Variable length field as defined in IOAM E2E Option.

Similar to NSH, a policy defined using Segment Routing for IPv6 can be verified using the in-situ OAM "Proof of Transit" approach. The Segment Routing Header (SRH) for IPv6 offers the ability to transport TLV structured data, similar to what NSH does (see ). In an domain where in-situ OAM is used, insertion of the in-situ OAM data is enabled at the required edge nodes (i.e. at the in-situ OAM encapsulating/decapsulating nodes) by means of configuration. A new "POT TLV" is defined for the SRH which is to carry proof of transit in situ OAM data.

To be assigned by IANA. 20. 8 bits. SHOULD be unset on transmission and MUST be ignored on receipt. 1 bit. Indicates which POT-profile is active. 0 means the even POT-profile is active, 1 means the odd POT-profile is active. 8 bits. No flags are defined in this document. 7-bit identifier of a particular POT variant that dictates the POT data that is included as defined in . 1-bit as defined in IOAM POT Option. 24-bit field MUST be filled with zeroes. 64-bit per-packet random number. 64-bit cumulative value that is updated at specific nodes that form the service path to be verified.

In-situ OAM "Proof of Transit" data can also be carried as part of the MPLS label stack. Details will be addressed in a future version of this document.

IANA considerations will be added in a future version of this document.

Manageability considerations will be addressed ín a later version of this document..

Security considerations will be addressed ín a later version of this document. For a discussion of security requirements of in-situ OAM, please refer to .

The authors would like to thank Eric Vyncke, Nalini Elkins, Srihari Raghavan, Ranganathan T S, Karthik Babu Harichandra Babu, Akshaya Nadahalli, Stefano Previdi, Hemant Singh, Erik Nordmark, LJ Wobker, and Andrew Yourtchenko for the comments and advice. The authors would like to acknowledge Craig Hill for contributing GRE IOAM encapsulation. For the IPv6 encapsulation, this document leverages and builds on top of several concepts described in . The authors would like to acknowledge the work done by the author Hiroshi Kitamura and people involved in writing it.