Ericsson

gregimirsky@gmail.com

Ciena

sboutros@ciena.com

Dell

david.black@dell.com

VMware

santosh.pallagatti@gmail.com

RTG nvo3 OAM Geneve (Generic Network Virtualization Encapsulation) is a flexible and extensible network virtualization overlay protocol designed to encapsulate network packets for transport across underlying physical networks. This document specifies the requirements and provides a framework for Operations, Administration, and Maintenance (OAM) in Geneve networks. It outlines the OAM functions necessary to monitor, diagnose, and troubleshoot Geneve overlay networks to ensure proper operation and performance. The document aims to guide the implementation of OAM mechanisms within the Geneve protocol to support network operators in maintaining reliable and efficient virtualized network environments.

Status of This Memo This is an Internet Standards Track document. This document is a product of the Internet Engineering Task Force (IETF). It represents the consensus of the IETF community. It has received public review and has been approved for publication by the Internet Engineering Steering Group (IESG). Further information on Internet Standards is available in Section 2 of RFC 7841. Information about the current status of this document, any errata, and how to provide feedback on it may be obtained at .

Copyright Notice Copyright (c) 2025 IETF Trust and the persons identified as the document authors. All rights reserved. This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents () in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.

Table of Contents

• .  Introduction
  • .  Conventions Used in This Document
    • .  Requirements Language
    • .  Acronyms
• .  The Applicability of Active OAM Protocols in Geneve Networks
  • .  Requirements for Active OAM Protocols in Geneve Networks
  • .  Defect Detection and Troubleshooting in Geneve Network with Active OAM
    • .  Echo Request and Echo Reply in Geneve Tunnel
  • .  Active OAM Encapsulation in Geneve
• .  IANA Considerations
• .  Security Considerations
• .  References
  • .  Normative References
  • .  Informative References
• Acknowledgments
• Authors' Addresses

Introduction Geneve is designed to support various scenarios of network virtualization. It encapsulates multiple protocols, such as Ethernet and IPv4/IPv6, and includes metadata within the Geneve message. Operations, Administration, and Maintenance (OAM) protocols provide fault management and performance monitoring functions necessary for comprehensive network operation. Active OAM protocols, as defined in , utilize specially constructed packets injected into the network. OAM protocols such as ICMP and ICMPv6 ( and respectively), Bidirectional Forwarding Detection (BFD) , and the Simple Two-way Active Measurement Protocol (STAMP) are examples of active OAM protocols. To ensure that performance metrics or detected failures are accurately related to a particular Geneve flow, it is critical that these OAM test packets share fate, i.e., are in-band, with the overlay data packets of that monitored flow when traversing the underlay network. In this document, "in-band OAM" is interpreted as follows:

• In-band OAM is an active or hybrid OAM method, as defined in , that traverses the same set of links and interfaces and receives the same Quality of Service treatment as the monitored object. In this context, the monitored object refers to either the entire Geneve tunnel or a specific tenant flow within a given Geneve tunnel.

lists the general requirements for active OAM protocols in the Geneve overlay network. IP encapsulation meets these requirements and is suitable for encapsulating active OAM protocols within a Geneve overlay network. Active OAM messages in a Geneve overlay network are exchanged between two Geneve tunnel endpoints; each endpoint may be the tenant-facing interface of the Network Virtualization Edge (NVE) or another device acting as a Geneve tunnel endpoint. Testing end-to-end between tenants is out of scope. For simplicity, this document uses an NVE to represent the Geneve tunnel endpoint. Refer to and for detailed definitions and descriptions of an NVE. The IP encapsulation of Geneve OAM defined in this document applies to an overlay service by introducing a Management Virtual Network Identifier (VNI), which can be used in combination with various values of the Protocol Type field in the Geneve header, such as Ethertypes for IPv4 or IPv6. The analysis and definition of other types of OAM encapsulation in Geneve are outside the scope of this document.

Conventions Used in This Document

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

Acronyms

Geneve:

Generic Network Virtualization Encapsulation

NVO3:

Network Virtualization over Layer 3

OAM:

Operations, Administration, and Maintenance

VNI:

Virtual Network Identifier

BFD:

Bidirectional Forwarding Detection

STAMP:

Simple Two-way Active Measurement Protocol

NVE:

Network Virtualization Edge

The Applicability of Active OAM Protocols in Geneve Networks

Requirements for Active OAM Protocols in Geneve Networks OAM protocols, whether part of fault management or performance monitoring, are intended to provide reliable information that can be used to detect a failure, identify the defect, and localize it, thus helping to identify and apply corrective actions to minimize the negative impact on service. Several OAM protocols are used to perform these functions; these protocols require demultiplexing at the receiving instance of Geneve. To improve the accuracy of the correlation between the condition experienced by the monitored Geneve tunnel and the state of the OAM protocol, the OAM encapsulation is required to comply with the following requirements:

1. Geneve OAM test packets MUST share the same fate as the data traffic of the monitored Geneve tunnel. Specifically, the OAM test packets MUST be in-band with the monitored traffic and follow the same overlay and transport paths as packets carrying data payloads in the forward direction, i.e., from the ingress toward the egress endpoint(s) of the OAM test.

An OAM protocol MAY be employed to monitor an entire Geneve tunnel. In this case, test packets could be in-band relative to a subset of tenant flows transported over the Geneve tunnel. If the goal is to monitor the conditions experienced by the flow of a particular tenant, the test packets MUST be in-band with that specific flow within the Geneve tunnel. Both scenarios are discussed in detail in .

2. The encapsulation of OAM control messages and data packets in the underlay network MUST be indistinguishable.
3. The presence of an OAM control message in a Geneve packet MUST be unambiguously identifiable to Geneve functionality, such as at endpoints of Geneve tunnels.
4. OAM test packets MUST NOT be forwarded to a tenant system.

A test packet generated by an active OAM protocol, whether for defect detection or performance measurement, MUST be in-band with the tunnel or data flow being monitored, as specified in Requirement 1. In environments where multiple paths through the domain are available, underlay transport nodes can be programmed to use characteristic information to balance the load across known paths. It is essential that test packets follow the same route -- that is, traverse the same set of nodes and links as a data packet of the monitored flow. Therefore, the following requirement supports OAM packet fate-sharing with the data flow.

5. There MUST be a way to encode entropy information into the underlay forwarding scheme so that OAM packets take the same data-flow paths as the transit traffic flows.

Defect Detection and Troubleshooting in Geneve Network with Active OAM This section considers two scenarios where active OAM is used to detect and localize defects in a Geneve network. presents an example of a Geneve domain.

An Example of a Geneve Domain +--------+ +--------+ | Tenant +--+ +----| Tenant | | VNI 28 | | | | VNI 35 | +--------+ | ................ | +--------+ | +----+ . . +----+ | | | NVE|--. .--| NVE| | +--| A | . . | B |---+ +----+ . . +----+ / . . / . Geneve . +--------+ / . Network . | Tenant +--+ . . | VNI 35 | . . +--------+ ................ | +----+ | NVE| | C | +----+ | | ===================== | | +--------+ +--------+ | Tenant | | Tenant | | VNI 28 | | VNI 35 | +--------+ +--------+

In the first case, consider when a communication problem between Network Virtualization Edge (NVE) device A and NVE C exists. Upon investigation, the operator discovers that the forwarding in the IP underlay network is working accordingly. Still, the Geneve connection is unstable for all NVE A and NVE C tenants. Detection, troubleshooting, and localization of the problem can be done regardless of the VNI value. In the second case, traffic on VNI 35 between NVE A and NVE B has no problems, as on VNI 28 between NVE A and NVE C. However, traffic on VNI 35 between NVE A and NVE C experiences problems, for example, excessive packet loss. The first case can be detected and investigated using any VNI value, whether it connects tenant systems or not; however, to conform to Requirement 4, OAM test packets SHOULD be transmitted on a VNI that doesn't have any tenants. Such a Geneve tunnel is dedicated to carrying only control and management data between the tunnel endpoints, so it is referred to as a "Geneve control channel" and that VNI is referred to as the "Management VNI". A configured VNI MAY be used to identify the control channel, but it is RECOMMENDED that the default value 1 be used as the Management VNI. Encapsulation of test packets using the Management VNI is discussed in . The control channel of a Geneve tunnel MUST NOT carry tenant data. As no tenants are connected using the control channel, a system that supports this specification MUST NOT forward a packet received over the control channel to any tenant. A packet received by the system that supports this specification over the control channel MUST be forwarded if and only if it is sent onto the control channel of the concatenated Geneve tunnel. Else, it MUST be terminated locally. The Management VNI SHOULD be terminated on the tenant-facing side of the Geneve encapsulation/decapsulation functionality, not the DC-network-facing side (per definitions in ), so that Geneve encap/decap functionality is included in its scope. This approach causes an active OAM packet, e.g., an ICMP echo request, to be decapsulated in the same fashion as any other received Geneve packet. In this example, the resulting ICMP packet is handed to NVE's local management functionality for the processing which generates an ICMP echo reply. The ICMP echo reply is encapsulated in Geneve (as specified in ) for forwarding it back to the NVE that sent the echo request. One advantage of this approach is that a repeated ICMP echo request/reply test could detect an intermittent problem in Geneve encap/decap hardware, which would not be tested if the Management VNI were handled as a "special case" at the DC-network-facing interface. The second case is when a test packet is transmitted using the VNI value associated with the monitored service flow. By doing that, the test packet experiences network treatment as the tenant's packets. An example of the realization of that scenario is discussed in .

Echo Request and Echo Reply in Geneve Tunnel ICMP and ICMPv6 ( and respectively), as noted above, are examples of an active OAM protocol. They provide required on-demand defect detection and failure localization. ICMP control messages immediately follow the inner IP header encapsulated in Geneve. ICMP extensions for Geneve networks use mechanisms defined in .

Active OAM Encapsulation in Geneve Active OAM over a Management VNI in the Geneve network uses an IP encapsulation. Protocols such as BFD and STAMP use UDP transport. The destination UDP port number in the inner UDP header () identifies the OAM protocol. This approach is well-known and has been used, for example, in MPLS networks . To use IP encapsulation for an active OAM protocol, the Protocol Type field of the Geneve header MUST be set to 0x0800 (IPv4) or 0x86DD (IPv6). describes the use of IP encapsulation for BFD.

An Example of Geneve IP/UDP Encapsulation of an Active OAM Packet 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Outer IPvX Header ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Outer UDP Header ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Geneve Header ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Inner IPvX Header ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Inner UDP Header ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | | ~ Active OAM Packet ~ | | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Inner IP header:

Destination IP:

The IP address MUST be set to the loopback address 127.0.0.1/32 for IPv4 version. For IPv6, the address MUST be selected from the Dummy IPv6 Prefix 100:0:0:1::/64 . A source-only IPv6 address is used as the destination to generate an exception and a reply message to the request message received.

Source IP:

IP address of the NVE.

TTL or Hop Limit:

MUST be set to 255 per . The receiver of an active OAM Geneve packet with IP/UDP encapsulation MUST drop packets whose TTL/Hop Limit is not 255.

IANA Considerations This document has no IANA actions.

Security Considerations As part of a Geneve network, active OAM inherits the security considerations discussed in . Additionally, a system MUST provide control to limit the rate of Geneve OAM packets punted to the Geneve control plane for processing in order to avoid overloading that control plane. OAM in Geneve packets uses the General TTL Security Mechanism , and any packet received with an inner TTL / Hop Count other than 255 MUST be discarded.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. Network virtualization involves the cooperation of devices with a wide variety of capabilities such as software and hardware tunnel endpoints, transit fabrics, and centralized control clusters. As a result of their role in tying together different elements of the system, the requirements on tunnels are influenced by all of these components. Therefore, flexibility is the most important aspect of a tunneling protocol if it is to keep pace with the evolution of technology. This document describes Geneve, an encapsulation protocol designed to recognize and accommodate these changing capabilities and needs. Informative References This document describes the format of a set of control messages used in ICMPv6 (Internet Control Message Protocol). ICMPv6 is the Internet Control Message Protocol for Internet Protocol version 6 (IPv6). [STANDARDS-TRACK] This document redefines selected ICMP messages to support multi-part operation. A multi-part ICMP message carries all of the information that ICMP messages carried previously, as well as additional information that applications may require. Multi-part messages are supported by an ICMP extension structure. The extension structure is situated at the end of the ICMP message. It includes an extension header followed by one or more extension objects. Each extension object contains an object header and object payload. All object headers share a common format. This document further redefines the above mentioned ICMP messages by specifying a length attribute. All of the currently defined ICMP messages to which an extension structure can be appended include an "original datagram" field. The "original datagram" field contains the initial octets of the datagram that elicited the ICMP error message. Although the original datagram field is of variable length, the ICMP message does not include a field that specifies its length. Therefore, in order to facilitate message parsing, this document allocates eight previously reserved bits to reflect the length of the "original datagram" field. The proposed modifications change the requirements for ICMP compliance. The impact of these changes on compliant implementations is discussed, and new requirements for future implementations are presented. This memo updates RFC 792 and RFC 4443. [STANDARDS-TRACK] The use of a packet's Time to Live (TTL) (IPv4) or Hop Limit (IPv6) to verify whether the packet was originated by an adjacent node on a connected link has been used in many recent protocols. This document generalizes this technique. This document obsoletes Experimental RFC 3682. [STANDARDS-TRACK] This document describes a protocol intended to detect faults in the bidirectional path between two forwarding engines, including interfaces, data link(s), and to the extent possible the forwarding engines themselves, with potentially very low latency. It operates independently of media, data protocols, and routing protocols. [STANDARDS-TRACK] This document provides a framework for Data Center (DC) Network Virtualization over Layer 3 (NVO3) and defines a reference model along with logical components required to design a solution. This memo provides clear definitions for Active and Passive performance assessment. The construction of Metrics and Methods can be described as either "Active" or "Passive". Some methods may use a subset of both Active and Passive attributes, and we refer to these as "Hybrid Methods". This memo also describes multiple dimensions to help evaluate new methods as they emerge. This document presents a high-level overview architecture for building data-center Network Virtualization over Layer 3 (NVO3) networks. The architecture is given at a high level, showing the major components of an overall system. An important goal is to divide the space into individual smaller components that can be implemented independently with clear inter-component interfaces and interactions. It should be possible to build and implement individual components in isolation and have them interoperate with other independently implemented components. That way, implementers have flexibility in implementing individual components and can optimize and innovate within their respective components without requiring changes to other components. This document describes a simple and efficient mechanism to detect data-plane failures in Multiprotocol Label Switching (MPLS) Label Switched Paths (LSPs). It defines a probe message called an "MPLS echo request" and a response message called an "MPLS echo reply" for returning the result of the probe. The MPLS echo request is intended to contain sufficient information to check correct operation of the data plane and to verify the data plane against the control plane, thereby localizing faults. This document obsoletes RFCs 4379, 6424, 6829, and 7537, and updates RFC 1122. This document describes the Simple Two-way Active Measurement Protocol (STAMP), which enables the measurement of both one-way and round-trip performance metrics, like delay, delay variation, and packet loss. This document describes the use of the Bidirectional Forwarding Detection (BFD) protocol in point-to-point Generic Network Virtualization Encapsulation (Geneve) unicast tunnels used to make up an overlay network.

Acknowledgments The authors express their appreciation to for his suggestions that improved the readability of the document.

Authors' Addresses Ericsson

gregimirsky@gmail.com

Ciena

sboutros@ciena.com

Dell

david.black@dell.com

VMware

santosh.pallagatti@gmail.com