Operations, Administration and Maintenance (OAM) for Overlay Networks: Gap Analysis

Operations, Administration, and Maintenance (OAM) toolset provides methods for fault management and performance monitoring in each layer of the network, in order to improve their ability to support services with guaranteed and strict Service Level Agreements (SLAs) while reducing operational costs. provided detailed analysis of OAM protocols. Since its completion several new protocols that define data plane encapsulation were introduced. That presented both need to re-evaluate existing set of OAM tools and opportunity to build it into set of tools that can be used and re-used for different data plane protocols. defines the set of requirements for OAM in Overlay networks. The OAM solution for Overlay networks, developed by the design team, has two objectives: The Overlay OAM toolset should be developed based on existing IP and IP/MPLS architecture, technology, and toolsets. The Overlay OAM operational experience should be similar to that in other, e.g. IP and IP/MPLS, networks.

Term "Overlay OAM" used in this document interchangeably with longer version "set of OAM protocols, methods and tools for Overlay networks". AIS Alarm Indication Signal BFD Bidirectional Forwarding Detection BIER Bit-Indexed Explicit Replication CC Continuity Check CV Connectivity Verification FM Fault Management G-ACh Generic Associated Channel Geneve Generic Network Virtualization Encapsulation GUE Generic UDP Encapsulation MPLS Multiprotocol Label Switching NTP Network Time Protocol NVO3 Network Virtalization Overlays OAM Operations, Administration, and Maintenance OWAMP One-Way Active Measurement Protocol PM Performance Measurement PTP Precision Time Protocol SFC Service Fundction Chaining SFP Service Function Path SLA Service Level Agreement TWAMP Two-Way Active Measurement Protocol VxLAN Virtual eXtensible Local Area Network VxLAN-GPE Generic Protocol Extension for VxLAN

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in .

It is expected that the encapsulation of an overlay network uses one of methods discussed in to distinctly identify the payload as OAM, i.e. non-user, packet. In its turn all Overlay OAM protocols share the common Overlay OAM Header. Format and processing of the header are outside the scope of this document and will be presented in the solution document.

Protocols that enable Fault Management functions of OAM toolset are comprised of protocols that perform proactive and on-demand defect detection and failure localization.

Bidirectional Forwarding Detection (BFD) has been designed as proactive Continuity Check protocol. defined extension to support Connectivity Verification in MPLS-TP networks . Following BFD specifications can be used in overlay networks: BFD for point-to-point as defined in , , , , , and ; BFD for multipoint network as defined in and ; S-BFD as defined in and ;

. Bit-Indexed Explicit Replication (BIER) provides the multicast service. For that BFD over multipoint network and are the most suitable of BFD family presents IP/UDP format of BFD over BIER in MPLS network.

Proto field MUST be set to IPv4 or IPv6 vlalue. Note that IP Destination address in must follow Section 7 , i.e. ?the destination IP address MUST be randomly chosen from the 127/8 range for IPv4 and from the 0:0:0:0:0:FFFF:7F00/104 range for IPv6.? BFD packets in the reverse direction of the BFD session will be transmitted on IP network to the IP address mapped to the BFIR-id and the destination UDP port number set as source UDP port number of the received BFD packet. IP/UDP format presents overhead, particularly in case of IPv6 address family. Thus option to avoid use of extra headers for OAM seems attractive. presents G-ACh format of BFD over BIER in MPLS network. Proto field of the BIER header MUST be set to OAM value. BFD control packet follows the BIER OAM header as defined in . According to the Section 3.1 of , Ver is set to 1; BFD control packet over multi-point without or with active tail accordingly identified in Message Type Field. The Proto field ?is used to define if there is any data packet immediately following the OAM payload?.

There is currently no WG document on proactive CC/CV. The individual requirements document covers this and there is a related proposal for BFD over VXLAN in .

On-demand Continuity Check and Connectivity Verification protocols include: MPLS Echo Request/Reply, a.k.a. LSP Ping, as defined in and its numerous extensions; LSP Self-ping, as defined in ; is a good example of generic troubleshooting and defect localization tool that can be extended and suited for more specific requirements of the particular type of an overlay network;

defines format of Echo Request/Reply control packet and set of TLVs that can be used to perform failure detection and isolation in BIER domain over MPLS network.

There is currently no WG document for on-demand CC/CV. Individual documents exist for tracing such as , and .

There is currently no WG document on Alarm Indication Signal. The individual draft suggests reusing ICMP errors for defect indications.

These protocols may be considered for Overlay Performance Measurement (PM) OAM: packet loss and delay measurement in MPLS networks, as defined in with ability to export measurement results for post-processing ; One-Way Active Measurement Protocol (OWAMP), as defined in , and Two-Way Active Measurement Protocol (TWAMP), as defined in , , and ; use of the Marking Method that, if accordingly supported by the overlay layer, can behave as close as technically possible to a passive method to measure performance, e.g. .

Requirements towards PM OAM for overlay networks are listed in the Section 4.2 . Two sets of performance measurement protocols had been developed at IETF so far: OWAMP and TWAMP each includes the control protocol to negotiate required parameters and control a test session as well as the test protocol itself that specify format and processing of a test packet. Historically, TWAMP, that enables measurement of the latency, packet loss both as one-way and round trip performance metric, has been implemented more often and thus gained wider deployment than OWAMP. There are several properties of the test protocol that may not be suitable for its use in overlay networks: the test protocol is targetted to IP layer and carries some IP-specific information; the format of the sent test and the reflected packets differ significantly and that complicates efficient HW-based implementation; latency and packet loss measurement operations are bundled together and that causes certain overhead when only one of performance metrics is to be measured; only Network Time Protocol (NTP) format of timestamp is currently supported that requires additional processing to convert from IEEE-1588 time format that broadly supported in many current packet forwarding engines. defines the test protocol that enables measurement of the latency and paket loss as one-way and round-trip perfomance metrics. Comparing to OWAMP/TWAMP RFC 6374 has certain advantages: the test protocol is flexible and these performance metrics can be measured independently or in the single test session; the protocol does not carry transport layer specific information; there's no difference between format of the packet transmitted by the sender and reflected by the responder as the test packets preallocates space for all necesary data it collects; both NTP and PTP time formats allowed to be used to record time for latency measurement. can be used as foundation of active PM OAM in overlay networks. The YANG data model of the packet loss and delay measurement based on can improve control and increase operational value of active performance measurement in overlay networks.

Currently there is no draft related to active PM OAM in the WG.

Performance management has been discussed in NVO3 but there is currently no draft in the WG.

describes how the Marking Method can be used in BIER domain over MPLS networks.

Marking has been discussed in NVO3 sessions, but there is no draft in the working group. The Generic Protocol Extension for VXLAN , Generic Network Virtualization Encapsulation , Generic UDP Encapsulation are just some examples of the new encapsulations to support network virtualization. NVO3 PM would be used to probe the NV Edge to NV Edge tunnels and NV Edge entity status for a DC network. The main requirement for Performance Management is to be able to support measurement of the frame loss, delay and delay variation between two NV Edge devices that support the same VNI within a given NVO3 domain on per VNI basis. Alternate Marking Method enables calculation of these metrics but sets forth requirements toward overlay encapsulation to make use of the AMM behave in the network as passive OAM per definition in .

In the SFC architecture SF, SFF, Classifier and NSH Proxy Agent are the elements that can incorporate the measurement agent functionality to support SFC performance measurement. The required OAM Performance Measurement, as described in highlight the capability to assess the monitoring at SF and SFF or a Set of SF/SFF, both in case of SFC-aware SF and SFC-unaware SF; the monitoring of SFP (and RSP) that comprises a set of SFs that may be ordered or unordered; the monitoring of the Classifiers operation and the monitoring of the SFC as a whole. Performance measurement includes measuring of packet loss, delay, delay variation and could be performed by the marking method proposed in . To make use of the marking method behave as passive OAM, as defined in , the overlay network encapsulation should allocate the field, preferrably two bits long, whose value does not affect how a packet is treated by the overlay network.

Excessive use of the in-band OAM channel may affect user flow and thus change network behavior. For example, if operator uses passive measurement exporting massive amount of data over the OAM channel may affect network. I think that a management channel should be used in such case. Obviously it may traverse the same nodes and links but may not require the same QoS. We can refer to LMAP Reference Model with Controller, Measurement Agent and Data Collector. proposes transport independent generic telemetry probe structure.

A common Overlay OAM header should be defined to support demultiplexing of OAM protocols. Existing modes of BFD protocol, primarily its Async mode, can be used either in IP/UDP or ACH format, as proactive continuity check mechanism in overlay networks. A new control packet to be used for on-demand CC/CV in overlay networks should be defined. Set of common TLVs may be defined while more specific TLVs to be defined by respective groups of experts. can be used as the foundation of active performance measurement OAM in overlay networks. YANG data model of the active PM OAM in overlay networks would improve control and increase operational value of the test methods. Performance measurement includes measuring of packet loss, delay, delay variation and could be performed by the marking method, for example as proposed in . To make use of the marking method behave as passive OAM, as defined in , the overlay network encapsulation should allocate the field, preferrably two bits long, whose value does not affect how a packet is treated by the overlay network.