]> Ciena

ndavis@ciena.com

Ciena

rrokui@ciena.com

Airtel

Praveen.Maheshwari@airtel.com

Jio Reliance

uday.n.joshi@ril.com

Vodafone/Idea

bhavit.vadhadiya1@vodafoneidea.com

Vodafone/Idea

xitij.dave@vodafoneidea.com

Futurewei Technologies

aihuaguo.ietf@gmail.com

Routing Area CCAMP Working Group coherent photonic pluggable plugs CMIS I2C OpenConfig Optical Packet This draft presents an additional architectural option for control of packet over optical networks by extending and complementing the IETF draft-poidt-ccamp-actn-poi-pluggable and by introducing an additional approach for control of optical plugs in packet devices. It provides the direct read/write access of coherent plugs to the optical controller, thereby allowing the end-to-end optical service management. The approach, introduced in this draft, can be further generalized to support other uses cases. The architectural option for control of packet over optical networks, introduced by this draft, also provides a clear separation between control of packet functions and control of optical functions. The packet and optical controls are partitioned so that the functions specializing in control of the optical capabilities can control corresponding functions in packet devices, optical devices or both without interfering with packet control functions. About This Document The latest revision of this draft can be found at . Status information for this document may be found at . Discussion of this document takes place on the WG Working Group mailing list (), which is archived at . Source for this draft and an issue tracker can be found at .

Terminology The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT" "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in th document are to be interpreted as described in . The following terms abbreviations are used in this document:

• Coherent plug: Coherent optical module refers to hot-pluggable coherent optical transceiver that uses coherent modulation (BPSK/QPSK/QAM) rather than amplitude modulation (RZ/NRZ/PAM4) and is typically used in high-bandwidth data communications applications.
• Coherent pluggable: Another term for coherent plug
• Optical controller: The control functions specializing in management/control of optical and photonic functions (virtual or physical) including network provisioning, assurance, viability analysis, network protection and restoration, network rebalancing and so on.
• Packet controller: The control functions specializing in management/control of packet functions (virtual or physical).
• MDSC: Multi-Domain Service Coordinator. See Framework for Abstraction and Control of TE Networks (ACTN) RFC 8453
• PNC: Provisioning Network Controller. See Framework for Abstraction and Control of TE Networks (ACTN) RFC 8453
• P-PNC: Packet PNC. Same as Packet controller
• O-PNC: Optical PNC. Same as Optical controller
• ROADM: A reconfigurable optical add-drop multiplexer that enables switching of bands of photonic spectrum (or wavelengths) from a wavelength-division multiplexing (WDM) system.
• Transponder: An optical transponder, short for "transmitter-receiver," is a device in optical communication systems which converts incoming electrical signal into optical signals for transmission over a fiber-optic network and vice versa. Optical transponders are essential components in modern fiber-optic networks and are used for various purposes, including signal regeneration, wavelength conversion, signal multiplexing, and format conversion.
• Muxponder: A muxponder, short for "multiplexer-demultiplexer-transponder," is a device used in optical communication systems to aggregate multiple lower-speed data streams into a higher-speed signal for transmission over a fiber-optic link. It also performs the reverse function by demultiplexing and converting incoming high-speed signals back into individual lower-speed data streams at the receiving end. Muxponders are commonly used in wavelength division multiplexing (WDM) systems to optimize the utilization of optical network resources and increase data capacity.
• DWDM: Dense wavelength-division multiplexing is an optical fiber multiplexing technology that increases the bandwidth of fiber networks. DWDM combines data signals from sources over a single pair of optical fibers and it maintains separation of the data streams
• CMIS: The Content Management Interoperability Services defines a management communication interface together with a generic management interaction protocol between hosts (e.g., packet devices) and managed modules (e.g. optical pluggable). See * Coherent plug: A small form factor module, supporting a transceiver with coherent optics, that plugs into a module in a device such as an IP router and is used in high-bandwidth data communications applications. It is typically hot-pluggable.
• FEC: Forward Error Correction where the transmitter sends additional data that enables the receiver to correct errors in the received signal
• gNMI: gRPC Network Management Interface. gNMI is an gRPC-based protocol for the both modification/retrieval of configuration from a target device, and control/generation of telemetry streams from a target device to a data collection system. The intention is that a single gRPC service definition can cover both configuration and telemetry. All messages within the gRPC service definition are defined as protocol buffers.
• I2C: Inter-Integrated Circuit (pronounced as “eye-squared-C”), alternatively known as I2C or IIC, is a synchronous, multi-master/multi-slave, packet switched, single-ended, serial communication bus and is widely used for attaching lower-speed peripheral ICs to processors and microcontrollers in short-distance, intra-board communication.

Introduction Packet traffic has been transferred over optical networks for many years blending the benefits of optical transmission and switching with packet switching. Optical systems have been separated from packet systems, both of which have had specific dedicated devices. In many existing network deployments, the packet and the optical networks are engineered, operated and controlled independently. The operation of these packet and optical networks is often siloed which results in non-optimal and inefficient networking. Both packet and optical systems have had relatively independent evolution. Optical systems have been developed with increasing capacity especially with the emergence of coherent optical techniques. Optical component design has continued to improve density to the point where a whole coherent optical terminal system that use to require many circuit packs can now fit onto a single small form factor "coherent plug". Placing coherent plugs in a device with packet functions can reduce network cost, power consumption and footprint as well as improve data transfer rates, reduce latency and expand capacity (although in some cases, other engineering and deployment considerations still lead to separate packet and optical solutions). Optical transmission/switching is analogue and requires complex and holistic control. Consequently, coordination of control of the coherent plugs (in a device with packet functions) with the control of the rest of the optical network is highly desirable as this best enables robust network functionality and simplifies network operations. The combination of these above trends along with the desire to select best in breed components has led to the emergence of open optical plugs that offer a standard bus for traffic and that use CMIS [OIF CMIS] for control. These plugs are such that a plug from vendor X can be installed in vendor Y's device with packet functions. Whilst basic applications can be handled by standardized modes of transmission such as ZR , to achieve optimum performance vendor proprietary modes are necessary. For many applications especially those in the core of the network where longer haul routes are prevalent, amplification and photonic switching is necessary. This leads to networks that utilise optical plugs in devices with packet functions interconnected to a ROADM mesh often including regenerators (where optical-electrical-optical conversion is necessary). Although in ZR applications it is possible to interoperate between plugs from different vendors, in the more strenuous core environments each optical path is terminated at each end using a plug from the same vendor. The optical plug encapsulates significant sophisticated photonic functions which often require specialist adjustment. As noted, the complex analogue nature of optical technology leads to the need for holistic control end to end (including the optical capabilities of the pluggables). The control functionality can be considered independent of specific functional deployment. However, it is important that any deployment approach does not significantly fragment the optical control. Several deployment approaches are examined:

• Network technology based partitioned domain controllers (i.e., separate packet and optical controller devices), with an optional higher level controller to deal with overall network abstractions
• Single controller dealing with all network technologies
• Single control fabric in which components, from various vendors, each focused on a specific control function, interact as peers to provide holistic control of the network

Likewise, the network capability can be considered in terms of functions independent of specific deployment. Control of a function including explosure of parameters etc. should be the same regardless of the hardware configuration of the function deployment. An important consideration is the approach to accessing the functionality. This is considered in detail.

Architectural Approaches for Control of Converged Packet Over Optical Networks Since the complete automation and control of packet and transport networks is vital for meeting emerging demand for high-bandwidth use cases, different Standards Development Organizations (SDO) proposed several approaches on how to control and manage the converged packet over optical networks where there are transponders/muxponders in the network or where there are coherent plugs in packet devices. For example, as proposed by an architecture analysis has already been carried out by the MANTRA sub-group in the OOPT / TIP group (Open Optical & Packet Transport / Telecom Infra Project). Also IETF considers the applicability of IETF Abstraction and Control of TE Networks (ACTN) architecture to Packet Optical Integration (POI) in the context of IP/MPLS and optical internetworking. Two architectural options to plug control are explored in which extends to the use cases where the DWDM optical coherent plugs are equipped in the packet device. To manage both optical and packet networks along with coherent plugs, section 2 of draft has proposed two architectural options, namely:

• Option-1: Dual SBI management of packet devices (i.e., IP routers)
• Option-2: Single SBI management of packet devices (i.e., IP routers)

shows option-1 where the packet devices allow the dual management, i.e., it allows both the packet controller and the optical controller have access to the coherent plugs on the packet devices. The interface between optical controller and packet devices are read-only and allows the following tasks:

• Device discovery, poll or stream configuration, state and static capabilities
• Performance monitoring, periodically poll or stream performance counters
• Fault notification, received asynchronous alarm notifications

All configuration operations on the coherent plugs are carried out by the packet controller.

Option-1 Dual SBI management Scenario

In option-2 shown in , the packet controller is the only control component which has access to the packet device, including the pluggables. The packet controller implements all management capabilities. The packet controller is in charge of the entire management life cycle of the pluggables including discovery, configuration/adjustment, monitoring performances and faults (via the asynchronous notifications coming for the pluggable). The information collected needs to be exposed to the optical controller via the packet controller NBI and higher layer controller. The information includes physical impairment data needed for the computation and validation of the optical channel. For both option-1 and 2, the higher layer controller MDSC is required not only to coordinate the end-to-end optical service configuration, but to provides the multi-layer coordination between IP and optical layers.

Option-2 Single SBI management Scenario

This draft extends draft by providing an additional architectural option-3 to control the multi-layer packet optical network when optical coherent plugs are equipped in the packet device. This third option enables direct control of the coherent plug by the optical controller.

• Option-3: Read/Write Optical controller access with dual SBI management on packet devices

The upcoming sections introduce the additional architectural option-3 for control of the coherent plug, identifying appropriate interfaces and parameters to be manipulated. As part of this explanation the networking aspects of control are considered and some use cases exploring initial planning, commissioning, general operation, restoration and adjustment are developed.

• Section 4 describes the architecture of option-3
• Section 5 outlines the details of the new architectural approach
• Section 6 describes the packet over optical architectural requirements for achieving full automation
• Section 7 demonstrates how option-3 addresses the packet over optical architecture requirements
• Section 8 describes a few multi-layer use-cases in a packet over optical networks

The YANG models are out of scope of this draft.

Architectural Option-3 for Control of Converged Packet Over Optical Networks As explained in Section 3, draft has proposed two architectural option-1 and option-2 to control both optical and packet networks along with coherent plug. This section explains an additional architectural option-3 shown in for control of the coherent plug, identifying appropriate interfaces and parameters to be manipulated. As part of this explanation the networking aspects of control are considered and some use cases exploring initial planning, commissioning, general operation, restoration and adjustment are developed. In option-3 the packet device has two management interfaces (A) and (B). Management interface (A) allows the packet controller read/write access to the packet functions of the device and management interface (B) allows the optical controller read/write access to the coherent plug functions. The packet device can realize the management interfaces (A) and (B) as a single physical interface or two separate interfaces. More details are provided in section 5. Unlike option-1 and 2 and where the higher layer controller MDSC is mandatory, in option-3 the MDSC is not needed for provisioning of the end-to-end optical services, which simplifies the design of MDSC and minimize the information flow between various controller. Having said that, option-3 can benefit from "higher layer controller" to provide various multi-layer packet over optical capabilities and operational benefits to operators such as multi-layer optimization, multi-layer assurance, multi-layer resiliency/protection/restoration multi-layer planning etc.

Option-3 Dual SBI architecture with Read/Write Access

Solution Details of proposed New Architectural Approach In a network with packet and optical devices (including converged devices), it is important for the packet controller and the optical controllers to have direct control of the corresponding capabilities in the devices. In other words, a packet controller should have direct control of the packet capabilities and an optical controller should have direct control of the optical capabilities of the device regardless of their physical assembly. The rationale and implications of this statement are explored in the following subsections.

Optical Plug in a Device with Packet Functions shows a packet device with an optical plug and also shows a connected optical device to the right. It highlights the control of the packet and optical functions and emphasizes that these functions can be decoupled such that there is no overlap between the optical and packet control functions. The shows the packet device in more detail. It shows interfaces (A), (B) and (C) as in but also exposes some internal detail highlighting:

• Separation of packet and coherent plug data where the packet controller has read/write access to the packet device data through (A) and the optical controller has read/write access to the coherent plug data through (B).
• Read only data to be made available through (D) as there is some information that both controllers need to see
• Access to coherent plug data to the plug through (E) and access to packet data to the packet data path through (F)
• The actual data flow between the coherent plug and the packet data function through (G)
• The flow of photonic signal through (H) that may also carry signalling from the optical device to the coherent plug

Network Device with packet function and an optical plug | Plug | | | | | Data | | Data | | | | | | | | | | | |-----------| |-----------| | | | . ^ | | | . (E) | | | | . | | | | . | | v (C) | . | | +---------+ | . (F) v | | | | |-------------| (G) |----------| (H) | Optical | | |Packet |<=======>| Coherent |===========| Device | | |Data function| | Plug | | | | |-------------| |----------| +---------+ | | | Packet Device | +---------------------------------+ Legend (A),(B),(C): Packet device, plug, optical device management interfaces (e.g., Netconf, gNMI, gRPC etc.) (D): Internal packet device dependency between plug and packet data (E): CMIS management interface via I2C (i.e., plug to packet device interface) (F): Internal interface between packet data and packet data function (G): Packet device data function Interfaces (H): Optical fiber connecting optical devices and/or optical pluggables ]]>

The packet device has several possible distinct realizations in :

1. Single config structure with both packet data and plug data in a single model tree where (A) and (B) are conceptually separate accesses, via separate sessions (at the same IP address)
2. Two distinct config structures, one for packet data and another one for plug data, where (A) accesses the packet data and (B) accesses the plug data. Interfaces (A) and (B) have distinct IP addresses and may use different transport protocols (e.g., interface (A) may use Netconf and interface (B) may use gRPC).
3. Two distinct config structures, one for packet data and another one for plug data, where (A) accesses the packet data and combination of (B) and (H) provide R/W access for plug data in some combinations
4. Two distinct config structures, one for packet data and another one for plug data, where (A) accesses the packet data and another method provides plug direct method (e.g., out of band management via data path)

Realizations (2), (3) and (4) above have clear separation between access to packet and plug data, whereas realization (1) requires some form of access limitation. The following approaches might be employed to provide this access limitation:

• Partition the responsibility by trust where it is understood by the designers of each controller what access the controller should have
• Provide restricted control using access control list protection
• Include use of config locking to ensure partial configs are not applied (as some configuration actions may require several parameters to be set to define a consistent configuration)

There is a need for the packet and plug data path functions to be configured at interface (G) to match each other. The configuration properties might include the number of lanes, protocol, electrical characteristics and FEC settings. To understand appropriate solutions for this, it is necessary to assess the likely approach to network operations. The choice of pluggable (where not yet installed) and settings of the pluggable, at any stage during installation and operation, are determined as a result of optical network analysis, including photonic path viability (See Appendix B), by the optical controller. To carry out this analysis, the optical controller needs to be aware of which types of pluggable the packet device can accommodate and also of the capabilities of those pluggable types. On completion of the analysis the selection of pluggable, where not already set, and pluggable configuration including CMIS AppSel, power, frequency, various shaping parameters and necessary settings for interface (G) are known. As the optical controller has all the necessary data, it is in the best position to set the pluggable properties directly. Considering this, interface (G) configuration can be performed in one of the following ways:

1. The settings applied by the optical controller are applied to both coherent plugs and packet data function at interface (G)
2. The settings applied by the optical controller to coherent plugs and are made visible to the packet controller via (D) and the packet controller configures the packet data functions at (G) to match

Note that the detail of the method used to provide data via (D) depend upon the realization approach. The data model for the coherent pluggable could be derived from any YANG data model such as OpenConfig, OIF etc. where the appropriate part of that YANG model could be mounted in the single YANG tree.

Optical Network Configuration and Operation This section explores different levels of optical network configuration and operation complexity. Networks using grey optic where no significant optical control is required are in scope of this document. This document focuses on networks where the DWDM is used to establish optical services which may include optical switching, amplification and regeneration using optical contrtollers. A practical deployment of packet over optical network might also be a mix of cases of grey optics, directly connected DWDM, switched DWDM etc. The networks are explored in more detail in the appendix (A).

Optical network boundary The optical network essentially includes the amplifiers, switches, transponders (including pluggables) and all other components that work with photonic signals as well as those that work with the electrical signals directly applied to the photonic signals. At the transponder, it includes the adaptation functions that support the client layers (i.e., packet function). The adaptation function includes the capability to allow multiple wavelengths to carry a single signal etc.

Optical network complexity and configuration/control implications Optical networks complexity depends upon transmission distance, need for flexibility in the photonic layer and need for photonic resilience. For directly connected DWDM, there will be a need for configuration of basic optical parameters during commissioning whereas for cases where there is longer reach, switching and/or restoration there is a need for sophisticated network analysis to determine viability and need for more complex parameter settings. For very long reach cases there may be a need to include optical regeneration (where the signal is taken from optical through electrical back to optical (O-E-O)). The regeneration node is not a packet device and is wholly controlled by the optical controller. Knowledge of the adaptation options in the regenerators and in the packet device is necessary for the correct choice and setup of regeneration nodes. In all cases, it is necessary to evaluate options to determine the best optical network solution [see for engineering considerations]. Hence, in a network where there are various solutions, even for situations where a basic interconnect turns out to be the right configuration, it is necessary to analyse the network in detail to determine this. Hence, it is necessary to involve the optical controller in all decisions about optical network configuration. For some restoration cases there is a need for near real time optical configuration. The pluggable settings may need to be adjusted during restoration control actions. For example, it is possible that regeneration may be identified as required during a restoration activity and that as a result of this or other considerations, optical parameter changes, including wavelength and power may need to be applied to a pluggable during normal operation.

Coherent plug commissioning and operation Commissioning of more capable, and hence expensive, coherent plugs in routers tends to follow a just-in-time deployment pattern where the pluggables are not installed in the router until they are required for service. These pluggables are used in more challenging applications that require sophisticated photonic viability assessment. The specialist photonic viability tools reside in the optical control function set. Where there is a need to install a new pluggable, the process will operate in a relatively slow time frame. Once the pluggables are detected the optical controller the parameters of the pluggables can be configured and progress through any necessary validation testing on the optical network. This testing may involve the need to control the pluggables optical parameters along with parameters of other devices supporting the optical/photonic signals.

Architecture of Converged packet optical Control Solution In deployments of converged packet over optical networks, there is a need for control functionality focussed on packet control (the packet controller) and control functionality focussed on optical control (the optical controller) as discussed in this document.

Generalized control Industry is progressing towards generalilzed optical viability functions (see Appendix (B)) but currently, these functions tend to be vendor specific and reside in the vendor controllers. Current deployments tend to have a generalized optical controller from a particular optical device vendor controlling other vendor optical controllers. The optical controller discussed in this document is the collection of all optical control capabilities including whatever assembly of vendor controllers and generalized controllers are present.

Control flow It is assumed that the overall request for capacity will originate from some use of the network, e.g., data transfer between data centers. This request will be analyzed against current network capability. In some cases it will be identified that an enhancement to the optical network is required. This is may:

• include the need for installation of new pluggables in existing routers
• entail rerouting of existing optical services
• enhancement to existing optical services to include protection/restoration

It is for the optical controller to determine network viability and most appropriate plug choice and configuration. Relevant bus lane configuration and adjustment will also be determined.

Control Solution structure A control solution will normally be built with a generalized topology model, potentially following or guided by a standard (such as IETF RFC8345, ONF TAPI, TMF SID etc.) around which functions dealing with technology specific control (for packet, optical, radio etc.) are arranged. These functions cover commissioning, provisioning and dynamic behavior for service setup, incident/problem analysis etc. Sophisticated control solutions (that follow a digital twin approach) close the loop taking action to ensure maintenance of both short-term and long-term intent. Current control solution architectures tend to be hierarchical in nature, partly driven by traditional strategies. The hierarchy often leads to the partitioning of control on a per-technology basis. This partitioning leads to packet domain controllers, optical domain controllers etc. The network control is pulled together by an orchestrator or high level controller. Many of the sophisticated optical control functions currently reside in a vendor specific controllers.

Separation of Control Solution In current networks, it is likely that there is a single optical domain controller overarching several lower level specific vendor controllers, where the assembly is considered to be the single optical controller and multiple packet controllers due to regional demarcation etc.

Evolution of Control Solution As solutions evolve it is likely that artificial domain partitioning and hierarchy will dematerialized in favour of vast-scale cloud based solutions. In these solutions, vendor controller demarcation will also dematerialize. This is essentially a disaggregation of the control solution. In this evolved solution, there will be a single point for each specific control consideration (e.g., single physical control capability, single upgrade capability) covering the entire network (bounded by commercial, political and regulatory considerations), working on any relevant grouping of things dependent upon the specific need at that moment. In this solution, each control capability will be appropriately independent and will coordinate with peers. These independent control capabilities will have the necessary direct access to the relevant parts of devices that they are responsible for. It is also anticipated that an intent (outcome oriented constraint based) interaction approach will apply at all points in the solution (including to the devices). Clearly, some interface technologies will be better suited to this style of interaction than others. Ideally, device capability exposure will match the division of control responsibility and appropriate views will be offered to each control function. Some traditional interfaces that expose a monolithic model will need to utilize locking strategies to account for multiple clients even where the responsibility demarcation is clearly such that there will be no collision of control.

Architectural Requirements to Achieve full Automation in Packet over Optical Converged Networks The automation of the multi-layer multi-domain packet over optical networks (i.e., IP/Optical convergence) is an area which captures the attention of most service providers. Any control architecture which covers the full automation of such networks shall address the following requirements:

R1: There shall be a "single functional entity" responsible for optical services life cycle management The following aspects of optical service life cycle shall be managed and controlled by a single functional entity in the network.

• Discovery of Optical networks inlcuding coherent pluggables, ROADMs, Amps, Regen, Transponder/Muxponder etc.
• Optical services viability
• End-to-end Optical services configuration/modification from plug to plug (or from transponder to transponder).
• Optical services telemetry collection and monitoring performances/faults including the asynchronous notificatons collected including coherent pluggables.

Please note that this requirement addresses the optical controller functional aspects but not how they are realized in the network and not how the information needed by the optical controller is collected from the network (See Requirement R2 on this aspect where there are serveral approaches to realization of an optical controller considered).

R2: There shall be a clear distinction between functional components of optical control vs. its realization In general, the industry agrees on the functional components that are needed for converged operations: there needs to be a multi-layer controllers that spans both the packet and optical domains in order to synthesize data from both domains and make optimal decisions regarding utilization of assets to deliver high-resiliency and high-performance network services. There is however a difference between packet and optical controllers functional components and their realization – there are different ways service providers can choose to deploy the multi-layer packet over optical controllers including coherent plugs to realize a solution that works in their specific operational environment. There are several realization approaches including a single control fabric where the optical and packet control functions are deployed in the cloud or simple distinct hierarchy etc. For examples shows a few possible schemes to realize the multi-layer packet over optical controllers. Please also note that in some realization approaches there is not need for higher layer controller as shown in "Realizatoin D".

A few Packet over Optical Realization Schemes

R3: Existing operational practices shall be supported Requirement R1 emphasizes that the packet and optical control architecture shall deal with any network deployment and administration approaches as coherent plugs are deployed without imposing significant change to the operator's current operational practices (including network planning, commissioning, provisioning, assurance, optimization and protection/restoration). As shown in , in a traditional packet and optical disaggregated (not converged) network, the packet devices are connected to optical network via transponders/muxponders (i.e., the optical nodes which do not process packets and just aggregating packet device traffic into a single optical channel). In this deployment model, the optical controller controls, plans and manages the end-to-end optical services between client ports of transponders/muxponders via the optical network. In this model, the existing operational practices related to optical networks assume a single optical controllers for the entire optical domain. The packet network is administered and controlled separately from the optical network. Both networks have dedicated management/control platforms etc. Appendix A and depicts other deployment models for packet over optical networks. There will be significant benefit to operators allowing them to continue to use their existing operational practices to provision and monitor optical services end to end either between transponders/muxpnders or between coherent plugs. For operators who have specific demarcation between operations of packet network and optical network (with separate physically partitioned controllers) as discussed, it is important that the introduction of converged optical-packet devices does not force a change to their existing operational practices. As a network evolves, the operational practice may need to change, however, it is clearly always the case that complex photonic networking will require sophisticated dedicated control functions regardless of how the administration is organized.

R4: Various existing YANG Data Models shall be accommodated The solution shall enable the use of various existing YANG data models, currently used to configure/monitor coherent transponders (e.g., OpenConfig, IETF etc.), for configuration/monitoring of coherent plugs in packet devices.

R5: Holistic control of optical network shall follow clear demarcation Where there is network technology based responsibility partitioning, the controllers should abide by the technology boundaries. The packet controller shall control packet functions and the optical controller shall control optical functions. The optical technology includes coherent terminal functions and hence these shall be controlled by the optical controller. The packet controller shall not need to be exposed to coherent plugs optical attributes. Optical technology is a separate, distinct and complex technology from packet technology.

R6: Higher level controller shall be optional The majority of the packet over optical deployments do not have or do not need a higher level controller. This requirement states that the existence of the higher level control shall be optional. Forcing the addition of a higher level controller makes the deployment more complex.

R7: Urgent optical control actions shall be supported in a timely manner During some of the operation and restoration/protection cycles of the converged packet and optical networks, urgent action on the configuration of the pluggable may be required where the decision to take the action and the action detail are the responsibility of the optical controller. For example, during the restoration and protection of the Optical services, there might be a need for re-tuning and re-coloring of optical services. This involves changes in both the coherent plugs and ROADM networks.

R8: The solution shall minimize fragmentation of optical parameter provisioning It is highly beneficial to closely coordinate setting of configuration parameter settings across the optical network including pluggables as inconsistent configuration can potentially cause disruption to other photonic paths.

R9: Access to the coherent plug properties shall be as transparent as possible It shall be possible to access all configuration information and telemetry data available from the coherent plug with no (or only limited) intermediate transformation of data. Transit through intermediate systems that process the syntax of the information, but that never take action on the information should be avoided.

R10: Network information shall take direct path to relevant controller It shall be possible to access all configuration information and telemetry data available from the coherent plug as directly as possible (ideally with no intervening transfer delays).

R11: Multi-layer operational benefits shall be addressed The multi-layer packet over optical capabilities and operational benefits among packet and optical controllers such as multi-layer optimization, multi-layer assurance, multi-layer resiliency/protection/restoration and multi-layer planning shall be addressed for full automation in a packet over optical networks.

R12: Coherent plug telemetry data shall be collected Telemetry data and any change in state should be provided by the plug to the optical controller via a stream in a timely manner.

R13: Mix of plugs and Transponders/Muxponders (inc. Regens) shall be supported Many optical services will have a coherent plug in a packet device at one end and a coherent plug, or coherent optics on some other circuit pack type, in a non-packet device (e.g., storage, application platform, optical regen) at the other end. The solution shall support all current and expected configurations in a uniform fashion.

R14: Optical deployments with protection/restoration shall be supported Some optical services will have protection. Some protection will be such that there are more than two ends (e.g., mixed layer protection) in the optical layer. This may be combined with regens, where there may be a regen in one protection branch and no regen in the other. The solution shall support all current and expected configurations in a uniform fashion.

R15: Evolution to expected future controller deployment approaches shall be supported The solution shall be designed to provide a clear evolution path to the probable future architecture (which is expected to be focussed on disaggregation of control). In this architecture it is expected that control components with specific focussed functions will have direct access to the corresponding functions in target systems and that asynchronous and simultaneous access to these functions will be vital.

R16: Evolution to future packet processing deployment approaches The solution shall be designed to provide a clear evolution path to support control of packet and optical functions deployed using emerging strategies (e.g., cloud based routing) where the routing functions are not constrained by physical boundaries. Operational approaches native to these environments should be supported.

How Option-3 Addresses the Architecture Requirements This section explains how the proposed architectural option-3 addresses the requirements covered in section 6 to achieve full automation in packet over optical converged networks. R1 - There shall be a "single functional entity" responsible for optical services life cycle management

• Requirement R1 emphasizes that one FUNCTIONAL entity is needed for life cycle management of the optical services. This requirment is supported by Option-3.

R2 - There shall be a clear distinction between functional components of an optical control vs. its realization

• This requirment is supported by Option-3 where it allows any combination of optical, packet and higher layer controllers to be realized in operator's networks.

R3 - Existing operational practices shall be supported

• Requirement R3 emphasizes that the packet and optical control architecture shall deal with any network deployment model without imposing significant changes to the operator's current operational approach in the area of network planning, commissioning, provisioning, assurance, optimization and protection/restoration).

• Option-3 covers deployments where there is already a significant optical solution, that solution is administered and controlled separately from a packet solution (with strong administrative demarcation between optical functions including coherent terminals and packet devices) and deployment of coherent plugs is occurring. Option-3 enables the same workflow for coherent pluggables as used with their existing coherent terminals, i.e., no need to change the current operational approaches. This is a significant benefit as the plug deployment progresses.

• In networks with limited deployment of basic optics where the business focus is mainly packet, there may be a single controller dealing with both the packet and the basic optical network. If more sophisticated optical capabilities are deployed and optical controller will become necessary. The operator may then choose to separate the optical and packet control, where option-3 focusses, or may choose to unify this control. Unified control will benefit from the direct access to the photonic capabilities of the packet device, as per option-3, especially where protection/restoration is required.

• In networks where there is limited packet deployment, where the business focus is mainly optical, there may be a single controller dealing with optical and the limited packet deployment. Option-3 maintains the direct control of optical components when deployed as coherent plugs on packet devices.

R4 - Various YANG Data Models shall be accommodated

• Requirement R4 states that the solution shall support various existing YANG data models used for current coherent optical solutions for coherent plugs (and various interfaces between the packet and optical controllers).

• Option-3 supports this requirement. Referring to , the optical controller can use packet device management interface (B) to control the coherent plugs. This interface can for example use the OpenConfig YANG data model , and over gNMI or any other YANG data models over Netconf, gNMI etc. As a result, the optical controller will have a holistic view of entire optical network from plugs to plugs.

R5 - Holistic control of optical network shall follow clear demarcation

• Requirement R5 states that the network controllers shall abide by technology boundaries.

• Option-3 inherently supports this requirement since the optical controller has access to the entire optical network for both provisioning/fulfilment and monitoring/assurance of optical services from plug to plug including all optical parameters.

R6 - Higher level controller shall be optional

• Option-3 inherently supports this requirement as there is no need for the higher level controller to be present to configure the coherent plug (since the entire optical network is controlled, managed and monitored directly by a single optical controller).

R7 - Urgent optical control actions shall be supported in a timely manner.

• Option-3 provides direct access from the optical controller to the coherent plug minimizing the time taken to apply any changes it has determined are necessary. The aim is to minimumize the round trip from detection of an issue in the coherent plug to application of any necessary action. This is achieved by direct access to data and to control of the coherent plug.

R8 - Solution shall minimize fragmentation of optical parameter provisioning.

• Option-3 minimizes fragmentation of optical parameter provisioning as the optical controller has direct access to all devices with optical capabilities including to the coherent plugs in packet devices.

• In many deployments, there will be multiple packet controllers with one optical controller. Clearly, localizing the direct control of the entire optical network to one optical controller minimizes fragmentation of control of the end-to-end optical services (whereas spreading the responsibility of optical services across the multiple packet controllers increases the fragmentation).

R9 - Access to the coherent plug properties shall be as transparent as possible.

• Option-3 offers direct access to the coherent plugs from the optical controller minimizing the number of intervening functions/devices. It is the optical controller that needs access to the parameters of the coherent plug. The packet controller does not need access to these parameters. Option-3 does not require the packet controller to interpret or map the parameters.

• In many deployments, there will be multiple packet controllers, potentially from different vendors, with one optical controller. In these deployment scenarios communication between optical controller and multiple packet controllers either directly or via a higher controller introduces many opportunities for misinterpretation and inappropriate mapping. As a result there is a greater likelihood of failure.

R10 - Network information shall take direct path to relevant controller

• Option-3 optimizes the path from optical network functions including the coherent plug to the optical controller.

• Communication between optical controller and packet controllers either directly or via a higher controller makes the path for communication of the optical parameters longer.

R11 - Multi-layer operational benefits shall be addressed

• It is beneficial to treat the network as a whole. Considering a packet over optical network, multi-layer operational benefits such as multi-layer optimization, multi-layer assurance, multi-layer resiliency/protection/restoration and multi-layer planning need to be achieved.

• Option-3 enables several solution approaches including: - a control hierarchy with a higher controller coordinating the optical controller and packet controllers - a single controller that provides an optimum integration of optical and packet control (both peer and hierarchical strategies as appropriate) - a disaggregated control solution where specialist functions interact together and with the network to provide a view of the whole network

R12 - Coherent plug telemetry data shall be collected

• Option-3 provides direct access to plug telemetry for the optical controller.

R13 - Mix of plugs and Transponders/Muxponders (inc. Regens) shall be supported

• Option-3 enables the optical controller to have direct access to coherent optical functions in devices regardless of the device.

R14 - Optical deployments with protection/restoration shall be supported

• Option-3 provides the optical controller with a full view of the optical protection/restoration schemes. Since in option-3 the optical controller has a complete view and control of end-to-end optical services between coherent plugs, upon any optical failure, it can perform the optical restoration/protection by calculating the new optical path, performing the optical viability on that path and establish the new optical path in the network. The optical controller has full knowldedge to perform all these actions.

R15 - Evolution to expected future controller deployment approaches shall be supported.

• Editor's note: This section will be completed.

R16 - Evolution to future packet processing deployment approaches

• Editor's note: This section will be completed.

Packet over optical Use-cases This section considers: - network planning focussing on development of a solution to the demand for more capacity in the packet network - Restoration of optical service

Use-case: Optical service creation to support creation of a new IP link shows a network consists of routers A and B. The intention of this use-case is to create a new IP link with 400 Gbps bandwidth between router ports which are occupied with coherent plugs A and B. However before doing so, the connectivity between plugs should be establish on optical network by creating a new optical service from plug A to plug B. In this use-case the new IP link will be mapped to a single optical service, i.e., there is 1:1 mapping between the IP link and optical service. So, the 400G IP link is mapped to a single 400G optical service because the optical network is capable of providing the 400G optical service, i.e., the optical service is viable. Use-case 8.3 will cover the viability where the optical service is not capable of providing 400G service. In this case, since the underlay connectivity between plugs A and B does not exist, the creation of the IP link workflow (which is part of the packet controller), should first invoke the optical controller to create a new 400G optical service from plug A to plug B considering the other optical devices in optical network (i.e., ROADM). When the optical service is created, the packet device controller can create the new IP link. Option-3 proposed in this draft will natively support this use-case since the interaction between packet and optical controllers just needed when the optical controller notifies the packet controller at the end of optical service creation. All other aspects of optical service creation is supported by optical controller because it has a complete visibility on optical network including the coherent plugs.

Use-case#1: Optical Service Creation to Support Creation of a New IP Link

Use-case: Optical service modification to support increase/decrease B/W of IP link shows the packet over optical network where the optical service between plugs A and B has been moved to a restoration path due to an issue on optical network (e.g., fiber cut). However, the optical network was not able to provide the original 400G data rate but in lower rate of 200G. As a result, the following tasks are needed to happen and option-3 should be able to addressed them:

• The reduced new optical service rate from 400G to 200G should be communicate to coherent plugs for further re-tuning and re-alignment
• The reduced new rate needed to be communicate to packet engine on both routers A and B which allows them to further re-configure the IP ports A and B accordingly [Reza any other aspects???]

Use-case#2: Optical Service Modification

Use-case: Considering the optical viability during IP link creation Optical transmission is an analogue technology where success is influenced and impacted by real physical conditions and where determining viability is complex. Other than for the most basic short direct optical links, it is necessary to employ optical viability tools to identify necessary intermediate components and define optimum optical set-up. Optical components are relatively expensive and are often not deployed in the network until needed. As a consequence, there is often no simple potential link opportunity, and instead understanding of optical interconnectability relies on knowledge of semi-abstracted interbuilding fibering, potential plug capabilities and device with packet functions compatibility. The combination of these two considerations means that it is often not possible to simply turn on an existing physical setup to cause further link capacity to be realized.

Use-case: Interaction between coherent plugs and optical network during restoration/protection of optical services is same network as where the details of the optical network is removed. This use-case covers the interaction between coherent plugs and the optical network when the optical services switches to the restoration/protection paths (due to a fault in the optical network such as fiber cut). When the optical service moves to the new restoration/protection path, the new optical path's characteristics such as optical power, optical frequency (i.e., Lambda or optical channel), optical model etc. might change which should be communicated to the coherent plugs. As a result, there is a need for communication and interaction between coherent plugs and optical network on fiber links (H1) and (H2). Since option-3 proposed in this draft provides the control of the coherent plugs and optical network on one optical controller, interaction between plugs and optical network on fiber links (H1) and (H2) is native and supported. No additional controller or network element needed to support the optical service restoration/protection.

Use-case#3: Interaction between Coherent Plug and Optical Network during optical service restoration/protection

Use-case: Plug coordination to support Optical network rebalancing/de-fragmentation we need to talk about re-coloring and re-assignment during the de-fragmentation which needs plug coordination with optical network This use-case has some similarities with use-case #4 but is more complex

Uses-case related to monitoring and telemetry collection Editor's note: This chapter will be completed.

Use-case when one side is plug and other side is xPonder Editor's note: This chapter will be completed.

Use-case for Regen Editor's note: This chapter will be completed.

Use-case: Optical service life cycle management Initial configuration

• • Many devices with packet functions are interconnected by photonics
  • Devices with packet functions have spare plug slot capacity
  • Service requirements are growing and changing over time

Requirement for additional capacity:

• Packet network balancing capability (in orchestrator or IP controller) identified additional bandwidth requirement between devices with packet functions:

• • This is probably a projection forward in time to allow procurement and/or delivery
  • The relevant information needs to be available to the orchestrator
  • There may be many alternative rearrangements

Developing the solution:

• Orchestrator requests optical control capabilities (planning, routing, provisioning etc.) to determine appropriate details to connect relevant devices with packet functions:

• • There may be several choices of device with packet functions, the bandwidth provisioning may have various options.

• Optical control identifies optimum route(s), determines use of ROADMs/Regens etc. and selects appropriate plugs with settings:

• • There may be options for plug procurement and use of device with packet functions plug slots
  • The control functions need to know CMIS lane capability of the equipment in the device with packet functions as some options may be eliminated based upon lane capability (speed/lane count etc.)

Evaluating the solution:

• Depending upon policy the optical controller:

• • If policy dictates that orchestrator must evaluate options (and there are options). Optical controller passes details to the orchestrator of the resolved intent request. The optical controller provides abstracted info to the orchestrator for evaluation. The orchestrator selects option and informs the optical controller.
  • If the optical controller is in charge of selecting a single solution (or there is only a single solution). Optical controller informs the orchestrator of the solution.

• Note that the solution has plug type, plug setting, fiber interconnect and lane setting details. Optical controller now has optical intent and full solution.

Acquiring the plug

• • Plug procurement or shipping is initiated
  • Work orders are initiated for plug installation and cabling where necessary

Lane settings:

• If the optical controller is in charge of the CMIS bus (lanes etc.) in the device with packet functions, then these are set. If not, then the orchestrator passes lane settings to the packet controller as part of link intent (the orchestrator was provided with these settings by the optical controller earlier in the process).

• • The packet controller uses the link intent with lane constraints to determine lane setting, FEC, etc.
  • The device with packet functions sets its side of CMIS (lane config (and separately FEC))to match the plug lane configuration as defined by intent passed to packet controller by the orchestrator (as determined by the optical controller).

• Some packet settings require to match at both ends, if the packet controller does not have visibility of both ends then this coordination must be performed by the orchestrator. The packet controller may pre-set the lanes etc. in the devices with packet functions (or it may leave this till optical configuration is detected).

Plug installation and power-up:

• When plugs are installed in the device with packet functions, the plug is powered up. Plug power-up to low power is the responsibility of the optical controller within the power budget of the device with packet functions.

• Once the plugs are powered up to low power state, the optical controller coordinates optical provisioning across plugs, ROADMs etc., including for the plugs entering full power-up. The optical controller sets specialist photonic parameters and other configuration as appropriate. Lane configuration of the plug and CMIS settings are achieved via the provision of AppSel value, via other parameter settings etc. (note that lane settings may require tweaking off-default for specific device with packet functions capability).

Commissioning:

• Optical controller tests plug-plug and coordinates any loopbacks, PRBS settings, measurements along the path etc.

Path enabled:

• The optical controller enables optical path:

• • device with packet functions detects link
  • device with packet functions network uses new capacity

Conclusion This document has considered control of optical plugs in devices with packet functions. Three specific control solution deployment strategies were examined:

• network technology partitioned domain controllers, with an orchestrator (higher level controller) to coordinate the domain controllers
• single controller dealing with all network technologies
• single control fabric in which components, from various vendors, each focused on a specific control function, interact as peers to provide holistic control of the network

Whilst the deployment strategies apply to control of a network in general, this draft focused on control of the optical network and in particular the control of the optical plug in a device with packet functions. The orchestrator strategy and the single controller strategy essentially exist to a degree in solutions today. The single control fabric strategy is an anticipated evolution of the control solutions. For all of the examined control solution deployment strategies, the control functions specializing in photonics determine the optical network setup on-going and these control functions directly control the photonics of the network including the optical plugs in devices with packet functions. Various indirect control options are not discussed in this draft. There is a clear separation of concerns between packet function control and optical function control such that the control can be partitioned so that control functions specializing in control of the packet network can control corresponding functions in the device with packet functions with no interference from the optical control functions and vice versa. The optical control functions do not control any aspect of the packet functions in those devices. To ensure that there are no transaction issues, locking of the config is recommended. Direct control of the optical plug by the optical control functions (in the optical controller) appears to be a natural and valid approach.

Security Considerations

IANA Considerations This document has no IANA actions.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. Informative References

Multi-layer Packet Over Optical Integration A packet over an optical network represents an efficient paradigm that harnesses the power of both packet-switching and optical technologies. In this approach, the IP or MPLS packets are transmitted through an underlying optical network which combines the advantages of packet-switching and optical transmission. The fusion of packet and optical networks offers a host of advantages, including accelerated data transfer rates, diminished latency, and expanded network capacity. Various deployment models can be employed to deploy the packet over optical networks. The first approach is disaggregated solution as shown in . This solution involves the separation of IP network from Optical network allowing for greater flexibility, scalability, and efficiency in network management and operation. In this setup, the IP (Internet Protocol) layer responsible for routing and forwarding is decoupled from the underlying optical transport layer. This approach offers several benefits, including the ability to scale each layer independently, optimize resource utilization, and simplify network management through centralized software control. Disaggregation enables network operators to choose best-of-breed components for each layer, fostering innovation and competition in the networking industry. However, implementing and managing a disaggregated network also comes with challenges related to interoperability, integration, and maintaining end-to-end performance across the various layers.

Packet over Optics Disaggregated Deployment Model

The second approach is to shrink the functions of the xPonders into a single small form factor plugs (aka Coherent pluggables) and then place them directly into the packet devices as shown in . This is possible because the optical component design continues to improve density to the point where a whole transponder and muxponder which use to require many circuit packs can now fit onto a single small form factor pluggable. Placing this small form factor pluggable in a device with packet functions can reduce network cost, power consumption and footprint (when these benefits are not outweighed by other engineering considerations). Depends on the application, distance between packet devices, quality of fibers and so on it might be that there is no need for a ROADM network, i.e., direct connectivity between packet devices via plugs. By incorporating coherent plugs into routers, network operators can achieve higher data rates, greater spectral efficiency, and improved tolerance to optical impairments. This is especially valuable in scenarios where traditional electronic signaling might encounter limitations. Coherent plugs enable routers to leverage advanced modulation schemes, digital signal processing, and error correction techniques, enhancing their ability to handle complex optical signals. One of the key advantages of using coherent plugs in routers is the potential to bridge the gap between long-haul and metro networks, providing a seamless and efficient transition of data across various network segments. This technology can contribute to the evolution of high-speed data centers, interconnection between data centers, and the overall growth of data-intensive applications. However, implementing coherent plugs in routers requires careful consideration of factors such as power consumption, form factor, cost, and integration with existing network infrastructure. As technology continues to evolve, coherent plugs hold the promise of extending the capabilities of routers and optical networks.

Packet over Optics with Plugs Model

Another flavor of is shown in where the transponder and muxponder functionality is provided by plugs but there is still a need for ROADM network. The packet over optical architecture is important for long-haul use cases since they exhibit several distinctive characteristics that make them suitable for efficiently transmitting data over vast distances such as:

• High Capacity: Optical networks leverage the immense bandwidth of fiber-optic cables, allowing them to transmit a large volume of data simultaneously
• Low Latency: Optical signals travel at nearly the speed of light, resulting in minimal propagation delay. This low latency is crucial for real-time applications such as video conferencing, online gaming, and financial transactions.
• Long Reach: Optical signals can travel over extensive distances without significant signal degradation. This makes long-haul optical networks ideal for interconnecting geographically distant locations.
• Signal Regeneration: Even over long distances, optical signals need periodic amplification to maintain their strength. Long-haul optical networks incorporate regenerators or amplifiers at intermediate points to ensure signal integrity.
• Modulation Techniques: Advanced modulation schemes, such as coherent modulation, are used to encode multiple bits of data onto a single optical signal. This improves spectral efficiency and enables high data rates.
• Wavelength Division Multiplexing (WDM): WDM allows multiple signals of different wavelengths (colors of light) to be transmitted concurrently over the same fiber. This further enhances the network's capacity and efficiency.
• Optical Cross-Connects: Optical cross-connects enable flexible routing and switching of optical signals, allowing dynamic provisioning of network resources to accommodate changing traffic patterns.
• Error Correction: Long-haul optical networks employ powerful error correction techniques to mitigate signal distortions and losses, ensuring reliable data transmission.
• Resiliency and Protection: These networks often incorporate redundancy and protection mechanisms to ensure network availability in case of fiber cuts or equipment failures.
• Multi-Protocol Support: Long-haul packet over optical networks can carry various types of data, including Ethernet, IP, and other protocols, making them versatile for different applications.
• Network Management and Control: Centralized management systems monitor network performance, optimize traffic routing, and facilitate rapid provisioning of resources.

Packet over Optics with Plugs along with ROADM network Model

Appendix: Optical network viability Optical transmission is an analogue technology where success is influenced and impacted by real physical conditions and where determining viability is complex. Other than for the most basic short direct optical links, it is necessary to employ optical viability tools to identify necessary intermediate components and define optimum optical set-up. Optical components are relatively expensive and are often not deployed in the network until needed. As a consequence, there is often no simple potential link opportunity, and instead understanding of optical interconnectability relies on knowledge of semi-abstracted interbuilding fibering, potential plug capabilities and device with packet functions compatibility. The combination of these two considerations means that it is often not possible to simply turn on an existing physical setup to cause further link capacity to be realized.

Network with unequipped plugs The diagram below shows a network with three devices with packet devices each with two sockets for optical plugs with the plugs not equipped. This can be generalized to multiple sockets. The connectors for the optical plugs are depicted with "{" and "}" in the devices with packet functions. The devices with packet functions are assumed to be on separate sites (packet sites). The diagram also shows four devices with only optical functions, "~" (could be OLS, Amplifier, ROADM Regenerator, protection switch unit etc.) which are interconnected with fibers "=". The devices with packet functions are not yet connected to the optical network but there are fibers with connectors, "{" and "}", that will enable interconnection when the optical plugs are equipped that are accessible in the packet sites.

Devices with packet functions, with no equipped plugs, and a optical network

Clearly, in general in a running network, devices with packet functions would have some plugs equipped and would be interconnected to other devices with packet functions via active optical networking. The devices with packet functions would have some plug sockets empty, and this would allow for network expansion.

Network Deployment Contexts The following sections set out key network forms that may result from photonic viability analysis. In all networks a device with packet functions, "ixi", straddles the packet domain and optical domains with the packet function in the packet domain and the optical functions of the optical plug in the optical domain. The devices with packet functions are in "packet sites" that are some distance apart.

Basic direct connect To determine that direct connection is viable, photonic tools need to be used to validate reach etc. As discussed above, plugs may not be present when viability is assessed.

Direct connection deployment

The direct interconnect may be viable with standard ZR plugs, or it may need a more capable vendor proprietary plug configurations.

Optical network with ROADMs and Amplifiers In the following diagram, the devices with packet functions are a significant distance apart requiring the use of more sophisticated optical/photonic capabilities including amplifiers and ROADMs. The network may be more complex than shown and may involve optical protection etc. (depending upon network strategy). The packet sites may also have optical equipments present so ROADM1 may be in the same site as Router A etc.

Optical network with ROADMs and amplifiers deployment

Optical network with regenerators In the following diagram, the devices with packet functions are a great distance apart requiring the use of optical regenerators. It is possible several regenerators may be required. As for the previous case, there may also be optical protection etc.

Optical network with ROADMs and Regenerators deployment

Control Architecture Options This section describes alternative control architecture aiming at disaggregated could-based realization solutions.

Acknowledgments Thanks to Sergio Belotti from Nokia (sergio.belotti@nokia.com) for his contribution to this document.

Contributors Huawei Technologies

italo.busi@huawei.com

Ciena

ialderdi@ciena.com

Ciena

mgibbon@ciena.com

ZTE

wang.qilei@zte.com.cn