Huawei Technologies

Divyashree Techno Park, Whitefield Bangalore Karnataka 560066 India dhruv.ietf@gmail.com

Huawei Technologies

5340 Legacy Drive, Building 3 Plano TX 75023 USA leeyoung@huawei.com

Ericsson

Torshamnsgatan,48 Stockholm Sweden daniele.ceccarelli@ericsson.com

Routing PCE Working Group Abstraction and Control of TE Networks (ACTN) refers to the set of virtual network (VN) operations needed to orchestrate, control and manage large-scale multi-domain TE networks so as to facilitate network programmability, automation, efficient resource sharing, and end-to-end virtual service aware connectivity and network function virtualization services. The Path Computation Element (PCE) is a component, application, or network node that is capable of computing a network path or route based on a network graph and applying computational constraints. The PCE serves requests from Path Computation Clients (PCCs) that communicate with it over a local API or using the Path Computation Element Communication Protocol (PCEP). This document examines the applicability of PCE to the ACTN framework.

The Path Computation Element Communication Protocol (PCEP) provides mechanisms for Path Computation Clients (PCCs) to request a Path Computation Element (PCE) to perform path computations. The ability to compute shortest constrained TE LSPs in Multiprotocol Label Switching (MPLS) and Generalized MPLS (GMPLS) networks across multiple domains has been identified as a key motivation for PCE development. A stateful PCE is capable of considering, for the purposes of path computation, not only the network state in terms of links and nodes (referred to as the Traffic Engineering Database or TED) but also the status of active services (previously computed paths), and currently reserved resources, stored in the Label Switched Paths Database (LSP-DB). describes general considerations for a stateful PCE deployment and examines its applicability and benefits, as well as its challenges and limitations through a number of use cases. describes a set of extensions to PCEP to provide stateful control. A stateful PCE has access to not only the information carried by the network's Interior Gateway Protocol (IGP), but also the set of active paths and their reserved resources for its computations. The additional state allows the PCE to compute constrained paths while considering individual LSPs and their interactions. describes the setup, maintenance and teardown of PCE-initiated LSPs under the stateful PCE model. also describes the active stateful PCE. The active PCE functionality allows a PCE to reroute an existing LSP or make changes to the attributes of an existing LSP, or a PCC to delegate control of specific LSPs to a new PCE.

Software-Defined Networking (SDN) refers to a separation between the control elements and the forwarding components so that software running in a centralized system called a controller, can act to program the devices in the network to behave in specific ways. A required element in an SDN architecture is a component that plans how the network resources will be used and how the devices will be programmed. It is possible to view this component as performing specific computations to place flows within the network given knowledge of the availability of network resources, how other forwarding devices are programmed, and the way that other flows are routed. It is concluded in , that this is the same function that a PCE might offer in a network operated using a dynamic control plane. This is the function and purpose of a PCE, and the way that a PCE integrates into a wider network control system including SDN is presented in Application-Based Network Operation (ABNO) .

Computing paths across large multi-domain environments requires special computational components and cooperation between entities in different domains capable of complex path computation. The PCE provides an architecture and a set of functional components to address this problem space. A PCE may be used to compute end-to-end paths across multi-domain environments using a per-domain path computation technique . The Backward Recursive PCE based path computation (BRPC) mechanism defines a PCE-based path computation procedure to compute inter-domain constrained MPLS and GMPLS TE networks. However, per-domain technique assumes that the sequence of domains to be crossed from source to destination is known, either fixed by the network operator or obtained by other means. BRPC can work best with a known domain sequence, and it will also work nicely with a small set of interconnected domains. However, it doesn't work well for is a large set of interconnected domains. describes a Hierarchical PCE (H-PCE) architecture which can be used for computing end-to-end paths for inter-domain MPLS Traffic Engineering (TE) and GMPLS Label Switched Paths (LSPs) when the domain sequence is not known. Within the Hierarchical PCE (H-PCE) architecture, the Parent PCE (P-PCE) is used to compute a multi-domain path based on the domain connectivity information. A Child PCE (C-PCE) may be responsible for a single domain or multiple domains, it is used to compute the intra-domain path based on its domain topology information. state the considerations for stateful PCEs in hierarchical PCE architecture. In particular, the behavior changes and additions to the existing stateful PCE mechanisms (including PCE- initiated LSP setup and active PCE usage) in the context of networks using the H-PCE architecture. describes a framework for applying the PCE-based architecture to inter-layer to (G)MPLS TE. It provides suggestions for the deployment of PCE in support of multi-layer networks. It also describes the relationship between PCE and a functional component in charge of the control and management of the Virtual Network Topology (VNT) , called the VNT Manager (VNTM).

introduces the architecture for PCE as a central controller (PCECC), it further examines the motivations and applicability for PCEP as a southbound interface, and introduces the implications for the protocol. Section 2.1.3 of describe a hierarchy of PCE-based controller as per the Hierarchy of PCE framework defined in .

describes the high-level ACTN requirements and the architecture model for ACTN including the entities Customer Network Controller (CNC), Multi-domain Service Coordinator (MDSC), and Provisioning Network Controller (PNC) and their interfaces. The ACTN reference architecture is shown in which is reproduced here from for convenience. remains the definitive reference for the ACTN architecture. As depicted in , the ACTN architecture identifies a three-tier hierarchy.

There are two interfaces with respect to the MDSC: one north of the MDSC (the CNC-MDSC Interface : CMI), and one south (the MDSC-PNC Interface : MPI). A hierarchy of MDSCs is possible with a recursive MPI interface. provides an information model for ACTN interfaces.

Abstraction and Control of TE Networks (ACTN) refers to the set of virtual network (VN) operations needed to orchestrate, control and manage large-scale multi-domain TE networks so as to facilitate network programmability, automation, efficient resource sharing, and end-to-end virtual service aware connectivity and network function virtualization services. The Path Computation Element (PCE) is a component, application, or network node that is capable of computing a network path or route based on a network graph and applying computational constraints. The PCE serves requests from Path Computation Clients (PCCs) that communicate with it over a local API or using the Path Computation Element Communication Protocol (PCEP). This document examines the PCE and ACTN architecture and describes how PCE architecture is applicable to ACTN. It also lists the PCEP extensions that are needed to use PCEP as an ACTN interface. This document also identifies any gaps in PCEP, that exist at the time of publication of this document. Further, ACTN, stateful H-PCE, and PCECC are based on the same basic hierarchy framework and thus compatible with each other.

The ACTN architecture is based on hierarchy and recursiveness of controllers. It defines three types of controllers (depending on the functionalities they implement). The main functionalities are - Multi-domain coordination Abstraction Customer mapping/translation Virtual service coordination Section 3 of describes these functions. It should be noted that this document lists all possible ways in which PCE could be used for each of the above functions, but all functions are not required to be implemented via PCE. Similarly, this document presents the ways in which PCEP could be used as the communications medium between functional components. Operators may choose to use the PCEP for multi-domain coordination via stateful H-PCE, but alternatively use RESTCONF or BGP-LS to get access to the topology and support abstraction function.

With the definition of domain being "everything that is under the control of the single logical controller", as per , it is needed to have a control entity that oversees the specific aspects of the different domains and to build a single abstracted end-to-end network topology in order to coordinate end-to-end path computation and path/service provisioning. The MDSC in ACTN framework realizes this function by coordinating the per-domain PNCs in a hierarchy of controllers. It also needs to detach from the underlying network technology and express customer concerns by business needs. and describe a hierarchy of PCEs with the Parent PCE coordinating multi-domain path computation function between Child PCEs. It is easy to see how these principles align, and thus how the stateful H-PCE architecture can be used to realize ACTN. The per domain stitched LSP in the Hierarchical stateful PCE architecture, described in Section 3.3.1 of is well suited for multi-domain coordination function. This includes domain sequence selection; End-to-End (E2E) path computation; Controller (PCE) initiated path setup and reporting. This is also applicable to multi-layer coordination in case of IP+optical networks. describes the procedures to allow a stateful communication between PCEs for various use-cases. The procedures and extensions are also applicable to Child and Parent PCE communication and thus useful for ACTN as well.

To realize ACTN, an abstracted view of the underlying network resources needs to be built. This includes global network-wide abstracted topology based on the underlying network resources of each domain. This also includes abstract topology created as per the customer service connectivity requests and represented as a VN slice allocated to each customer. In order to compute and provide optimal paths, PCEs require an accurate and timely Traffic Engineering Database (TED). Traditionally this TED has been obtained from a link state (LS) routing protocol supporting traffic engineering extensions. PCE may construct its TED by participating in the IGP ( and for MPLS-TE; and for GMPLS). An alternative is offered by BGP-LS . In case of H-PCE , the Parent PCE needs to build the domain topology map of the child domains and their interconnectivity. and suggest that BGP-LS could be used as a "northbound" TE advertisement from the Child PCE to the Parent PCE. proposes another approach for learning and maintaining the Link-State and TE information as an alternative to IGPs and BGP flooding, using PCEP itself. The Child PCE can use this mechanism to transport Link-State and TE information from Child PCE to a Parent PCE using PCEP. In ACTN, there is a need to control the level of abstraction based on the deployment scenario and business relationship between the controllers. The mechanism used to disseminate information from PNC (Child PCE) to MDSC (Parent PCE) should support abstraction. describes a few alternative approaches of abstraction. The resulting abstracted topology can be encoded using the PCEP-LS mechanisms and its optical network extension . PCEP-LS is an attractive option when the operator would wish to have a single control plane protocol (PCEP) to achieve ACTN functions. discusses two ways to build abstract topology from an MDSC standpoint with interaction with PNCs. The primary method is called automatic generation of abstract topology by configuration. With this method, automatic generation is based on the abstraction/summarization of the whole domain by the PNC and its advertisement on the MPI. The secondary method is called on-demand generation of supplementary topology via Path Compute Request/Reply. This method may be needed to obtain further complementary information such as potential connectivity from Child PCEs in order to facilitate an end-to-end path provisioning. PCEP is well suited to support both methods.

In ACTN, there is a need to map customer virtual network (VN) requirements into a network provisioning request to the PNC. That is, the customer requests/commands are mapped by the MDSC into network provisioning requests that can be sent to the PNC. Specifically, the MDSC provides mapping and translation of a customer's service request into a set of parameters that are specific to a network type and technology such that network configuration process is made possible. describes the setup, maintenance and teardown of PCE-initiated LSPs under the stateful PCE model, without the need for local configuration on the PCC, thus allowing for a dynamic network that is centrally controlled and deployed. To instantiate or delete an LSP, the PCE sends the Path Computation LSP Initiate Request (PCInitiate) message to the PCC. As described in , for inter-domain LSP in Hierarchical PCE architecture, the initiation operations can be carried out at the Parent PCE. In which case, after Parent PCE finishes the E2E path computation, it can send the PCInitiate message to the Child PCE, the Child PCE further propagates the initiate request to the Label Switching Router (LSR). The customer request is received by the MDSC (Parent PCE) and based on the business logic, global abstracted topology, network conditions and local policy, the MDSC (Parent PCE) translates this into per domain LSP initiation request that a PNC (Child PCE) can understand and act on. This can be done via the PCInitiate message. PCEP extensions for associating opaque policy between PCEP peer can be used.

Virtual service coordination function in ACTN incorporates customer service-related information into the virtual network service operations in order to seamlessly operate virtual networks while meeting customer's service requirements. describes the need for associating a set of LSPs with a VN "construct" to facilitate VN operations in PCE architecture. This association allows the PCEs to identify which LSPs belong to a certain VN. This association based on VN is useful for various optimizations at the VN level which can be applied to all the LSPs that are part of the VN slice. During path computation, the impact of a path for an LSP is compared against the paths of other LSPs in the VN. This is to make sure that the overall optimization and SLA of the VN rather than of a single LSP. Similarly, during re-optimization, advanced path computation algorithm and optimization technique can be considered for all the LSPs belonging to a VN/customer and optimize them all together.

As per , to allow virtualization and multi-domain coordination, the network has to provide open, programmable interfaces, in which customer applications can create, replace and modify virtual network resources and services in an interactive, flexible and dynamic fashion while having no impact on other customers. The two ACTN interfaces are - The CNC-MDSC Interface (CMI) is an interface between a Customer Network Controller and a Multi-Domain Service Coordinator. It requests the creation of the network resources, topology or services for the applications. The MDSC may also report potential network topology availability if queried for current capability from the Customer Network Controller. The MDSC-PNC Interface (MPI) is an interface between a Multi-Domain Service Coordinator and a Provisioning Network Controller. It communicates the creation request, if required, of new connectivity of bandwidth changes in the physical network, via the PNC. In multi-domain environments, the MDSC needs to establish multiple MPIs, one for each PNC, as there are multiple PNCs responsible for its domain control. In the case of hierarchy MDSCs, the MPI is applied recursively. From an abstraction point of view, the top level MDSC which interfaces the CNC operates on a higher level of abstraction (i.e., less granular level) than the lower level MSDCs. PCEP is especially suitable on the MPI as it meets the requirement and the functions as set out in the ACTN framework . Its recursive nature is well suited via the multi-level hierarchy of PCE. PCEP can also be applied to the CMI as the CNC can be a path computation client while the MDSC can be a path computation server. describes how PCE and PCEP could help realize ACTN on the MPI.

As per the example in , there are 4 domains, each with its own PNC and an MDSC on top. The PNC and MDSC need PCE as a important function. The PNC (or Child PCE) already uses PCEP to communicate to the network device. It can utilize the PCEP as the MPI to communicate between controllers too.

Parent PCE PNC -> Child PCE MPI -> PCEP ]]>

Building Domain Topology at MDSC: PNC (or Child PCE) needs to have the TED to compute path in its domain. As described in , it can learn the topology via IGP or BGP-LS. PCEP-LS is also a proposed mechanism to carry link state and traffic engineering information within PCEP. A mechanism to carry abstracted topology while hiding technology specific information between PNC and MDSC is described in . At the end of this step the MDSC (or Parent PCE) has the abstracted topology from each of its PNC (or Child PCE). This could be as simple as a domain topology map as described in or it can have full topology information of all domains. The latter is not scalable and thus an abstracted topology of each domain interconnected by inter-domain links is the most common case. Topology Change: When the PNC learns of any topology change, the PNC needs to decide if the change needs to be notified to the MDSC. This is dependent on the level of abstraction between the MDSC and the PNC. VN Instantiate: When an MDSC is requested to instantiate a VN, the minimal information that is required would be a VN identifier and a set of end points. Various path computation, setup constraints and objective functions may also be provided. In PCE terms, a VN Instantiate can be considered as a set of paths belonging to the same VN. As described in and the VN association can help in identifying the set of paths that belong to a VN. The rest of the information like the endpoints, constraints and objective function (OF) is already defined in PCEP in terms of a single path. Path Computation: As per the example in , the VN instantiate requires two end to end paths between (A in Domain 1 to B in Domain 3) and (A in Domain 1 to C in Domain 4). The MDSC (or Parent PCE) triggers the end to end path computation for these two paths. MDSC can do path computation based on the abstracted domain topology that it already has or it may use the H-PCE procedures () using the PCReq and PCRep messages to get the end to end path with the help of the Child PCEs (PNC). Either way, the resultant E2E paths may be broken into per-domain paths. A-B: (A-B13,B13-B31,B31-B) A-C: (A-B13,B13-B31,B34-B43,B43-C) Per Domain Path Instantiation: Based on the above path computation, MDSC can issue the path instantiation request to each PNC via PCInitiate message (see and ). A suitable stitching mechanism would be used to stitch these per domain LSPs. One such mechanism is described in , where PCEP is extended to support stitching in stateful H-PCE context. Per Domain Path Report: Each PNC should report the status of the per-domain LSP to the MDSC via PCRpt message, as per the Hierarchy of stateful PCE (). The status of the end to end LSP (A-B and A-C) is made up when all the per domain LSP are reported up by the PNCs. Delegation: It is suggested that the per domain LSPs are delegated to respective PNC, so that they can control the path and attributes based on each domain network conditions. State Synchronization: The state needs to be synchronized between the Parent PCE and Child PCE. The mechanism described in can be used. VN Modify: MDSC is requested to modify a VN, for example the bandwidth for VN is increased. This may trigger path computation at MDSC as described in the previous step and can trigger an update to existing per-intra-domain path (via PCUpd message) or creation (or deletion) of a per-domain path (via PCInitiate message). As described in , this should be done in make-before-break fashion. VN Delete: MDSC is requested to delete a VN, in this case, based on the E2E paths and the resulting per-domain paths need to be removed (via PCInitiate message). VN Update (based on network changes): Any change in the per-domain LSP is reported to the MDSC (via PCRpt message) as per . This may result in changes in the E2E path or VN status. This may also trigger a re-optimization leading to a new per-domain path, update to existing path, or deletion of the path. VN Protection: The VN protection/restoration requirements, need to applied to each E2E path as well as each per domain path. The MDSC needs to play a crucial role in coordinating the right protection/restoration policy across each PNC. The existing protection/restoration mechanism of PCEP can be applied on each path. In case PNC generates an abstract topology to the MDSC, the PCInitiate/PCUpd messages from the MDSC to a PNC will contain a path with abstract nodes and links. PNC would need to take that as an input for path computation to get a path with physical nodes and links. Similarly, a PNC would convert the path received from the device (with physical nodes and links) into abstract path (based on the abstract topology generated before with abstract nodes and links) and reported to the MDSC.

This document makes no requests for IANA action.

Various security considerations for PCEP are described in , , and . Further, this document lists various extensions of PCEP that are applicable, each of them specify various security considerations which continue to apply here. The ACTN framework described in defines key components and interfaces for managed traffic engineered networks. It also lists various security considerations such as request and control of resources, confidentially of the information, and availability of function which should be taken into consideration. As per , securing the request and control of resources, confidentiality of the information, and availability of function should all be critical security considerations when deploying and operating ACTN platforms. From a security and reliability perspective, ACTN may encounter many risks such as malicious attack and rogue elements attempting to connect to various ACTN components (with PCE being one of them). Furthermore, some ACTN components represent a single point of failure and threat vector and must also manage policy conflicts and eavesdropping of communication between different ACTN components. further states that all protocols used to realize the ACTN framework should have rich security features, and customer, application and network data should be stored in encrypted data stores. When PCEP is used as an ACTN interface, the security of PCEP provided by Transport Layer Security (TLS) , as per the recommendations and best current practices in , is used. As per , regarding the MPI, a PKI- based mechanism is suggested, such as building a TLS or HTTPS connection between the MDSC and PNCs, to ensure trust between the physical network layer control components and the MDSC. Which MDSC the PNC exports topology information to, and the level of detail (full or abstracted), should also be authenticated, and specific access restrictions and topology views should be configurable and/or policy based. When PCEP is used in MPI, the security functions as per are used to fulfill these requirements. As per , regarding the CMI, suitable authentication and authorization of each CNC connecting to the MDSC will be required. If PCEP is used in CMI, the security functions as per can be used to support peer authentication, message encryption, and integrity checks.

The authors would like to thank Jonathan Hardwick for the inspiration behind this document. Further thanks to Avantika for her comments with suggested text. Thanks to Adrian Farrel and Daniel King for their substantial reviews.

In the paper , the application of the ACTN architecture is presented to demonstrate the control of a multi-domain flexi-grid optical network, by proposing, adopting and extending - the Hierarchical active stateful PCE architectures and protocols the PCEP protocol to support efficient and incremental link state topological reporting, known as PCEP-LS the per link partitioning of the optical spectrum based on variable-sized allocated frequency slots enabling network sharing and virtualization the use of a model-based interface to dynamically request the instantiation of virtual networks for specific clients / tenants. The design and the implementation of the testbed are reported in order to validate the approach.