Optimizations of Label Switched Path State Synchronization Procedures for a Stateful PCE

The Path Computation Element Communication Protocol (PCEP) provides mechanisms for Path Computation Elements (PCEs) to perform path computations in response to Path Computation Clients (PCCs) requests. describes a set of extensions to PCEP to provide stateful control. A stateful PCE has access to not only the information carried by the network's Interior Gateway Protocol (IGP), but also the set of active paths and their reserved resources for its computations. The additional state allows the PCE to compute constrained paths while considering individual LSPs and their interactions. This requires a reliable state synchronization mechanism between the PCE and the network, PCE and PCC, and between cooperating PCEs. describes the basic mechanism for state synchronization. This draft specifies following optimizations for state synchronization and the corresponding PCEP procedures and extensions: State Synchronization Avoidance: To skip state synchronization if the state has survived and not changed during session restart. (See .) Incremental State Synchronization: To do incremental (delta) state synchronization when possible. (See .) PCE-triggered Initial Synchronization: To let PCE control the timing of the initial state synchronization. (See .) PCE-triggered Re-synchronization: To let PCE re-synchronize the state for sanity check. (See .)

This document uses the following terms defined in : PCC, PCE, PCEP Peer. This document uses the following terms defined in : Delegation, Redelegation Timeout Interval, LSP State Report, LSP Update Request, LSP State Database. Within this document, when describing PCE-PCE communications, the requesting PCE fills the role of a PCC. This provides a saving in documentation without loss of function.

The purpose of state synchronization is to provide a checkpoint-in-time state replica of a PCC's LSP state in a stateful PCE. State synchronization is performed immediately after the initialization phase (). describes the basic mechanism for state synchronization. State synchronization is not always necessary following a PCEP session restart. If the state of both PCEP peers did not change, the synchronization phase may be skipped. This can result in significant savings in both control-plane data exchanges and the time it takes for the stateful PCE to become fully operational.

State synchronization MAY be skipped following a PCEP session restart if the state of both PCEP peers did not change during the period prior to session re-initialization. To be able to make this determination, state must be exchanged and maintained by both PCE and PCC during normal operation. This is accomplished by keeping track of the changes to the LSP state database, using a version tracking field called the LSP State Database Version Number. The LSP State Database Version Number, carried in LSP-DB-VERSION TLV (see ), is owned by a PCC and it MUST be incremented by 1 for each successive change in the PCC's LSP state database. The LSP State Database Version Number MUST start at 1 and may wrap around. Values 0 and 0xFFFFFFFFFFFFFFFF are reserved. If either of the two values are used during LSP state (re)-synchronization, the PCE speaker receiving this node should send back a PCErr with Error-type 20 Error-value TBD (suggested value - 6) 'Received an invalid LSP DB Version Number', and close the PCEP session. Operations that trigger a change to the local LSP state database include a change in the LSP operational state, delegation of an LSP, removal or setup of an LSP or change in any of the LSP attributes that would trigger a report to the PCE. If state synchronization avoidance is enabled, a PCC MUST increment its LSP State Database Version Number when the 'Redelegation Timeout Interval' timer expires (see ) for the use of the Redelegation Timeout Interval). State synchronization avoidance is advertised on a PCEP session during session startup using the INCLUDE-DB-VERSION (S) bit in the capabilities TLV (see ). The peer may move in the network, either physically or logically, which may cause its connectivity details and transport-level identity (such as IP address) to change. To ensure that a PCEP peer can recognize a previously connected peer even in face of such mobility, each PCEP peer includes the SPEAKER-ENTITY-ID TLV described in in the OPEN message. If both PCEP speakers set the S flag in the OPEN object's STATEFUL-PCE-CAPABILITY TLV to 1, the PCC MUST include the LSP-DB-VERSION TLV in each LSP object of the PCRpt message. If the LSP-DB-VERSION TLV is missing in a PCRpt message, the PCE will generate an error with Error-Type 6 (mandatory object missing) and Error-Value TBD (suggested value - 12) 'LSP-DB-VERSION TLV missing' and close the session. If state synchronization avoidance has not been enabled on a PCEP session, the PCC SHOULD NOT include the LSP-DB-VERSION TLV in the LSP Object and the PCE SHOULD ignore it were it to receive one. If a PCE's LSP state database survived the restart of a PCEP session, the PCE will include the LSP-DB-VERSION TLV in its OPEN object, and the TLV will contain the last LSP State Database Version Number received on an LSP State Report from the PCC in the previous PCEP session. If a PCC's LSP State Database survived the restart of a PCEP session, the PCC will include the LSP-DB-VERSION TLV in its OPEN object and the TLV will contain the latest LSP State Database Version Number. If a PCEP speaker's LSP state database did not survive the restart of a PCEP session, the PCEP speaker MUST NOT include the LSP-DB-VERSION TLV in the OPEN object. If both PCEP speakers include the LSP-DB-VERSION TLV in the OPEN Object and the TLV values match, the PCC MAY skip state synchronization. Otherwise, the PCC MUST perform full state synchronization (see ) or incremental state synchronization (see ) to the stateful PCE. If the PCC attempts to skip state synchronization, by setting the SYNC Flag to 0 and PLSP-ID to a non-zero value on the first LSP State Report from the PCC as per , the PCE MUST send back a PCErr with Error-Type 20 Error-Value TBD (suggested value - 2) 'LSP Database version mismatch', and close the PCEP session. If state synchronization is required, then prior to completing the initialization phase, the PCE MUST mark any LSPs in the LSP database that were previously reported by the PCC as stale. When the PCC reports an LSP during state synchronization, if the LSP already exists in the LSP database, the PCE MUST update the LSP database and clear the stale marker from the LSP. When it has finished state synchronization, the PCC MUST immediately send an end of synchronization marker. The end of synchronization marker is a Path Computation State Report (PCRpt) message with an LSP object containing a PLSP-ID of 0 and with the SYNC flag set to 0 (). The LSP-DB-VERSION TLV MUST be included in this PCRpt message. On receiving this state report, the PCE MUST purge any LSPs from the LSP database that are still marked as stale. Note that a PCE/PCC MAY force state synchronization by not including the LSP-DB-VERSION TLV in its OPEN object. Since a PCE does not make changes to the LSP State Database Version Number, a PCC should never encounter this TLV in a message from the PCE (other than the OPEN message). A PCC SHOULD ignore the LSP-DB-VERSION TLV, were it to receive one from a PCE. shows an example sequence where the state synchronization is skipped.

A new INCLUDE-DB-VERSION (S) bit is added in the stateful capabilities TLV (see for details).

The LSP State Database Version Number (LSP-DB-VERSION) TLV is an optional TLV that MAY be included in the OPEN object and the LSP object. The format of the LSP-DB-VERSION TLV is shown in the following figure:

The type of the TLV is [TBD] and it has a fixed length of 8 octets. The value contains a 64-bit unsigned integer, representing the LSP State DB Version Number.

The Speaker Entity Identifier TLV (SPEAKER-ENTITY-ID) is an optional TLV that MAY be included in the OPEN Object when a PCEP speaker wishes to determine if state synchronization can be skipped when a PCEP session is restarted. It contains a unique identifier for the node that does not change during the lifetime of the PCEP speaker. It identifies the PCEP speaker to its peers even if the speaker's IP address is changed. In case of a remote peer IP address change, a PCEP speaker would learn the speaker entity identifier on receiving the open message but it MAY have already sent its open message without realizing that it is a known PCEP peer. In such a case, either a full synchronization is done or PCEP session is terminated. This may be a local policy decision. The new IP address is associated with the speaker entity identifier for future either way. In the latter case when PCEP session is re-established, it would be correctly associated with speaker entity identifier and not be considered as an unknown peer. The format of the SPEAKER-ENTITY-ID TLV is shown in the following figure:

The type of the TLV is [TBD] and it has a variable length, which MUST be greater than 0. The Value is padded to 4-octet alignment. The padding is not included in the Length field. The value contains the entity identifier of the speaker transmitting this TLV. This identifier is required to be unique within its scope of visibility, which is usually limited to a single domain. It MAY be configured by the operator. Alternatively, it can be derived automatically from a suitably-stable unique identifier, such as a MAC address, serial number, Traffic Engineering Router ID, or similar. In the case of inter-domain connections, the speaker SHOULD prefix its usual identifier with the domain identifier of its residence, such as Autonomous System number, IGP area identifier, or similar. The relationship between this identifier and entities in the Traffic Engineering database is intentionally left undefined. From a manageability point of view, a PCE or PCC implementation SHOULD allow the operator to configure this Speaker Entity Identifier.

describes the LSP state synchronization mechanism between PCCs and stateful PCEs. During the state synchronization, a PCC sends the information of all its LSPs (i.e., the full LSP-DB) to the stateful PCE. In order to reduce the state synchronization overhead when there is a small number of LSP state change in the network between PCEP session restart, this section defines a mechanism for incremental (Delta) LSP Database (LSP-DB) synchronization.

According to , if a PCE restarts and its LSP-DB survived, PCCs with mismatched LSP State Database Version Number will send all their LSPs information (full LSP-DB) to the stateful PCE, even if only a small number of LSPs underwent state change. It can take a long time and consume large communication channel bandwidth. shows an example of LSP state synchronization.

Assuming there are 320 LSPs in the network, with each PCC having 80 LSPs. During the time when the PCEP session is down, 20 LSPs of each PCC (i.e., 80 LSPs in total), are changed. Hence when PCEP session restarts, the stateful PCE needs to synchronize 320 LSPs with all PCCs. But actually, 240 LSPs stay the same. If performing full LSP state synchronization, it can take a long time to carry out the synchronization of all LSPs. It is especially true when only a low bandwidth communication channel is available (e.g., in-band control channel for optical transport networks) and there is a substantial number of LSPs in the network. Another disadvantage of full LSP synchronization is that it is a waste of communication bandwidth to perform full LSP synchronization given the fact that the number of LSP changes can be small during the time when PCEP session is down. An incremental (Delta) LSP Database (LSP-DB) state synchronization is described in this section, where only the LSPs underwent state change are synchronized between the session restart. This may include new/modified/deleted LSPs.

describes state synchronization and describes state synchronization avoidance by using LSP-DB-VERSION TLV in its OPEN object. This section extends this idea to only synchronize the delta (changes) in case of version mismatch. If both PCEP speakers include the LSP-DB-VERSION TLV in the OPEN object and the LSP-DB-VERSION TLV values match, the PCC MAY skip state synchronization. Otherwise, the PCC MUST perform state synchronization. Incremental State synchronization capability is advertised on a PCEP session during session startup using the DELTA-LSP-SYNC-CAPABILITY (D) bit in the capabilities TLV (see ). Instead of dumping full LSP-DB to the stateful PCE again, the PCC synchronizes the delta (changes) as described in when D flag and S flag is set to 1 by both PCC and PCE. Other combinations of D and S flags setting by PCC and PCE result in full LSP-DB synchronization procedure as described in . The PCC MAY force a full LSP DB synchronization by setting the D flag to zero in the OPEN message.

| (Expect Delta sync) (Do sync)|<--------` | (DONOT Purge LSP (Delta) | | State) | | (Delta Sync starts) |--PCRpt,DBv=46,SYNC=1-->| | . | | . | | . | | . | |--PCRpt,DBv=46,SYNC=0-->| (Sync done, | | PLSP-ID=0) | | |--PCRpt,DBv=47,SYNC=0-->| (Regular | | LSP State Report) |--PCRpt,DBv=48,SYNC=0-->| (Regular | | LSP State Report) |--PCRpt,DBv=49,SYNC=0-->| | | ]]> As per , the LSP State Database Version Number is incremented each time a change is made to the PCC's local LSP State Database. Each LSP is associated with the DB version at the time of its state change. This is needed to determine which LSP and what information needs to be synchronized in incremental state synchronization. It is not necessary for a PCC to store a complete history of LSP Database change, but rather remember the LSP state changes (including LSP modification, setup and deletion) that happened between the PCEP session(s) restart in order to carry out incremental state synchronization. After the synchronization procedure finishes, the PCC can dump this history information. In the example shown in , the PCC needs to store the LSP state changes that happened between DB Version 43 to 46 and synchronizes these changes only when performing incremental LSP state update. So a PCC needs to remember at least the LSP state changes that happened after an existing PCEP session with a stateful PCE goes down to have any chance of doing incremental synchronisation when the session is re-established. If a PCC finds out it does not have sufficient information to complete incremental synchronisation after advertising incremental LSP state synchronization capability, it MUST send a PCErr with Error-Type 20 and Error-Value 5 'A PCC indicates to a PCE that it can not complete the state synchronization' (defined in ) and terminate the session. The PCC SHOULD re-establish the session with the D bit set to 0 in the OPEN message. The other procedures and error checks remain unchanged from the full state synchronization ().

In networks such as optical transport networks, the control channel between network nodes can be realized through in-band overhead thus has limited bandwidth. With a stateful PCE connected to the network via one network node, it is desirable to control the timing of PCC state synchronization so as not to overload the low communication channel available in the network during the initial synchronization (be it incremental or full) when the session restarts , when there is comparatively large amount of control information needing to be synchronized between the stateful PCE and the network. The method proposed, i.e., allowing PCE to trigger the state synchronization, is similar to the function proposed in but is used in different scenarios and for different purposes.

Support of PCE-triggered initial state synchronization is advertised during session startup using the TRIGGERED-INITIAL-SYNC (F) bit in the STATEFUL-PCE-CAPABILITY TLV (see ). In order to allow a stateful PCE to control the LSP-DB synchronization after establishing a PCEP session, both PCEP speakers MUST set F bit to 1 in the OPEN message. If the TRIGGERED-INITIAL-SYNC capability is not advertised by a PCE and the PCC receives a PCUpd with the SYNC flag set to 1, it MUST send a PCErr with the SRP-ID-number of the PCUpd, Error-Type 20 and Error-Value TBD (suggested value - 4) 'Attempt to trigger synchronization when the TRIGGERED-SYNC capability has not been advertised' (see ). If the LSP-DB Version is mis-matched, it can send a PCUpd message with PLSP-ID = 0 and SYNC = 1 in order to trigger the LSP-DB synchronization process. In this way, the PCE can control the sequence of LSP synchronization among all the PCCs that are re-establishing PCEP sessions with it. When the capability of PCE control is enabled, only after a PCC receives this message, it will start sending information to the PCE. The PCC SHOULD NOT send PCRpt messages to the stateful PCE before it triggers the State Synchronization. This PCE-triggering capability can be applied to both full and incremental state synchronization. If applied to the later, the PCCs only send information that PCE does not possess, which is inferred from the LSP-DB version information exchanged in the OPEN message (see for detailed procedure). Once the initial state synchronization is triggered by the PCE, the procedures and error checks remain unchanged from the full state synchronization ().

The accuracy of the computations performed by the PCE is tied to the accuracy of the view the PCE has on the state of the LSPs. Therefore, it can be beneficial to be able to re-synchronize this state even after the session has been established. The PCE may use this approach to continuously sanity check its state against the network, or to recover from error conditions without having to tear down sessions.

Support of PCE-triggered state synchronization is advertised by both PCEP speakers during session startup using the TRIGGERED-RESYNC (T) bit in the STATEFUL-PCE-CAPABILITY TLV (see ). The PCE can choose to re-synchronize its entire LSP database or a single LSP. To trigger re-synchronization for an LSP, the PCE MUST first mark the LSP as stale and then send a Path Computation State Update (PCUpd) for it, with the SYNC flag in the LSP object set to 1. The PCE SHOULD NOT include any parameter updates for the LSP, and the PCC SHOULD ignore such updates if the SYNC flag is set. The PCC MUST respond with a PCRpt message with the LSP state, SYNC Flag set to 0 and MUST include the SRP-ID-number of the PCUpd message that triggered the resynchronization. The PCE can also trigger re-synchronization of the entire LSP database. The PCE MUST first mark all LSPs in the LSP database that were previously reported by the PCC as stale and then send a PCUpd with an LSP object containing a PLSP-ID of 0 and with the SYNC flag set to 1. This PCUpd message is the trigger for the PCC to enter the synchronization phase as described in and start sending PCRpt messages. After the receipt of the end-of-synchronization marker, the PCE will purge LSPs which were not refreshed. The SRP-ID-number of the PCUpd that triggered the re-synchronization SHOULD be included in each of the PCRpt messages. If the TRIGGERED-RESYNC capability is not advertised by a PCE and the PCC receives a PCUpd with the SYNC flag set to 1, it MUST send a PCErr with the SRP-ID-number of the PCUpd, Error-Type 20 and Error-Value TBD (suggested value - 4) 'Attempt to trigger synchronization when the TRIGGERED-SYNC capability has not been advertised' (see ). Once the state re-synchronization is triggered by the PCE, the procedures and error checks remain unchanged from the full state synchronization (). This would also include PCE triggering multiple state re-synchronization requests while synchronization is in progress.

Support for each of the optimizations described in this document requires advertising the corresponding capabilities during session establishment time. New flags are defined for the STATEFUL-PCE-CAPABILITY TLV defined in . Its format is shown in the following figure:

The value comprises a single field - Flags (32 bits): defined in . if set to 1 by both PCEP Speakers, the PCC will include the LSP-DB-VERSION TLV in each LSP Object. See for details. defined in . if set to 1 by both PCEP Speakers, the PCE can trigger re-synchronization of LSPs at any point in the life of the session. See for details. if set to 1 by a PCEP speaker, it indicates that the PCEP speaker allows incremental (delta) state synchronization. See for details. if set to 1 by both PCEP Speakers, the PCE SHOULD trigger initial (first) state synchronization. See for details.

This document requests IANA actions to allocate code points for the protocol elements defined in this document.

IANA is requested to make the following allocation in the "PCEP-ERROR Object Error Types and Values" registry.

IANA is requested to make the following allocation in the "PCEP TLV Type Indicators" registry. Value Meaning Reference TBD(suggested value 23) LSP-DB-VERSION This document TBD(suggested value 24) SPEAKER-ENTITY-ID This document

The STATEFUL-PCE-CAPABILITY TLV is defined in and a registry is requested to be created to manage the flags in the TLV. IANA is requested to make the following allocation in the aforementioned registry. Bit Description Reference TBD(suggested value 26) TRIGGERED-INITIAL-SYNC This document TBD(suggested value 27) DELTA-LSP-SYNC-CAPABILITY This document TBD(suggested value 28) TRIGGERED-RESYNC This document TBD(suggested value 30) INCLUDE-DB-VERSION This document

All manageability requirements and considerations listed in and apply to PCEP protocol extensions defined in this document. In addition, requirements and considerations listed in this section apply.

A PCE or PCC implementation MUST allow configuring the state synchronization optimization capabilities as described in this document. The implementation SHOULD also allow the operator to configure the Speaker Entity Identifier ( ).

An implementation SHOULD allow the operator to view the stateful capabilities advertised by each peer, and the current synchronization status with each peer. To serve this purpose, the PCEP MIB module can be extended to include advertised stateful capabilities, and synchronization status.

Mechanisms defined in this document do not imply any new liveness detection and monitoring requirements in addition to those already listed in .

Mechanisms defined in this document do not imply any new operation verification requirements in addition to those already listed in and .

Mechanisms defined in this document do not imply any new requirements on other protocols.

Mechanisms defined in this document do not have any impact on network operations in addition to those already listed in and .

The security considerations listed in apply to this document as well. However, because the protocol modifications outlined in this document allow the PCE to control state (re)-synchronization timing and sequence, it also introduces a new attack vector: an attacker may flood the PCC with triggered re-synchronization request at a rate which exceeds the PCC's ability to process them, either by spoofing messages or by compromising the PCE itself. The PCC is free to drop any trigger re-synchronization request without additional processing.

We would like to thank Young Lee, Jonathan Hardwick, Sergio Belotti and Cyril Margaria for their comments and discussions.

Gang Xie Huawei Technologies F3-5-B R&D Center, Huawei Industrial Base, Bantian, Longgang District Shenzhen, Guangdong, 518129 P.R. China Email: xiegang09@huawei.com