Micro-loop avoidance using SPRING

+----+ | R4 | SRGB:1000-2000 +----+ SID:9 / \ 5 / \ 5 / \ SRGB:1000-2000 SID:1 / \ SID:2 SID:3 SID:4 SID:5 +----+ 10 +----+ 10 +----+ 10 +----+ 10 +----+ | S1 |----| R1 |----| S |-------| E |----| D1 | +----+ +----+ +----+ +----+ +----+ \ \ / 10 \ \ 100 / 60 \ SRGB:1000-2000 \ / \ +----+ +----+ +--| R2 |---------| R3 |SID:7 SID:6 +----+ 30 +----+SRGB:1000-2000 / / 10 +----+ | S2 |SID:8 +----+SRGB:1000-2000 The above sample topology is provided with basic SPRING configurations of SRGB and the indices corresponding to each node. Each node has an SRGB 1000-2000 configured on the node. Same SRGB on all nodes is used for simplifying the example and the procedures are equally applicable when there is different SRGB configured on multiple nodes. Each node is provisioned with a MAX_CONVERGENCE_DELAY value that corresponds to its RIB to FIB convergence time. The information for support of the micro-loop prevention feature and the MAX_CONVERGENCE_DELAY value are flooded across the IGP domain (ISIS level/OSPF area). Each node in the IGP domain sets the MAX_CONVERGENCE_DELAY to the maximum of the values received in the domain.

When the S->E link goes down, all the nodes in the network receive the event via IGP database flooding. Each node supporting the micro-loop prevention mechanism specified in this document SHOULD perform the steps below. The PLRs (S and E) perform FRR local repair for destinations affected by the failure of the link. Each computing node identifies the destinations affected by the topology change.In the example above, the destination D1 is affected by S->E link down for nodes S1,R1,R2, and R4. For S2, although the path to D1 changes there is no change in the immediate next-hop and hence its not necessary for S2 to perform any specific actions to prevent micro-loops. For each affected destination, identify the nearest PLR advertising the change. The link-down event is advertised by both S and E. S is the nearest PLR for the nodes S1,R1,R2, and R4. Let the S->E link down event occurs at time T0. Start a timer T1 = max (all MAXIMUM_CONVERGENCE_DELAY) at all non-PLR nodes with affected destinations. Start a timer T2 = 2 * T1 at the PLR. For IP routes, modify the FIB for the affected destinations so that the nearest PLR's node-sid is pushed on the packet's label stack. For MPLS ingress and transit routes, modify the FIB for the affected destinations with a two label stack, the inner label corresponding to the destination and the outer label corresponding to the nearest PLR. In the case of ECMP paths to the nearest PLR, both tunnelled paths are used. S1 has ECMP paths to the destination D1 and both the paths are impacted. Both the paths are modified to carry two label stacks containing the nearest PLR on top and the destination label at the bottom. After the expiry of timer T1 all the non-PLR nodes modify their FIBs to use the shortest path as computed by the IGP, and they no longer push the node-SID of the nearest PLR on the packets. After the expiry of T2, the PLR converges and updates the FIB to represent shortest path. The ingress MPLS routes at various nodes for destination D1 at specified time intervals is mentioned below.

+======+==========+==========+==============+===========+===========+ | Node | Incoming | Before | T0-T1 | T1-T2 | After T2 | | | Label | T0 | | | | +======+==========+==========+==============+===========+===========+ | S1 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | 1003(top), | 1005, Fwd | 1005, Fwd | | | | Fwd to | Fwd to R1 | to R2 | to R2 | | | | R1 | | | | | | +----------+--------------+-----------+-----------+ | | | Push | Push 1005, | | | | | | 1005, | 1003(top), | | | | | | Fwd to | Fwd to R4 | | | | | | R4 | | | | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to R1 | 1003, Fwd | 1003, Fwd | | | | Fwd to | | to R2 | to R2 | | | | R1 | | | | +======+==========+==========+==============+===========+===========+ | S2 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | Fwd to R2 | 1005, Fwd | 1005, Fwd | | | | Fwd to | | to R2 | to R2 | | | | R2 | | | | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to R1 | 1003, Fwd | 1003, Fwd | | | | Fwd to | | to R2 | to R2 | | | | R1 | | | | +======+==========+==========+==============+===========+===========+ | R1 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | Fwd to S | 1005, Fwd | 1005, Fwd | | | | Fwd to S | | to R4 | to R4 | | | +----------+--------------+-----------+-----------+ | | | | | Push | Push | | | | | | 1005, Fwd | 1005, Fwd | | | | | | to S1 | to S1 | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to S | 1003, Fwd | 1003, Fwd | | | | Fwd to S | | to S | to S | +======+==========+==========+==============+===========+===========+ | R2 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | 1003(top), | 1005, Fwd | 1005, Fwd | | | | Fwd to | Fwd to S1 | to R3 | to R3 | | | | S1 | | | | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to S1 | 1003, Fwd | 1003, Fwd | | | | Fwd to | | to S1 | to S1 | | | | S1 | | | | +======+==========+==========+==============+===========+===========+ | R3 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | 1003(top), | 1005, Fwd | 1005, Fwd | | | | Fwd to E | Fwd to E | to E | to E | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to R2 | 1003, Fwd | 1003, Fwd | | | | Fwd to | | to R2 | to R2 | | | | R2 | | | | +======+==========+==========+==============+===========+===========+ | R4 | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | 1003(top), | 1005, Fwd | 1005, Fwd | | | | Fwd to | Fwd to R1 | to S1 | to S1 | | | | R1 | | | | | +----------+----------+--------------+-----------+-----------+ | | 1003 | Push | Push 1003, | Push | Push | | | | 1003, | Fwd to R1 | 1003, Fwd | 1003, Fwd | | | | Fwd to | | to R1 | to R1 | | | | R1 | | | | +======+==========+==========+==============+===========+===========+ | S | 1005 | Push | Push 1005, | Push | Push | | | | 1005, | Fwd to R3 * | 1005, Fwd | 1005, Fwd | | | | Fwd to E | | to R3 * | to R1 | | | +----------+--------------+-----------+-----------+ | | | Push | | | Push | | | | 1005, | | | 1005, Fwd | | | | Fwd to | | | to R3 * | | | | R3 * | | | | | +----------+----------+--------------+-----------+-----------+ | | 1003 | -- | -- | -- | -- | +======+==========+==========+==============+===========+===========+ | E | 1005 | Pop, Fwd | Pop, Fwd to | Pop, Fwd | Pop, Fwd | | | | to D1 | D1 | to D1 | to D1 | +======+==========+==========+==============+===========+===========+ * - Indicates backup path.

When a new-link is added to the network, the PLR needs to update the FIB before it announces the change. First the PLR converges, updates the FIB as per the new-link based topology and then announces the new-link addition to the rest of the network. The other network nodes SHOULD follow the procedure exactly same as described in sec 3.1. They SHOULD update their FIB to tunnel the traffic to the closest node corresponding to the change.After MAX_CONVERGENCE_DELAY the nodes SHOULD update the FIB with the shortest path next-hops.

SRGB:1000-2000 SID:1 SID:2 SID:3 SID:4 SID:5 +----+ 10 +----+ 10 +----+ 10 +----+ 10 +----+ | S1 |----| R1 |----| S |---X---| E |----| D1 | +----+ +----+ +----+ +----+ +----+ \ \ / 10 \ \ 10 / 100 \ SRGB:1000-2000 \ / \ +----+ +----+ +--| R2 |---------| R3 |SID:7 SID:6 +----+ 10 +----+SRGB:1000-2000 / / 10 +----+ | S2 |SID:8 +----+SRGB:1000-2000 In the figure above, when the S->E link is added (or restored back), PLR S processes the event and programs the FIB with new path for the affected destinations. PLR delays flooding the event for MAX_CONVERGENCE_DELAY interval. This step prevents possible local micro-loop between S and R3. Once PLR floods the event, non PLR nodes in the network identify the destinations affected by the database change. This is done by SPF computation and examining the next-hop change. The destination D1 is affected by S->E link up for nodes S1, R1, R2 and R3. For each affected destination, identify the nearest PLR advertising the change. The link-up event is advertised by both S and E. S is the nearest PLR for the nodes S1,R1,R2 and R3. When there are ECMP paths to the destination and a new ECMP path is added, the new ECMP path follows the micro-loop prevention mechanisms and tunnels the traffic towards nearest PLR. Start a timer T3 = max (all MAXIMUM_CONVERGENCE_DELAY) at all non-PLR nodes. For IP routes, update the FIB for the affected destinations so that the nearest PLR's node-sid is pushed on the packet's label stack. For MPLS ingress and transit router update the path with two label stack, the inner label corresponding to the destination and the outer label corresponding to the nearest PLR. This step prevents the possible remote micro-loop between S1 and R2. After the expiry of timer T3 all the non-PLR nodes perform global convergence and update the FIB to represent the shortest path. Other management events like metric change are handled similar to the link-down/link-up cases for metric increase/metric decrease cases respectively.

When a network event is received by a node via the IGP database change notification, a node has to compute the nearest PLR corresponding to that advertisement. The first database change advertisement may be received from any of the PLRs, nearest or farthest.

When a link goes down, IGPs generate a fresh LSP/Router LSA with the affected link removed. The computing node has to identify the missing link by walking over the LSP/LSA and compare the contents with an older version. Once the affected link is identified, the cost to reach both ends of the link should be examined. The nearest PLR is chosen based on the cost to reach the ends.

When a node goes down, it is identified by the neighbouring nodes via link-down event. the neighbouring routers generate a fresh LSP/Router LSA with the affected link removed. The computing node has to identify the missing link by walking over the LSP/LSA and compare the contents with an older version. Once the affected link is identified, the cost to reach both ends of the link should be examined. The nearest PLR is chosen based on the cost to reach the ends. When an advertisement from the farthest node is received before the nearest node, it is possible that the node that went down is chosen as the nearest PLR, as the node that went down might be still lingering in the database. In such cases node protection mechanisms for the deceased node at the previous-hop should prevent traffic loss. The details of such a mechanism is outside the scope of this document.

It is important to categorize the received events as belonging to one network event or multiple network events. The link-down/link-up event is advertised by both ends of the link. The node-down/node-up event is advertised by all the neighbouring nodes.When an event is received, the computing node should analyse the changes in the database advertisements and compare with previous database.The micro-loop prevention procedures SHOULD be started when the first notification is received. The node SHOULD record the event for which micro-loop prevention procedures are being performed. If there are more database changes received during this time, the change should be mapped to the already on-going micro-loop prevention procedures.If the event is same then the micro-loop prevention procedures MUST continue, otherwise the micro-loop prevention procedures SHOULD be aborted. sec 6.2 describes mechanisms to handle the SRLG failures. If the received failure advertisement is part of an SRLG advertised in the IGP TE advertisement, the links on the path sharing same SRLG are identified and the tunnel is built with multiple label stack corresponding to the nearest PLR of each SRLG member. When a failure is received, and the failure does not belong to the same SRLG as the already on-going micro-loop prevention, the micro-loop prevention procedures MUST be aborted and the normal convergence procedures SHOULD be followed.

Consider a sample network as shown above with S->E and S1->R1 belonging to same SRLG group. The symmetric link metrics are shown in the figure and the SRGB is 1000-2000 on all nodes. When the S->E link goes down, all the links belonging to the same SRLG are considered to be down and the route is modified to carry multiple node-sids along the path.

SRGB:1000-2000 SID:1 SID:2 SID:3 SID:4 SID:5 +----+ 10 +----+ 10 +----+ 10 +----+ 10 +----+ | S1 |-------| R1 |----| S |-------| E |----| D1 | +----+ SRLG=5+----+ +----+ SRLG=5+----+ +----+ \ \ / 10 \ \ 10 / 100 \ SRGB:1000-2000 \ / \ +----+ +----+ +--| R2 |---------| R3 |SID:7 SID:6 +----+ 10 +----+SRGB:1000-2000 / / 10 +----+ | S2 |SID:8 +----+SRGB:1000-2000 when the S->E link goes down, S and E generate the link down event, update their Router-LSA/ LSP and flood the updated information across the IGP domain. The nodes in the IGP domain process the link-down event for affected destinations.If there are any other links with same SRLG on the path to destination, the nearest PLRs for those links are identified. In this example topology S1->R1 and S->E belong to same SRLG. For destination D1, R2 identifies two PLRs S1 and S for the S->E link down event. The nodes build the tunnelled path having multiple labels for each of the identified links. for ex, R2 builds a stack containing node-sid of S1 and S. The tunnelled path at R2 looks as shown in below.

When a network event is received, if the the change causes only one of the ECMP paths to change, then the micro-loop prevention mechanisms described in sec 3.1 and 3.2 are applied to the changed path only. As described in section 3.1 and 3.2 , if there is an ECMP path to the nearest PLR, then all ECMP paths are used to tunnel the traffic during convergence.

When a link goes down, both the ends of the link report the event by updating their LSP/LSA and flood it across the IGP domain. It is possible that the same network event being reported by two nodes is perceived as two different network events by the nodes in the IGP domain. The nodes processing the network events SHOULD evaluate if the received multiple events correspond to a single event by comparing the both ends of the reported link and also by looking at the previous event for which micro-loop prevention is being performed. If the event is same then micro-loop prevention procedures MUST be allowed to continue and MUST NOT be aborted. Node down or new node addition events are reported by removing a link or adding a new link by all the adjacent nodes. In addition Node up event also comprises of a new LSA advertisement. The criteria to recognize if the event is same is to look at both ends of the changed link. If one end of the changed link maps to previously reported events and the other end of the link (advertising router) changes for each successive event, then the event is SHOULD be recognized as a new node addition or a node deletion. Micro-loop procedures MUST be allowed to continue and MUST NOT be aborted.

The micro-loop mechanisms described in this document, are very effective and safe when all the nodes in the network support this feature and apply it when a network event happens. However, in some topologies, when all the nodes do not support the micro-loop prevention mechanism, the time duration of the loop can increase when only some nodes apply the procedures described in this document and some nodes do not. For example, consider the sample topology described in the figure below.

+-----+ | S3 | +-----+ / / +----+ 10 +----+ 10 +----+ 10 +----+ 10 +----+ | S1 |----| R1 |----| S |-------| E |----| D1 | +----+ +----+ +----+ +----+ +----+ \ \ / \ 10 \ 100 / 60 \ \ / \ +----+ +----+ +--| R2 |---------| R3 | +----+ 30 +----+ / / 10 +----+ | S2 | +----+ In this topology, S1, S2, and S3 are traffic sources and D1 is the destination. For each of the sources, shows the path before the failure (the before path) and the path after the failure (the post convergence path)..

+----+------+-------------------------+-----------------------------+ | Sr | Dest | Original Path | Post-Convergence Path | | c | | | | +----+------+-------------------------+-----------------------------+ | S1 | D1 | S1->R1->S->E->D1 | S1->R2->R3->E->D1 | +----+------+-------------------------+-----------------------------+ | S2 | D1 | S2->R2->S1->R1->S->E->D1| S2->R2->R3->E->D1 | +----+------+-------------------------+-----------------------------+ | S3 | D1 | S3->S->E->D1 | S3->S->R1->S1->R2->R3->E->D1| +----+------+-------------------------+-----------------------------+ In the above topology, if the PLR S does not support the micro-loop prevention mechanism but all other nodes support and apply this mechanism, then there is a possibility that the duration of traffic looping is higher than when the micro-loop prevention mechanisms are not applied at all. To mitigate this issue, protocol extensions to negotiate the support of this feature in the IGP domain is needed. describes the protocol mechanisms to advertise the support of this feature in OSPF and ISIS. However, in certain deployments and topologies, it MAY be safe to apply the micro-loop prevention procedures even when all the nodes in the network do not support this feature, especially in topologies where the post convergence path from PLR does not traverse the nodes in P space of the PLR with respect to the the node or link being protected.