Skip to main content

Configuring Dual Router High Availability using iBGP

The SSR provides significant flexibility for high availability configurations. Like traditional routers, the SSR software can be deployed as a single router instance on multiple platforms, with high availability configured in a dual router configuration. Additionally, the SSR can deploy multiple software instances (referred to as nodes) within the same single installation, providing high availability across router nodes.

The release of the 5.4 software includes VRRP as a configuration option as well as a new service route parameter, enable-failover, that provides stateful failover on the dual node HA configuration. For configuration options using VRRP, please refer to Configuring Dual Router High Availability and VRRP

Dual Router High Availability​

This document covers dual router high availability using iBGP - two instances of the SSR software each configured as separate routers. This is characterized by:

  • An iBGP interface shared between the two devices in lieu of a "fabric" interface in the dual node deployment.
  • No shared-phys-address (and hence no shared interfaces) between the two devices. Interface protection in a dual router HA deployment is accomplished using traditional routing protocols (Layer 3) rather than IP/MAC takeover (Layer 2).
  • No state synchronization between the two devices (and hence no "HA link"). While this improves overall performance for the routers (since there is no overhead incurred due to state synchronization), the implication is that there are some capabilities not supported in this design. See Unsupported Features, below.

Dual router high availability is recommended for data center designs where there will be a large volume of traffic; the elimination of shared state synchronization yields a simpler design, benefitting critical infrastructure.

Unsupported Features​

When deploying two nodes in a dual router high availability deployment, several features that rely on synchronized state between nodes are no longer available.

  • Source NAT. When source NAT is enabled, a system will allocate ephemeral ports on its egress interface as sessions leave the SSR router. Because there is no state synchronization between the two nodes in this deployment model, these ephemeral ports are not shared. Thus, if the active node fails and traffic starts transiting its counterpart, the source NAT allocation on the newly active system will be different. This will impact application flows.
  • Shared interfaces. A router node cannot perform gratuitous ARP takeover of an interface from another, distinct router node.

Design Constraints​

Due to the way that dual router high availability operates without state synchronization, in the event of a failure to one router in a pair, all traffic will be sent to the counterpart router (once routing converges). The now-active router does not have a shared database to reconstruct session state; it will receive mid-session packets (from the client's and server's perspectives) and need to set up forwarding state.

For this reason, services that leverage a dual router HA pair must reference a service-policy that has transport-state-enforcement allow configured for TCP packets. Otherwise, mid-session TCP packets cause the SSR device to send a TCP RST to the sender.

Design Overview​

Dual router high availability uses an inter-router iBGP connection to forward packets.

note

It is possible to configure multiple inter-router connections for added resiliency if there are spare physical connections available between the two routers.

The sample high level topology we will discuss in this document is as follows:

dual-router-ha-diagrams

Notes about the Topology​

In this sample configuration, each of the two routers (routerA and routerB) have two WAN interfaces and one LAN interface. Between them is an interconnect cable; for collocated routers, this is a simple "crossover" cable between the two systems.

Routing Overview​

Routing within the dual router redundancy model must be accomplished "by hand;" i.e., discretely on each individual system. This consists of two components:

  1. Each router uses BGPoSVR to exchange routes with the other.
  2. For services that use peer-type service-routes to reach another SSR instance, these service-routes will need to include the complementary router as an additional next-peer. Meaning each router in the HA pair will point to the next-peer SSR as well as a next-peer for one another.

Sample Configuration​

Each of the two routers that comprise the highly available pair will be configured similarly. Here is our routerA:

admin@node1.bernstein# show config runn auth router routerA

config

    authority
    
        router  routerA
            name                 routerA
            router-group         headend-datacenter1
            inter-node-security  internal
    
            node                 node1
                name                  node1
                asset-id              797b06cb-2187-406a-a933-4e93fc991fb0
                description           "routerA standalone node"
                role                  combo
                forwarding-core-mode  automatic
    
                device-interface      eno1
                    name               eno1
                    type               ethernet
                    pci-address        0000:00:01.0
                    forwarding         true
    
                    network-interface  interrouter
                        name                   interrouter
                        global-id              10
    
                        neighborhood           dc1-interrouter
                            name               dc1-interrouter
                            peer-connectivity  bidirectional
                            topology           mesh
                        exit
                        inter-router-security  unencrypted
    
                        address                169.254.1.28
                            ip-address     169.254.1.28
                            prefix-length  31
                        exit
                    exit
                exit
    
                device-interface      eno2
                    name               eno2
                    description        "LAN interface"
                    type               ethernet
                    pci-address        0000:00:02.0
                    forwarding         true
    
                    network-interface  lan
                        name         lan
                        global-id    11
                        description  "Data center LAN"
                        vlan         0
    
                        address      10.0.128.2
                            ip-address     10.0.128.2
                            prefix-length  24
                        exit
                    exit
                exit
    
                device-interface      eno3
                    name               eno3
                    description        "WAN interface 1"
                    type               ethernet
                    pci-address        0000:00:03.0
                    forwarding         true
    
                    network-interface  wan1
                        name         wan1
                        global-id    12
                        description  "WAN 1"
                        vlan         0
    
                        address      198.51.100.2
                            ip-address     198.51.100.2
                            prefix-length  24
                        exit
                    exit
                exit
    
                device-interface      eno4
                    name               eno4
                    description        "WAN interface 2"
                    type               ethernet
                    pci-address        0000:00:04.0
                    forwarding         true
    
                    network-interface  wan2
                        name         wan2
                        global-id    13
                        description  "WAN 2"
                        vlan         0
    
                        address      203.0.113.2
                            ip-address     203.0.113.2
                            prefix-length  24
                        exit
                    exit
                exit
            exit
    
            routing              default-instance
                type              default-instance
    
                interface         lo1
                    name        lo1
                    ip-address  169.254.2.1
                exit
    
                routing-protocol  bgp
                    description   "Dual router HA iBGP"
                    type       bgp
                    local-as   64512
                    router-id  169.254.2.1
    
                    neighbor   169.254.2.2
                        neighbor-address  169.254.2.2
                        neighbor-as       64512
                        shutdown          false
    
                        timers
                            connect-retry                   30
                            minimum-advertisement-interval  30
                        exit
    
                        transport
                            passive-mode   false
    
                            local-address
                                routing-interface  lo1
                            exit
                        exit
    
                        address-family    ipv4-unicast
                            afi-safi       ipv4-unicast
                            next-hop-self  true
                        exit
                    exit
                exit
            exit
        exit
    exit
exit

Routing Configuration​

The dual router high availability design does not synchronize state between routers. Instead, the two devices exchange reachability information using iBGP. This is implemented on the SSR using BGP over SVR (BGPoSVR), as shown in the sample configuration.

In our sample configuration we use device-interface eno1 as our iBGP link. The sample uses link-local IP addresses, presuming that the two nodes are located next to one another in the same data center. The neighborhood dc1-interrouter configuration is provisioned on routerB, and indicates to conductor that the two devices are mutually reachable. These pieces combined with the loopback interfaces are what creates the peering relationship, the services, and the service-routes in support of BGPoSVR.

The iBGP also interacts with other routing protocols to exchange reachability information with one another. For example, you may redistribute ospf into this iBGP, so that each device is aware of the other's reachability, as shown here:

                routing-protocol  bgp
                    description   "Dual router HA iBGP"
                    type          bgp
                    local-as      64512
                    router-id     169.254.2.1

                    neighbor      169.254.2.2
                        neighbor-address  169.254.2.2
                        neighbor-as       64512
                        shutdown          false

                        timers
                            connect-retry                   30
                            minimum-advertisement-interval  30
                        exit

                        transport
                            passive-mode   false

                            local-address
                                routing-interface  lo1
                            exit
                        exit

                        address-family    ipv4-unicast
                            afi-safi       ipv4-unicast
                            next-hop-self  true
                        exit
                    exit

                    redistribute  ospf
                        protocol  ospf
                    exit
                exit

Service Policy Configuration​

As mentioned in the design constraints section, due to the fact that there is no shared state between the routers in a dual router HA deployment, when a router fails its counterpart will pick up all traffic for active sessions as soon as routing converges. This will result in mid-flow TCP packets arriving at the new router, and the default behavior for the SSR router is to reject all mid-flow TCP packets by sending a TCP RST back to the sender.

This behavior is governed by the transport-state-enforcement field in the service-policy. Any service traffic that is conveyed to or from a system that is configured as a dual router HA deployment must have transport-state-enforcement set to allow, or else all TCP-based traffic will need to be restarted post-switchover. Below is an example service-policy showing the transport-state-enforcement set to allow:

admin@labsystem1.fiedler# show config running authority service-policy data-mission-critical

config

    authority

        service-policy  data-mission-critical
            name                         data-mission-critical
            description                  "Mission-critical data"
            service-class                MultimediaStreaming
            path-quality-filter          true
            best-effort                  true
            max-latency                  250
            transport-state-enforcement  allow
        exit
    exit
exit

For more information on this setting, refer to the section on transport-state-enforcement under service-policy.

Service Route Configuration​

There are two considerations when constructing service-route configuration in a dual router HA environment:

  • Routing traffic from a router to a remote set of routers configured for dual router HA
  • Routing traffic from the dual router HA to other peers

In both of these cases, the service-route will use the next-peer method.

Routing to a Dual Router HA​

Below is an example service-route that would typify a branch router's path to a data center dual router HA pair:

            service-route         rte-datacenter-service-1
                name          rte-datacenter-service-1
                service-name  service-1
                next-peer     routerA
                next-peer     routerB
            exit

In this example, the branch router's traffic matching service-1 will have two possible peers it can leverage, routerA and routerB.

note

While it is possible to manually configure these yourself, the conductor will typically autogenerate these types of service-route elements on your behalf.

Routing from a Dual Router HA​

Below is an example service-route that typifies a dual router HA system sending traffic to a branch site (using a summary service):

            service-route         rte-branch1-summary
                name          rte-branch1-summary
                service-name  branch1-summary
                next-peer     branch1
                next-peer     routerB
            exit

This example would be found on routerA; note that while we still use next-peer, the hops are different. First is branch1, the target of our summary service. Second, however, is routerB. This is effectively a manually-created dual router HA version of a dual node HA's "fabric" path: routerA has a path to reach branch1 via routerB.

There will also be a corresponding service-route on routerB that looks slightly different:

            service-route         rte-branch1-summary
                name          rte-branch1-summary
                service-name  branch1-summary
                next-peer     branch1
                next-peer     routerA
            exit

Here we see that routerB will "hop through" routerA to get to branch1, if its direct path to branch1 is down. (I.e., all peer paths are unavailable.)

note

These service-route configuration elements are not be built by the conductor, and must be manually created. (They are typically part of provisioning templates. As new routers are deployed the configuration templates will include these service-route elements for the new peers.)