Cisco | IPNET - Part 3

Overlay MPLS over provider backbone

Imagine that you have two or more sites which you want to connect together using MPLS technology. You cannot afford dark fiber and your Service Provider cannot offer you L2 connections of any kind. The only thing your SP can offer is L3 transport. Still, you want to build your own MPLS environment and there is no way to convince your SP to enable CsC.

I will use the following topology to demonstrate one of the possibilities to build an overlay MPLS over SP backbone.

The provider has already his MPLS backbone configured (P1, PE1, PE2 and PE3) and it’s offering you, as customer, IP transport over it’s backbone. Usually, from customer perspective, you don’t get to see the SP backbone, but just for reference, it is using ISIS for IGP, MP-BGP and MPLS VPN to transport our prefixes.

From IP prefixes allocation I’m using “xy” in the third octet (x – lower router number, y – higher router number) and “z” in the last octet (router number) with a /24 mask.

We have three locations named CPE1, CPE2 and CPE3. Currently between my CPE and provider PE I have enabled BGP, but you can use any protocol (even static) if your SP is able to route your IP prefixes over its backbone. On each CPE device I have a Loopback interface and its IP address will be the only prefix you announce (through BGP in this demonstration) to SP.

Let’s establish the BGP connection from our CPE to SP PE. As I’m playing the role of customer here, only the CPE exhibits will be shown:

CPE5

interface Loopback0
 ip address 5.5.5.5 255.255.255.255
!
router bgp 65001
 bgp router-id 5.5.5.5
 bgp log-neighbor-changes
 neighbor 10.0.35.3 remote-as 65000
 neighbor 10.0.35.3 description R3PE3
 neighbor 10.0.35.3 timers 5 20
 !
 address-family ipv4
  neighbor 10.0.35.3 activate
  no auto-summary
  no synchronization
  network 5.5.5.5 mask 255.255.255.255
 exit-address-family

CPE6

interface Loopback0
 ip address 6.6.6.6 255.255.255.255
!
router bgp 65001
 bgp router-id 6.6.6.6
 bgp log-neighbor-changes
 neighbor 10.0.26.2 remote-as 65000
 neighbor 10.0.26.2 description R2PE2
 neighbor 10.0.26.2 timers 5 20
 !
 address-family ipv4
  neighbor 10.0.26.2 activate
  no auto-summary
  no synchronization
  network 6.6.6.6 mask 255.255.255.255
 exit-address-family

CPE7

interface Loopback0
 ip address 7.7.7.7 255.255.255.255
!
router bgp 65001
 bgp router-id 7.7.7.7
 bgp log-neighbor-changes
 neighbor 10.0.47.4 remote-as 65000
 neighbor 10.0.47.4 description R4PE4
 neighbor 10.0.47.4 timers 5 20
 !
 address-family ipv4
  neighbor 10.0.47.4 activate
  no auto-summary
  no synchronization
  network 7.7.7.7 mask 255.255.255.255
 exit-address-family

BGP neighborship suppose to be up now and on each CPE I should receive the Loopback prefixes of the other two CPE devices.

R5CPE5#sh ip bgp sum | b Nei
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.35.3       4 65000     126     125        2    0    0 00:10:04        0
!
R6CPE6#sh ip bgp sum | b Nei
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.26.2       4 65000     136     135        2    0    0 00:10:50        0
!
R7CPE7#sh ip bgp sum | b Nei
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.0.47.4       4 65000     134     134        4    0    0 00:10:46        0

BGP neighborship is up alright, but where are my prefixes? State/PfxRcd is 0, when it should show 2.
I did that on purpose.

Notice that we are using the same AS number on all our sites. I think you already know that the rule in BGP is that if our own ASN is seen in the AS-Path of a particular IP prefix, BGP will not install that prefix in BGP table. This is fixable:
1. We ask our provider to have a little “as-override” command in its BGP configuration for our neighbor
2. We use different ASN on each site (assuming that we are using private ASN)
3. We configure “allowas-in” on BGP neighborship with SP

Basically you may use any of the three methods (or other if you can think of any other), but in my case I don’t want to ask the SP nor I want to change my ASN scheme. I’ll go with the third option and be careful not run into loop issues (consider this is an Enterprise environment I think it’s doable).

CPE5

router bgp 65001
  neighbor 10.0.35.3 allowas-in

CPE6

router bgp 65001
  neighbor 10.0.26.2 allowas-in

CPE7

router bgp 65001
  neighbor 10.0.47.4 allowas-in

Let’s check again and do some testing. I will use CPE5

R5CPE5#sh ip route bgp
     6.0.0.0/32 is subnetted, 1 subnets
B       6.6.6.6 [20/0] via 10.0.35.3, 00:04:02
     7.0.0.0/32 is subnetted, 1 subnets
B       7.7.7.7 [20/0] via 10.0.35.3, 00:04:02
!
R5CPE5#ping 6.6.6.6 source 5.5.5.5
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 6.6.6.6, timeout is 2 seconds:
Packet sent with a source address of 5.5.5.5 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/44/64 ms
 
R5CPE5#ping 7.7.7.7 source 5.5.5.5
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 7.7.7.7, timeout is 2 seconds:
Packet sent with a source address of 5.5.5.5 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 28/52/96 ms

Next part involves the creation of Tunnels interfaces, to have a full mesh connection between the three sites, enable of IGP / MPLS and creation of a second Loopback interface which we will use later for the iBGP configuration. I did chose IS-IS for IGP and LDP for MPLS. The new Loopback interface will be routed using the IS-IS protocol.

A note from my side. Since I had limited number of routers, my CPE devices will be kind of P / PE / CE router in my overlay MPLS demonstration.

CPE5

int Tun56
tunnel source lo0
tunnel destination 6.6.6.6
ip address 192.168.56.5 255.255.255.0
mpls ip
ip router isis
!
int Tun57
tunnel source lo0
tunnel destination 7.7.7.7
ip address 192.168.57.5 255.255.255.0
mpls ip
ip router isis
!
int Lo1
ip address 55.55.55.55 255.255.255.255
!
router isis
net 47.0005.0005.0005.0005.00
passive-interface lo1
is-type level-2-only

CPE6

int Tun56
tunnel source lo0
tunnel destination 5.5.5.5
ip address 192.168.56.6 255.255.255.0
mpls ip
ip router isis
!
int Tun67
tunnel source lo0
tunnel destination 7.7.7.7
ip address 192.168.67.6 255.255.255.0
mpls ip
ip router isis
!
int Lo1
ip address 66.66.66.66 255.255.255.255
!
router isis
net 47.0006.0006.0006.0006.00
passive-interface lo1
is-type level-2-only

CPE7

int Tun57
tunnel source lo0
tunnel destination 5.5.5.5
ip address 192.168.57.7 255.255.255.0
mpls ip
ip router isis
!
int Tun67
tunnel source lo0
tunnel destination 6.6.6.6
ip address 192.168.67.7 255.255.255.0
mpls ip
ip router isis
!
int Lo1
ip address 77.77.77.77 255.255.255.255
!
router isis
net 47.0007.0007.0007.0007.00
is-type level-2-only
passive-interface lo1

I will use CPE5 for some show commands output and to check that everything is running fine:

R5CPE5#ping 192.168.56.6
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.56.6, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 44/48/52 ms
 
R5CPE5#ping 192.168.57.7
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.57.7, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 32/40/52 ms
 
!
R5CPE5#sh isis topology
 
IS-IS paths to level-2 routers
System Id            Metric     Next-Hop             Interface   SNPA
R5CPE5               --
R6CPE6               10         R6CPE6               Tu56        *Tunnel*       
R7CPE7               10         R7CPE7               Tu57        *Tunnel* 
!
R5CPE5#show mpls interfaces 
Interface              IP            Tunnel   Operational
Tunnel56               Yes (ldp)     No       Yes         
Tunnel57               Yes (ldp)     No       Yes 
!
R5CPE5#show mpls ldp neighbor 
    Peer LDP Ident: 6.6.6.6:0; Local LDP Ident 5.5.5.5:0
	TCP connection: 6.6.6.6.64820 - 5.5.5.5.646
	State: Oper; Msgs sent/rcvd: 16/16; Downstream
	Up time: 00:07:47
	LDP discovery sources:
	  Tunnel56, Src IP addr: 192.168.56.6
        Addresses bound to peer LDP Ident:
          10.0.26.6       6.6.6.6         192.168.56.6    192.168.67.6    
    Peer LDP Ident: 7.7.7.7:0; Local LDP Ident 5.5.5.5:0
	TCP connection: 7.7.7.7.11545 - 5.5.5.5.646
	State: Oper; Msgs sent/rcvd: 16/16; Downstream
	Up time: 00:07:26
	LDP discovery sources:
	  Tunnel57, Src IP addr: 192.168.57.7
        Addresses bound to peer LDP Ident:
          10.0.47.7       7.7.7.7         192.168.57.7    192.168.67.7

I will create now two VRF instances as I want to separate Financial department traffic from the Technical one.

On all three CPE devices:

ip vrf FIN
rd 65001:1
route-target import 65001:1
route-target export 65001:1
!
ip vrf TEK
rd 65001:2
route-target import 65001:2
route-target export 65001:2

Configuration of iBGP depends on the previous Loopback1 interfaces, so be sure that you have that interfaces reachable through IS-IS. Because it’s an iBGP and I don’t want to type the same command over and over, I will use peer-groups.

CPE5

router bgp 65001
!
neighbor OM peer-group
neighbor OM remote-as 65001
neighbor OM timers 5 20
neighbor OM description Overlay-MPLS
neighbor OM update-source lo0
!
address-family vpnv4
neighbor 66.66.66.66 peer-group OM
neighbor 77.77.77.77 peer-group OM

CPE6

router bgp 65001
neighbor OM peer-group
neighbor OM remote-as 65001
neighbor OM timers 5 20
neighbor OM description Overlay-MPLS
neighbor OM update-source lo1
!
address-family vpnv4
neighbor 55.55.55.55 peer-group OM
neighbor 77.77.77.77 peer-group OM

CPE7

router bgp 65001
neighbor OM peer-group
neighbor OM remote-as 65001
neighbor OM timers 5 20
neighbor OM description Overlay-MPLS
neighbor OM update-source lo1
!
address-family vpnv4
neighbor 55.55.55.55 peer-group OM
neighbor 66.66.66.66 peer-group OM

We should check that everything is up. I will use again CPE5:

R5CPE5#show ip bgp vpnv4 all sum | b Nei
Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
66.66.66.66     4 65001      46      46        1    0    0 00:03:30        0
77.77.77.77     4 65001      31      31        1    0    0 00:02:18        0

Finally we are getting somewhere. In real world you will have the CPE routers connected to downstream devices, using subinterfaces in particular VRF and so on. I’m short on devices, so I will use some additional Loopback interfaces and add them to VRF FIN and TEK for testing.

CPE5

int Lo51
ip vrf forwarding FIN
ip address 10.51.51.51 255.255.255.255
!
int Lo52
ip vrf forwarding TEK
ip address 10.52.52.52 255.255.255.255
!
router bgp 65001
!
address-family ipv4 vrf FIN
network 10.51.51.51 mask 255.255.255.255
!
address-family ipv4 vrf TEK
network 10.52.52.52 mask 255.255.255.255

CPE6

int Lo61
ip vrf forwarding FIN
ip address 10.61.61.61 255.255.255.255
!
int Lo62
ip vrf forwarding TEK
ip address 10.62.62.62 255.255.255.255
!
router bgp 65001
!
address-family ipv4 vrf FIN
network 10.61.61.61 mask 255.255.255.255
!
address-family ipv4 vrf TEK
network 10.62.62.62 mask 255.255.255.255

CPE7

int Lo71
ip vrf forwarding FIN
ip address 10.71.71.71 255.255.255.255
!
int Lo72
ip vrf forwarding TEK
ip address 10.72.72.72 255.255.255.255
!
router bgp 65001
!
address-family ipv4 vrf FIN
network 10.71.71.71 mask 255.255.255.255
!
address-family ipv4 vrf TEK
network 10.72.72.72 mask 255.255.255.255

To check if everything is working fine, I will use CPE5 for some tests:

R5CPE5#sh ip route vrf FIN | b Ga
Gateway of last resort is not set
 
     10.0.0.0/32 is subnetted, 3 subnets
B       10.61.61.61 [200/0] via 66.66.66.66, 00:04:42
C       10.51.51.51 is directly connected, Loopback51
B       10.71.71.71 [200/0] via 77.77.77.77, 00:02:46
!
R5CPE5#sh ip route vrf TEK | b Ga
Gateway of last resort is not set
 
     10.0.0.0/32 is subnetted, 3 subnets
B       10.62.62.62 [200/0] via 66.66.66.66, 00:04:59
C       10.52.52.52 is directly connected, Loopback52
B       10.72.72.72 [200/0] via 77.77.77.77, 00:03:04
!
R5CPE5#ping vrf FIN 10.71.71.71 source 10.51.51.51
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.71.71.71, timeout is 2 seconds:
Packet sent with a source address of 10.51.51.51 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/44/48 ms
!
R5CPE5#ping vrf TEK 10.62.62.62 source 10.52.52.52
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.62.62.62, timeout is 2 seconds:
Packet sent with a source address of 10.52.52.52 
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 40/47/52 ms
!
R5CPE5#show mpls forwarding-table 
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop    
tag    tag or VC   or Tunnel Id      switched   interface              
16     Pop tag     192.168.67.0/24   0          Tu57       point2point  
       Pop tag     192.168.67.0/24   0          Tu56       point2point  
17     Pop tag     66.66.66.66/32    0          Tu56       point2point  
18     Pop tag     77.77.77.77/32    0          Tu57       point2point  
19     Aggregate   10.51.51.51/32[V] 1040                               
20     Aggregate   10.52.52.52/32[V] 520

You may wonder why somebody would put together such a complex configuration. There may be multiple reasons, beyond the scope of this example, but I would like to add MPLS TE, Encrypted Site-to-Site traffic with route manipulation, independent configuration from the SP, learning purposes and many more.

Can we encounter problems with this configuration? Well, yes.
If the provider has a very low MTU size support, you may get a lot of fragmentation. Also maintenance and operation of Tunnels may be tricky for a very large environment, but there are solutions to limit the number of tunnels. Still the benefits exit.

To enumerate one benefit from real world. Applying this configuration in Enterprise environment, you have the possibility to change your SP without too much of a dazzle as long as your new provider can transport the IP address of your primary Loopback interface. The rest, stays the same.

Please let me know if you have questions or if something in my explanation is wrong.

Cisco: IPv6 over IPv4 MPLS (6PE)

Let’s assume that you have an IPv4 core backbone on which you did enable MPLS. Your customers are connected to your PE routers and they need to have IPv6 reachability between their sites. To have things more clear, please have a look to the following network topology:

As you can see in the above image, customers have the IPv6 addresses 2001::1/128 and 2001::2/128. Actually in this topology, “my customers” are two Loopback addresses (Lo200) configured under PE routers, to minimize the used resources (routers).

We have a typical MPLS (LDP) construction with following characteristics:
– OSPF is used for IGP
– each router has a L0 interface advertised into IGP
– L0 of each router is configured as MPLS LDP router-id
– PE routers have an additional L100 interface advertised into IGP
– iBGP on the PE routers is configured using L100 interface as neighbour peer points

You can find below some output of iBGP and MPLS configuration:

PE1

router bgp 65300
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 22.22.22.22 remote-as 65300
 neighbor 22.22.22.22 update-source Loopback0
 neighbor 22.22.22.22 timers 1 4
 !
 address-family vpnv4
  neighbor 22.22.22.22 activate
  neighbor 22.22.22.22 send-community extended
 exit-address-family
!
!
PE1#sh bgp vpnv4 unicast all sum | i 65300
BGP router identifier 11.11.11.11, local AS number 65300
22.22.22.22     4 65300    7374    7376       19    0    0 00:04:11        1
!
!
PE1#show mpls forwarding-table 
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop    
tag    tag or VC   or Tunnel Id      switched   interface              
16     16          22.22.22.22/32    0          Fa0/0      10.10.11.1   
17     18          10.10.22.0/24     0          Fa0/0      10.10.11.1   
18     Pop tag     10.10.12.0/24     0          Fa0/0      10.10.11.1    
20     Pop tag     1.1.1.1/32        0          Fa0/0      10.10.11.1   
21     19          2.2.2.2/32        0          Fa0/0      10.10.11.1

PE2

router bgp 65300
 no bgp default ipv4-unicast
 bgp log-neighbor-changes
 neighbor 11.11.11.11 remote-as 65300
 neighbor 11.11.11.11 update-source Loopback0
 neighbor 11.11.11.11 timers 1 4
 !
 address-family vpnv4
  neighbor 11.11.11.11 activate
  neighbor 11.11.11.11 send-community extended
 exit-address-family
!
!
PE2#show bgp vpnv4 unicast all summary | i 65300
BGP router identifier 22.22.22.22, local AS number 65300
11.11.11.11     4 65300    7518    7516       19    0    0 00:06:33        1
!
!
PE2#show mpls forwarding-table 
Local  Outgoing    Prefix            Bytes tag  Outgoing   Next Hop    
tag    tag or VC   or Tunnel Id      switched   interface              
16     17          11.11.11.11/32    0          Fa0/0      10.10.22.1   
17     Pop tag     10.10.12.0/24     0          Fa0/0      10.10.22.1   
18     18          10.10.11.0/24     0          Fa0/0      10.10.22.1                                
20     19          1.1.1.1/32        0          Fa0/0      10.10.22.1   
21     Pop tag     2.2.2.2/32        0          Fa0/0      10.10.22.1

At this point we have a functional IPv4 MPLS. Now, let’s use the MPLS to transport IPv6 networks. First, on the PE routers we have to enable IPv6 routing support and CEF:

ipv6 unicast-routing
ipv6 cef

Next, I’ll have the L200 interface configured:

PE1#show ipv6 int lo200
Loopback200 is up, line protocol is up
  IPv6 is enabled, link-local address is FE80::C002:15FF:FE10:0 
  No Virtual link-local address(es):
  Global unicast address(es):
    2001::1, subnet is 2001::1/128 
  Joined group address(es):
    FF02::1
    FF02::2
    FF02::1:FF00:1
    FF02::1:FF10:0
  MTU is 1514 bytes
  ICMP error messages limited to one every 100 milliseconds
  ICMP redirects are enabled
  ICMP unreachables are sent
  ND DAD is not supported
  ND reachable time is 30000 milliseconds
  Hosts use stateless autoconfig for addresses.

PE2#sh ipv6 int lo200
Loopback200 is up, line protocol is up
  IPv6 is enabled, link-local address is FE80::C003:15FF:FE10:0 
  No Virtual link-local address(es):
  Global unicast address(es):
    2001::2, subnet is 2001::2/128 
  Joined group address(es):
    FF02::1
    FF02::2
    FF02::1:FF00:2
    FF02::1:FF10:0
  MTU is 1514 bytes
  ICMP error messages limited to one every 100 milliseconds
  ICMP redirects are enabled
  ICMP unreachables are sent
  ND DAD is not supported
  ND reachable time is 30000 milliseconds
  Hosts use stateless autoconfig for addresses.

Now let’s make the magic and configure the PE routers so that the IPv6 networks are reachable. We will make this by using the 6PE method. This solution include that the Provider edge routers (PE) are IPv6 capable and it enables IPv6 sites to communicate with each other over an MPLS IPv4 core network using MPLS label switched paths (LSPs). To make it clear the Provider routers (P) inside MPLS have not idea about any IPv6 addressing or routing.

The 6PE routers are configured to be dual-stack using IPv4 and IPv6, and use the IPv4 mapped IPv6 address for IPv6 prefix reachability exchange. The 6PE uses multiprotocol internal BGP (MP-iBGP) to distribute IPv6 routes over the MPLS IPv4 core network and to attach an MPLS label to each route.

Let’s enable this. The BGP configuration looks like this:

PE1#
router bgp 65300
 !
 address-family ipv6 
  neighbor 22.22.22.22 activate 
  neighbor 22.22.22.22 send-label 
  network 2001::1/128 
  no synchronization 
 exit-address-family

PE2#
router bgp 65300
 !
 address-family ipv6
  neighbor 11.11.11.11 activate
  neighbor 11.11.11.11 send-label
  network 2001::2/128
  no synchronization
 exit-address-family

I did enable the BGP neighbour under IPv6 address family using the IPv4 addresses. The IPv4 addresses are already know in MPLS using the IPv4 IGP protocol (OSPF). Very important here is the send-label option as it enables binding and advertisement of aggregate labels when advertising IPv6 prefixes in BGP.

Let’s check on the PE routers if we receive properly the IPv6 addresses:

PE1#show bgp ipv6 unicast 2001::2/128
BGP routing table entry for 2001::2/128, version 16
Paths: (1 available, best #1, table Global-IPv6-Table)
  Not advertised to any peer
  Local
    ::FFFF:22.22.22.22 (metric 31) from 22.22.22.22 (22.22.22.22)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      mpls labels in/out nolabel/22
!
!
PE1#show ipv6 route 2001::2/128 | b 2001::2
B   2001::2/128 [200/0]
     via ::FFFF:22.22.22.22, IPv6-mpls

PE2#show bgp ipv6 unicast 2001::1/128
BGP routing table entry for 2001::1/128, version 14
Paths: (1 available, best #1, table Global-IPv6-Table)
  Not advertised to any peer
  Local
    ::FFFF:11.11.11.11 (metric 31) from 11.11.11.11 (11.11.11.11)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      mpls labels in/out nolabel/23
!
!
PE2#show ipv6 route 2001::1/128 | b 2001
B   2001::1/128 [200/0]
     via ::FFFF:11.11.11.11, IPv6-mpls

And the last test:

PE2#ping ipv6 2001::1 source 2001::2
 
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 2001::1, timeout is 2 seconds:
Packet sent with a source address of 2001::2
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 12/17/24 ms

If you have questions or there is something which is not clear please let me know by using the Comments section below.

Cisco vs Juniper: different eBGP behavior

Last week I had to troubleshoot a problem about eBGP peering with an external provider and I think my findings will be interesting for some of you out there.

Let me start with some background information. I have two locations, same ASN, both connected to the same provider network using eBGP as routing protocol. Due to the looping prevention mechanism, default behavior in eBGP peering between Cisco equipment, is not to accept in BGP table prefixes that have in the AS-Path the same ASN as the local BGP router. You can still accept these prefixes, if you use the “allowas-in” trick on Cisco routers:

neighbor x.x.x.x allowas-in

The above command will tell BGP protocol on Cisco routers to ignore the presence of its own ASN in the AS-Path and to accept the prefixes. If all other attributes are in good standing, the routes to those prefixes will be installed in the routing table.

As a short example, take the following tolopogy:

I have a typical eBGP peering configuration. On one of the edge Cisco routers I have a Lo0 interface with 5.5.5.5/32 IP address which I advertise into BGP. Checking the middle Cisco router (provider) I can see that the BGP table has the 5.5.5.5/32 prefix installed:

sh ip bgp 5.5.5.5
BGP routing table entry for 5.5.5.5/32, version 2
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
  Advertised to update-groups:
        1
  65082
    10.10.12.1 from 10.10.12.1 (10.10.12.1)
      Origin IGP, metric 0, localpref 100, valid, external, best

Next I check if the provider router does advertise this prefix to the next neighbor:

sh ip bgp neigh 10.10.13.1 advertised-routes | i 5.5.5.5
*> 5.5.5.5/32       10.10.12.1               0             0 65082 i

From this perspective everything is fine. On the edge Cisco router, I cannot see this prefix in the BGP table, which is normal due to the reasons I explained above:

R6#sh ip bgp 5.5.5.5
% Network not in table

As soon as I add the “allowas-in” configuration on the edge Cisco router, the BGP table will contain the prefix 5.5.5.5/32:

router bgp 65082
 no synchronization
 bgp log-neighbor-changes
 neighbor 10.10.13.2 remote-as 65086
 neighbor 10.10.13.2 allowas-in
 no auto-summary

sh ip bgp 5.5.5.5    
BGP routing table entry for 5.5.5.5/32, version 2
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
  Not advertised to any peer
  65086 65082
    10.10.13.2 from 10.10.13.2 (10.10.13.2)
      Origin IGP, localpref 100, valid, external, best

Now let take the example when the provider is using Juniper technology. The eBGP peering is now between Cisco and Juniper devices:

I have configured the eBGP in the same way and I have advertised from one Cisco router the 5.5.5.5/32 prefix.

On Juniper the BGP configuration looks like this:

yotis@J1# run show configuration protocols        
bgp {
    local-as 65086;
    group TST {
        peer-as 65082;
        neighbor 10.10.12.1;
        neighbor 10.10.13.1;
    }
}

Checking on the Juniper device I can see the prefix in the BGP table and routing table:

yotis@J1# run show route protocol bgp receive-protocol bgp 10.10.12.1    
 
inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
  Prefix		  Nexthop	       MED     Lclpref    AS path
* 5.5.5.5/32              10.10.12.1           0                  65082 I
 
__juniper_private2__.inet.0: 1 destinations, 1 routes (0 active, 0 holddown, 1 hidden)

yotis@J1# run show route 5.5.5.5                                         
 
inet.0: 7 destinations, 7 routes (7 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both
 
5.5.5.5/32         *[BGP/170] 01:19:15, MED 0, localpref 100
                      AS path: 65082 I
                    &gt; to 10.10.12.1 via em1.0

On the second Cisco router, I did not change anything. The “allowas-in” command is still there:

router bgp 65082
 no synchronization
 bgp log-neighbor-changes
 neighbor 10.10.13.2 remote-as 65086
 neighbor 10.10.13.2 allowas-in
 no auto-summary

Still the prefix is not in the BGP table:

R6#show ip bgp 5.5.5.5
% Network not in table

Here is a big difference on the way eBGP protocol behave on Cisco vs. Juniper. On Cisco devices, the eBGP will send the prefixes, no matter of the peer ASN and it’s the task of the peer to apply the default policy when its own ASN is present in the AS-Path and deny the prefixes to be installed in the BGP table.

With Juniper, eBGP protocol will not even announce the prefixes if the ASN of its peer is already present in the AS-Path.

According to http://www.juniper.net/techpubs/software/junos/junos94/swconfig-routing/disabling-suppression-of-route-advertisements.html#id-13255463

“The JUNOS software does not advertise the routes learned from one external BGP (EBGP) peer back to the same EBGP peer. In addition, the software does not advertise those routes back to any EBGP peers that are in the same AS as the originating peer, regardless of the routing instance.”

There is a solution to overcome this behavior and it’s pretty simple. You can set this configuration on global basis or per group in Junos BGP configuration:

[edit protocols bgp group TST]
yotis@J1# set advertise-peer-as

Now the BGP configuration looks like this:

yotis@J1# run show configuration protocols        
bgp {
    local-as 65086;
    group TST {
        advertise-peer-as;
        peer-as 65082;
        neighbor 10.10.12.1;
        neighbor 10.10.13.1;
    }
}

If I check now the Cisco BGP table, the 5.5.5.5/32 prefix will be there:

show ip bgp 5.5.5.5
BGP routing table entry for 5.5.5.5/32, version 4
Paths: (1 available, best #1, table Default-IP-Routing-Table)
Flag: 0x820
  Not advertised to any peer
  65086 65082
    10.10.13.2 from 10.10.13.2 (10.10.12.2)
      Origin IGP, localpref 100, valid, external, best

Don’t forget to check my Juniper, first steps after power-on the device if you are new with Juniper products. You may be interested also in my older posts Cisco: BGP path selection for outgoing traffic and Cisco: BGP path selection for inbound traffic if you just start to work with external providers.

[Cisco Live] SDN controller interview

The original name of this video is “SDN controller DEMO”. I think the “demo” word there is a bit inappropriate used, as actually is more like a Cisco marketing video than demo. Don’t be so surprised, you know how Cisco promote their products.

Just my 2 cents about the SDN/OpenFlow trend that is coming up these days. I don’t mind innovation, I’m glad if and when I can get in contact with new technologies, but what disturb me is that in a lot of presentations that I’ve seen until now, SDN is presented like the magic wand that does everything with point and click.
No knowledge needed, no network understanding, no effort to see where or how the packets travel through network, you just have to point and click, slice the network however you want, plug the toy and you’re ready to go. If this is all true, then someone please explain to me with I’m busting my…head to learn and really understand what’s actually going on in the network. I really hope this kind of presentations implies that this is a new product that needs to be promoted and if they make it sound too complex, nobody will buy it.

Disclaimer: This video is not mine and I don’t claim any rights on it. My thanks go to Jimmy Ray Purse, TechWiseTV, Networking 101 Show, Cisco and last but not least to YouTube for hosting it and let us embed this video.

Selective BGP Dampening and parameters tuning

Some time ago, I wrote about BGP Dampening and how this feature can improve the stability of the network. A lot happened since then and during my experience with different service providers I have seen that BGP dampening can help in the same measure at it can harm your network. An endless discussion can be started on this topic, but this is not what I want to do here.

One thing that I did learn is that fine tuning of any feature can help a lot in some cases making the difference between stable network and a total disaster. In regard to BGP dampening, I have the following scenario. Imagine that you would like to use BGP dampening, but only for some networks, which are proven to be more stable than others. I will base my example on the following scenario:

Task

Router 1 in the above scenario has three networks that are advertised into BGP:
L0 – 1.1.1.0 /24
L1 – 11.11.11.0 /24
L2 – 111.111.111.0 /24
Imagine this are being remote networks and that 11.11.11.0 /24 is very unstable. To simulate an unstable network that triggers BGP, shut / no shut multiple times.

On Router 2, we want to use the BGP Dampening feature, but only for this network. We know already that we can do something like:

conf t
router bgp 200
bgp dampening

This will enable the BGP dampening feature. We can even fine tune some parameters like:

conf t
router bgp 200
bgp dampening 15 750 5000 30

In this way we increase the limit at which a route will be dampened (5000) and decrease the maximum dampening time to 30 minutes. Unfortunately this parameters are applied globally and all routes (stable and unstable) will play by this rules.

Solution

Going back to the idea of this post, use of selective BGP Dampening, we can configure Router 2 like this:

conf t
access-list 11 permit 11.11.11.0 0.0.0.255
!
route-map DAM permit 10
match ip address 11
set dampening 15 750 2000 60
!
route-map DAM deny 1000

What we just did is to match the unstable prefix in an ACL. Use the ACL in a route-map with permit policy. Mandatory set the dampening parameters. They can be the same as original values, but if you don’t set anything here, you will meet the following error when trying to apply the BGP dampening.

%BGP-3-BADROUTEMAP: Bad parameters in the route-map DAM applied for Dampening

At the end we have a deny policy in the same route-map to avoid matching any other prefixes. We can not apply it to BGP:

conf t
router bgp 200
bgp dampening route-map DAM

We want to check that BGP Dampening feature is activated:

R2#sh ip bgp dampening parameters
 dampening 15 750 2000 60 (route-map DAM 10)
  Half-life time      : 15 mins       Decay Time       : 2320 secs
  Max suppress penalty: 12000         Max suppress time: 60 mins
  Suppress penalty    :  2000         Reuse penalty    : 750

By the way, if you check the output immediately after applying the BGP dampening feature, you might see the following error:

% dampening reconfiguration in progress for IPv4 Unicast

Verification

Let us see if there are any flaps on going:

R2#sh ip bgp dampening flap-statistics 
 
R2#

Now we can shut / no shut L1 interface on R1 to create an instability of this network. After doing so couple of times we can see that the BGP dampening is active:

R2#sh ip bgp dampening flap-statistics | b Net
   Network          From            Flaps Duration Reuse    Path
 h 11.11.11.0/24    10.1.12.1       1     00:00:22          100

If we continue to play with shut / no shut, soon we will see that 11.11.11.0 /24 is marked as dampened:

R2#sh ip bgp dampening dampened-paths | b Net
   Network          From             Reuse    Path
*d 11.11.11.0/24    10.1.12.1        00:06:29 100 i

Now I want to prove that the same BGP dampening policies does NOT apply to other networks like 111.111.111.0 /24. I will try to play the same shut / no shut game with L2 on R1. After 5 minutes of this game I can see the following:

R2#sh ip bgp dampening dampened-paths | b Net
   Network          From             Reuse    Path
*d 11.11.11.0/24    10.1.12.1        00:02:09 100 i
 
R2#sh ip bgp dampening flap-statistics | b Net
   Network          From            Flaps Duration Reuse    Path
*d 11.11.11.0/24    10.1.12.1       3     00:07:51 00:01:49 100 
 
R2#sh ip bgp | b Net  
   Network          Next Hop            Metric LocPrf Weight Path
*> 2.2.2.0/24       0.0.0.0                  0         32768 i
*> 3.3.3.0/24       10.1.23.3                0             0 300 i
*d 11.11.11.0/24    10.1.12.1                0             0 100 i
*> 111.111.111.0/24 10.1.12.1                0             0 100 i

You can see that 111.111.111.0 /24 does not appear in any dampening report.

I tried this in multiple scenarios and every time I got the expected result. If you test this and get different results, please let me know in comments and we can discuss.